Archive | Apache

Get Top 10 User Agents

By admin on May 14, 2024 in Apache, Bash Scripting

Extract a list of the top user agents from an Apache format log file and sort in descending order by number of occurrences:

awk -F\" '{print $6}' access_log | sort | uniq -c | sort -r -n | head -20

1	awk -F\" '{print $6}' access_log \| sort \| uniq -c \| sort -r -n \| head -20

Results similar to this:

  94786 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.75 Safari/537.36
  68390 Amazon CloudFront
  36131 Mozilla/5.0 (Windows NT 6.2; WOW64; rv:17.0) Gecko/20100101 Firefox/17.0
  19725 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36
  11865 Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/116.0.1938.76 Safari/537.36
  11428 Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html)
  10056 Mozilla/5.0 (compatible; AhrefsBot/7.0; +http://ahrefs.com/robot/)
   6311 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
....

94786 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.75 Safari/537.36

68390 Amazon CloudFront

36131 Mozilla/5.0 (Windows NT 6.2; WOW64; rv:17.0) Gecko/20100101 Firefox/17.0

19725 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36

11865 Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/116.0.1938.76 Safari/537.36

11428 Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html)

10056 Mozilla/5.0 (compatible; AhrefsBot/7.0; +http://ahrefs.com/robot/)

6311 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

....

Fixing “Size of a request header field exceeds server limit.” error

By admin on October 15, 2021 in Apache, Uncategorized

This error from Apache is typically caused by a very large cookie. In Apache the “LimitRequestFieldSize” directive sets this limit. The default is 8190 bytes. To change the file create a file at:

/etc/httpd/conf.d/custom.conf

/etc/apache2/conf.d/custom.conf

containing the desired change:

LimitRequestFieldSize 16384

1	LimitRequestFieldSize 16384

Reload Apache after adding the file with:

/sbin/service apache2 restart

/sbin/service httpd restart

Clearing PHP opcache

By admin on May 5, 2021 in Apache, PHP, Plesk

Great article on how to clear the PHP opcache:

https://ma.ttias.be/how-to-clear-php-opcache/

To facilitate use on a Plesk server created script and mapped into namespace:

https://acme.com/opcache_clear

Here’s the script:

<?php
echo "Clearing cache ... ";
opcache_reset();
echo "DONE!";
?>

<?php

echo "Clearing cache ... ";

opcache_reset();

echo "DONE!";

And here’s the Apache config to map into a virtual host:

Alias /opcache_clear /var/www/vhosts/acme.com/scripts/opcache_clear.php
<Directory /var/www/vhosts/acme.com/scripts>
  <Files ~ (\.php$)>
    SetHandler proxy:unix:///var/www/vhosts/system/acme.com/php-fpm.sock|fcgi://127.0.0.1:9000
  </Files>
  AuthType Basic
  AuthName " "
  AuthUserFile "/var/www/vhosts/system/acme.com/pd/d..httpdocs@opcache_clear"
  require valid-user
</Directory>

Alias /opcache_clear /var/www/vhosts/acme.com/scripts/opcache_clear.php

SetHandler proxy:unix:///var/www/vhosts/system/acme.com/php-fpm.sock|fcgi://127.0.0.1:9000

</Files>

AuthType Basic

AuthName " "

AuthUserFile "/var/www/vhosts/system/acme.com/pd/d..httpdocs@opcache_clear"

require valid-user

</Directory>

This above config should be placed in the “Apache & nginx setting” screen in the “Additional directives for https” field. In addition to setting the alias path we’re also setting the PHP handler.

Additionally under “Password protected directories” added password protection for “/opcache_clear”. This is just a placeholder for the password file which is referenced in the above config.

Apply Basic Auth To Site With Excluded Virtual Path Under Plesk With Apache 2.4

By admin on March 23, 2021 in Apache, Plesk

We want to password protect a WordPress development site but allow unauthenticated access to the wp-json/ path. Hosting platform is Plesk with Apache 2.4. We’ll assume the domain is “acme.com” and the assigned IP is “w.x.y.z”.

There are probably better ways to accomplish the goal but this approach seems to work.

Step 1. – In Plesk add a Protected Directory named “/donotremove” and add appropriate user/passwords.

Step 2. – In Plesk on the “Apache & nginx Setting” screen under “Additional nginx directives” add the following:

location ~ /wp-json {
	proxy_set_header AUTH_OVERRIDE true;
	proxy_pass http://w.x.y.z:7080;
	proxy_set_header Host $host;
	proxy_set_header X-Real-IP $remote_addr;
	proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
	proxy_set_header X-Accel-Internal /internal-nginx-static-location;
	access_log on;
}

location ~ /wp-json {

proxy_set_header AUTH_OVERRIDE true;

proxy_pass http://w.x.y.z:7080;

proxy_set_header Host $host;

proxy_set_header X-Real-IP $remote_addr;

proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

proxy_set_header X-Accel-Internal /internal-nginx-static-location;

access_log on;

}

Step 3. – In Plesk on the “Apache & nginx Setting” screen under “Additional directives for HTTP” and “Additional directives for HTTPS” add the following:

<Directory "/var/www/vhosts/acme.com/httpdocs" >

	# set an environment variable if there is an AUTH_OVERRIDE header
	SetEnvIf AUTH_OVERRIDE ^true AUTH_REQUEST

	# basic auth
	AuthName "Dev Login"
	AuthType Basic

	# password file is maintained from Plesk
	AuthUserFile /var/www/vhosts/system/acme.com/pd/d..httpdocs@donotremove
	<RequireAny>
		# require login
		Require valid-user
		# allow request if the environment variable is set
		Require env AUTH_REQUEST
	</RequireAny>
</Directory>

# set an environment variable if there is an AUTH_OVERRIDE header

SetEnvIf AUTH_OVERRIDE ^true AUTH_REQUEST

# basic auth

AuthName "Dev Login"

AuthType Basic

# password file is maintained from Plesk

AuthUserFile /var/www/vhosts/system/acme.com/pd/d..httpdocs@donotremove

# require login

Require valid-user

# allow request if the environment variable is set

Require env AUTH_REQUEST

</RequireAny>

</Directory>

.htaccess – negative file type match

By admin on March 20, 2019 in Apache

Here’s an example of how to block all files extensions not listed in the regular expression:

<FilesMatch "(?<!\.xml|\.css|\.jpg|\.jpeg|\.png|\.gif|\.js|\.pdf|\.svg|\.ttf|\.woff|\.woff2|\.woff3|\.otf|\.eot)$">
        Order Allow,Deny
        Deny from all
</FilesMatch>

Order Allow,Deny

Deny from all

</FilesMatch>

1 2 … 18 Next →

Need help immediately?

$100/hour
Satisfaction Guaranteed

Top Nav

Navigation