Ask Apache not to log certain requests in access_log

Apache web serverApache logs all requests in its log file access_log. But sometimes we don’t need it to log all the requests, for example we don’t want the requests from a certain IP to be logged, or we don’t need to log all the requests to image files. This guide shows how we can use Apache’s directive to prevent from logging unwanted requests.

The SetEnvIf directive that provided by mod_setenvif allows the setting of environment variables based on characteristics of client request. With the help of these variables, we’re able to tell apache do not log certain requests. Check my former post for a quick user guide on SetEnvIf.

Let’s assume that your monitoring system is requesting the file /status/online.html once half a minute to check if Apache is running well. Obviously we don’t want to log these requests because they are not from a real user. Therefore we can use the following settings in apache’s configuration file:

SetEnvIf Request_URI "^/status/online.html$" dontlog

Then we need to tell Apache that it must not log all requests labelled with dontlog. Find the CustomLog directive in your Apache configuration file, and add env=!dontlog to the end, like this:

CustomLog /var/log/apache2/access.log combined env=!dontlog

or if you have cronolog enabled:

CustomLog "|/usr/bin/cronolog --symlink=/var/log/httpd/access.log /var/log/httpd/access.log_%Y_%m_%d" combined env=!dontlog

Restart Apache afterwards. Now it won’t log any request anymore that is labelled with dontlog, it’s the file /status/online.html here.

To prevent all requests made with a certain browser, e.g. Internet Explorer, from getting logged, you could use:

SetEnvIf User_Agent "(MSIE)" dontlog

To not log requests from any client whose hostname ends in, use:

SetEnvIf Remote_Host "$" dontlog

To not log requests from any client whose hostname begins with example, use:

SetEnvIf Remote_Host "^example" dontlog

To not log requests from a certain IP address, use something like:

SetEnvIf Remote_Addr "" dontlog

If you don’t want requests of your robots.txt to get logged, use:

SetEnvIf Request_URI "^/robots.txt$" dontlog

Apart from SetEnvIf, which is case-sensitive, you can use SetEnvIfNoCase which is case-insensitive.

For example, in order not to log certain search engine spiders, you could use:

SetEnvIFNoCase User-Agent "Slurp/cat" dontlog
SetEnvIFNoCase User-Agent "Ask Jeeves/Teoma" dontlog
SetEnvIFNoCase User-Agent "Googlebot" dontlog

Or to not log certain file extensions, use something like this:

SetEnvIfNoCase Request_URI ".(gif)|(jpg)|(png)|(css)|(js)|(ico)$" dontlog

To not log certain referrals (e.g. from your own domain), use something like:

SetEnvIfNoCase Referer "www.domainname.tld" dontlog
Share Button

Leave a comment

Your email address will not be published. Required fields are marked *