Apache Log files

This section will deal with how to analyze your log files for your web server, dealing with Apache web server in general, as it is the most popular.

When a visitor visits your website, the TCP/IP stack sends the request to the server, along with environmental variables, and the server sends back HTTP headers back to the client. Here are a few standard Apache environmental variables and what they mean:

REMOTE_ADDR: Remote I.P. Address or hostname of the client

REQUEST_METHOD: POST, GET, or HEAD. GET is the most common form, meaning the client is requesting a document from the server.

REQUEST_URI: The requested document relative to the document root. ( e.g.. /test/123.html )

Note: The above variables are requests sent from the CLIENT that the server logs.

Here are some common HTTP headers:

HTTP_HOST: The base URL of the host ( e.g. www.icehousedesigns.com )

HTTP_REFERER: The URL of the page that made the request. If linked from e-mail this value will be null.

HTTP_USER_AGENT: The browser ID or user-agent string identifying the browser (nominally defined by RFC 1945 and RFC 2068). Read here for more info about the HTTP_USER_AGENT header.

Note: The above are HTTP headers. This is requests the the server sends to the client, and the client responds to...this info is logged also.

Here is a sample line of a log file from my server. We will break it down to see what it all means.

255.255.255.255 - - [18/Jul/2001:10:18:08 -0400] "GET /portfolio/logos.php3 HTTP/1.1" 200 23745 "http://www.google.com/search?hl=hr&safe=off&q=free+flash+logo+design" "Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 4.0)"

OK lets break down the above log entry.

255.255.255.255 : The I.P. address of the visitor ( IP changed to protect identity )

[18/Jul/2001:10:18:08 -0400] - The date and time of the visit

GET - The request method

/portfolio/logos.php3 - The document relative to root that was requested

HTTP/1.1 - The content type of the attached info.

200 - The server code returned ( 200 OK, 404 not found, 500 server error, etc. )

23745 - The document size in bytes

"http://www.google.com/search?hl=hr&safe=off&q=free+flash+logo+design" - The referring URL ( the URL the visitor came from )

"Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 4.0)" - The User-Agent of the visitor.

Apache log files are generally found in your /log/httpd directory. What they are named and the exact path is up to your Apache configuration. The error log may be stored in a separate file. Find out from your web host if you don't know. Also, not all information as I have above maybe logged. You may have to request from your host ( or do it yourself ) that certain features of your log files be enabled in your Apache config.

There are many different ways to analyze your log files. There are some programs that do it for you, and there are other programs that will create their own log files for you. I recommend Web Trends Log Analyzer. It is a very fast, clean, and and very useful program.

If you have any questions, feel free to visit our forums.

Table of Contents