Analyzing Site Traffic

There are a number of tools you can use to analyze your web site’s traffic but they fall into two primary categories – Web based tools and Server based tools. What’s the difference and how can they affect your sites stats? Read on…

Web based tools work by having you insert a small snippet of code on every page in your website. When the page loads, that small snippet of code reports statistics back to the website of the analytics company. The most popular of this type of tool is Google Analytics, however there are a number of other good ones, including GetClicky and CrazyEgg.

The advantage of this method is that there are a number of excellent tools available, many of them free, and it’s easy to get them up and running. Just copy the snippet of code into each page of your site and then login to the analytics company’s site and run reports on your traffic. The downside of this method is that these services are not always completely accurate. Google analytics, for example, instructs you to place their snippet of code at the bottom of each page. If your page doesn’t load completely, due to a slow server, slow internet connection or a visitor leaving the page quickly before it completely loads, then Google Analytics will not register the visit at all. This is a frequently reported problem among FaceBook advertisers who find that they number of clicks that FaceBook charges them is much larger than the number of visits that Google Analytics shows as having come from FaceBook.

Server based tools don’t require copying anything into your site. They work by analyzing the log files that every web server generates. The advantage of these tools is that they are 100% accurate. If a request is made to your web server, it will show up in the log.

The downside is that these tools generally must be purchased by your web hosting company and run from their servers. Therefore, you will be stuck with whatever tool your web hosting company offers, or no tool at all if they don’t provide one. It is technically possible for you to download your web log files and analyze them yourself with a tool of this type, however, log files can quickly grow to gigabytes in size, making it difficult and slow to download and extremely slow to analyze on a personal computer. Plus, this means you have to find, download and possibly purchase the analysis program, many of which cost several hundred if not several thousand dollars since they are typically being sold to large hosting companies.

Both types of analytics solutions have their pros and cons, but they all suffer from a key shortcoming. This shortcoming has to do with the fundamental data that they are able to collect.

At the most basic level, there is only so much that a web server or a small snippet of code can ascertain about a visitor to your site. The main pieces of information they can record are the page that was visited, the time that page was visited, the IP address of the visitor, the web page that referred them to your page (if there was one) and the type of browser that the visitor was using (Internet Explorer, FireFox, etc.). From this basic information, these programs must infer all the other data they present to you. Much of this inference is presented as fact whereas it is actually only interpretation.

For example, take the statistic called “unique visits”, which most analytics programs report. This stat represents the actual number of people that visited your site. However, analytics programs can’t actually know this. They only know when a page is accessed and by what IP address. So, they must infer that if the same IP address requests 4 different pages within a 20 minute period of time, that must represent a single person requesting 4 different pages so it is considered one visit. This leads to all sorts of inaccuracies and reporting differences between analytics programs. For instance, some analytics programs may consider the period of time to be only 10 minutes in which case they would report the above example as two visits instead of one. Many companies have 10s if not 100s of users that share a single IP address which means that if two people from the same company visit your site within the 20 minute time-frame, they will be recorded as a single visit instead of as the two unique visits which they actually are.

There are different types of analytics programs and they each have their pros and cons but they all suffer from similar basic reporting inaccuracies due to the way they interpret the data they have access to.