This meter runs funny

Patrick Ruffini has figured out that a limitation of SiteMeter has “inflated” the daily counts at DailyKos, which obviously isn’t Kos’ fault and doesn’t affect his King of the Hill status — though, as Ruffini says, the hill isn’t quite as large as it seems. (Meryl Yourish, who took issue with Ruffini’s methodology early on, has since come around to partial agreement.)

What tipped Ruffini off:

First of all, I looked at the Detail view showing the last 100 visitors. Overwhelmingly it showed visitors hitting the site only once, with a visit time of zero (you need to hit a second page for it to register any time spent). Contrasted with my traffic, with an average visit length of three minutes, this seemed highly improbable.

Then it hit me: SiteMeter only accounts for the last 100 visitors individually. On a site like Daily Kos, the 100th most recent visitor could have been 15 seconds ago. If you are the 101st most recent visitor and you click on a new page, you are counted as a new unique visitor in SiteMeter’s all important count. On a normal site, this wouldn’t matter, since it’s highly unlikely you’ll stick around long enough to have 100 others show up after you. On a site with hundreds of thousands of page views a day, it’s extremely likely you will.

Yourish pointed out that SiteMeter doesn’t actually count uniques, only page views. Ruffini looked at the page view per visitor ratio, and noted:

We now know that the only thing we can trust about the SiteMeter numbers are the page views. And from that we can arrive at a more realistic number of daily unique visitors for Daily Kos and other leading blogs.

How so? The best guide we probably have are other netroots blogs like MyDD and OpenLeft built on open community platforms. They have low enough traffic that SiteMeter’s inflationary effect is minimal at best. Using Scoop (what Kos uses) and SoapBlox respectively, both have a ratio of about 1.9 page views for every visit (itself a less stringent measure than “unique visitor”). On Red State, where there is likely a little bit of this effect, it’s about 1.8 to 1. On a WordPress-style blog without diaries, the ratio averages 1.5 page views per visit.

I average a very consistent 1.4 page views per visit; this number has not varied up or down more than 0.2 in this century. (I first installed SiteMeter in 1999.) Whether this means I should take my numbers more seriously than I should some others, I don’t know for sure; there’s one additional variable in the mix, and that’s that I’m a paid SiteMeter customer, which may or may not get me slightly greater accuracy.

A more serious deficiency, and one which I don’t think can be easily addressed, is the difficulty of determining overall readership when so much of it comes from syndication feeds. During the first three days of this month, SiteMeter reported 2,040 visitors and 2,684 page views; however, according to the server log, there were 2,474 requests for the RSS and/or Atom feed. (These periods do not exactly coincide, due to time-zone differences, but are both 72 hours long.) This does not include the 1,137 requests for index.rdf, which covers RSS 0.91/1.0; in other words, while I’m getting six hundred-odd people every day to load enough of a page to trip the meter, I’m getting twice that many who don’t.

I’d better stop here before it looks like I’m actually concerned about counting heads.

1 comment

  1. Richard from GoStats »

    6 October 2007 · 1:52 pm

    On that note, try GoStats (you can keep sitemeter while you try GoStats – it’s free anyway). GoStats has a more advanced unique visitor determination.

RSS feed for comments on this post