13 Jan 2009 12:01:20
In further testing using real-world data, we have a couple of interesting new findings, but nothing that changes our conclusions.
In a smaller sample size of real world testing with some of our testing partners' sites, we found that Google Analytics' Referring Sites feature was (as with our initial testing) highly variable in its results. One test site saw only 50% of the visits as being referred by us, whereas another site (with a larger sample size) saw 135% of visits we sent over as having come from us. After a bit of digging around, the only suggestion we have here is that some of the clicks we filter out as having come from bots, spiders or user agents were not filtered out by Google Analytics.
So, overall our initial conclusion remains: one cannot rely 100% (and maybe not even 90%) on the accuracy of Google Analytics. In the case of Referring Sites data, we confirm that the discrepancies are often even larger.
We'd welcome any feedback on this study, from our advertisers or elsewhere.