In further testing using real-world data, we have a couple of interesting new findings, but nothing that changes our conclusions.

In a sample size of ~2000, we found that our own Google Analytics verified our click throughs only 85% of the time. That's a drop from our initial 'closed system' where we found 90% accuracy. We speculate that the causes could be network connectivity issues, JavaScript-disabled browsers, failure for the page to completely load before an ad is clicked, or some combination of all three.

In a smaller sample size of real world testing with some of our testing partners' sites, we found that Google Analytics' Referring Sites feature was (as with our initial testing) highly variable in its results. One test site saw only 50% of the visits as being referred by us, whereas another site (with a larger sample size) saw 135% of visits we sent over as having come from us. After a bit of digging around, the only suggestion we have here is that some of the clicks we filter out as having come from bots, spiders or user agents were not filtered out by Google Analytics.

So, overall our initial conclusion remains: one cannot rely 100% (and maybe not even 90%) on the accuracy of Google Analytics. In the case of Referring Sites data, we confirm that the discrepancies are often even larger.

We'd welcome any feedback on this study, from our advertisers or elsewhere.