Web metrics that actually add up

By: Rebecca Lieb

Everything on the Web is measurable in so many disparate ways, it’s easy to see why many marketers despair they’ll ever get a real grip on Web metrics and analytics.

Anyone who’s ever compared two analytics solutions side by side knows even the most basic stats can differ considerably, often by a wide margin. And of course, there’s no all-in-one solution. Online marketers and advertisers rely on an array of packages and solutions to measure and analyze their Web sites, e-mail campaigns, blogs, RSS feeds, branding, conversions, and anything related either to online-to-offline or offline-to-online activity.

Web measurement is strikingly similar to counting in Japanese. I recently hit a wall in my efforts to learn that language when we reached the chapter on counting stuff.

It doesn’t suffice to learn ichi, ni, san, shi — not by a long shot. No one warned me that Japanese has one counting system for round things, another for flat things, one for floors of a building, one for big animals, another for small animals (watch out — birds and rabbits have their own category, fish are in yet another).

Think that’s complicated? It’s the tip of the iceberg.

So wacky and complex is the Japanese numeric system it’s why scholars believe Japanese is unrelated to any other language. Web metrics, the way marketers keep score, not to mention allocate literally billions of dollars, is sounding more and more like Japanese.

Analytics experts agree marketers would do well to use the analytics tools they acquire from vendors, assuming, of course, they’re adept at using these incredibly complex measurement tools (no mean feat). For research spanning the entire Web, things rapidly become much murkier.

Several days ago, the Interactive Advertising Bureau (IAB) very publicly challenged online measurement firms comScore and Nielsen//NetRatings (NNR) to open their kimonos and come clean about their respective methodologies. IAB CEO Randall Rothenberg says he’s aghast both firms still rely on antiquated panel-based measurement and what he terms “flawed media-research methodologies.”

The IAB is asking for transparency, and its demand is backed by the other major industry organizations, including the Association of National Advertisers, the American Association of Advertising Agencies, and the Online Publishers Association. The IAB claims online advertising growth will suffer if advertisers can’t count on reliable, accurate site audience and ad impression reports.

NNR and comScore measure the same things; yet to advertisers’ frustration, it often seems one set of numbers is metric and the other imperial. A recent Forrester report cites NNR reporting online search grew 39 percent between 2006 and 2007, while comScore reported 10.7 percent growth for the same period. In some cases, one company will report growth in an area when the other reports a decline.

Why the discrepancies? For starters, panels are extremely problematic. On top of that, the companies have widely differing recruitment methodologies and weight their data differently. Moreover, online measurement is nowhere near as clear cut as it initially appears.

Yet advertisers and their agencies pay a pretty penny for NNR and comScore data and stake even more on their numbers, both in terms of spend and return on those investments. The IAB’s calling for transparency, accreditation, audits, and standards and for methodologies that take the technologies of both the present and the future into consideration. As the IAB SVP and GM Sheryl Draizen pointed out at ClickZ’s Web Analytics conference this week, “A rise in the use of AJAX programming has publishers scrambling to find a metric to replace one that could be made obsolete: the pageview.”

The goal, says Draizen, is to provide real accountability and to spur the growth of online advertising and marketing.

The IAB’s method of doing this, publishing an open letter, seems hard-ass to some, not the least of all to the measurement companies themselves. I’ve spoken to people at both firms who claim they were working along these lines all along, a fact the IAB was aware of. Draizen says the open letter became open as the result of a leak, prompting the decision to publish the document.

Either way, the IAB turned up the heat, and the tactic seems to be getting the desired results. ComScore and NNR will sit down with the leading industry trade organizations on May 16.

This can’t be a bad thing. Better numbers and greater transparency mean better online advertising. That’s not just a good thing for advertisers, publishers, and the trade organizations that represent them; it’s ultimately good for the measurement companies that count this constituency as the lion’s share of their client base, too.