Uncharted territory

Uncharted territory
Rick Gibson

By Rick Gibson

April 13th 2012 at 10:00AM

Rick Gibson looks at the benefits and challenges offered up by sales data charts

The ambition of ‘digital’ charts is to provide data-starved games companies – usually publishers – with sales data crowdsourced in time-honoured fashion from their peers.

How valuable or representative is this data? Following UKIE’s new chart launch, I’ll assess the challenges faced by such charts through the lens of the different metrics used in different categories of game.

Coverage is the first hurdle. UKIE offers a UK-only sales chart for core PC downloads, hoping to add DLC and short-form console games.

This data would undoubtedly be useful for some, but many major companies in this space – market leaders like Valve and some of the largest publishers – remain reluctant to share per-title sales figures.

Even if UKIE bypasses operators like Valve to gather all the publishers’ global data – no mean feat – this would cover just eight per cent of the overall online and mobile games market. The remaining 92 per cent will be even more problematic.

UKIE wants to add social, MMOGs and mobile, markets with tens of thousands of companies. It will struggle to gather much below the top tier, and geography is another massive barrier.

With significant proportions of the market missing and charts largely populated by traditional publishers, this data will inevitably resemble Swiss cheese.

METRICS OF THE TRADE

The next issue surrounds what data is gathered. Accurate sales data for the MMOs, virtual worlds, mobile, browser, streamed, social network and skill games that comprise the bulk of the network games market would obviously be extremely beneficial for the entire industry.

However, sales data would only paint part of the picture. Few companies use just one commercial model these days.

The best stack them. Leading casual browser and download games portals for instance use four-to-five models, only one of which is unit sales, so to accurately describe their market (of similar scale to the core PC download market), one must profile them all.

Many commercial models, particularly freemium ones, require a broad range of usage, conversion and payment metrics to provide genuinely useful indications of commercial performance.

These metrics are almost never reported, even by listed companies, due to high commercial sensitivity. We gather this data and know that partial snapshots can be misleading

For example, some companies outside Facebook’s top 50 MAU/DAU charts would hit the top ten of gross revenues, whereas some top charters are making pocket change.

Equally, MMO companies often quote registered user numbers but these are near-meaningless because they always include lapsed, dormant, single-visit and non-playing users.

To fulfil its ambitions of informing contributors’ strategic decisions, UKIE will have to provide more than simply a sales chart.

Then there’s verification. Gathering data from developers means verifying its accuracy from potentially thousands of companies in different game markets, a daunting task given how rare such disclosure is in network gaming, and how much hype hits the fan of trade media.

Definitions are particularly problematic and vary between companies. Unlike social games, where an active user means someone interacting in the last month, on Xbox Live an active user is someone interacting in the previous six.

Equalising different accounting policies and establishing common definitions will be demanding to say the least.

Next, there’s the apples and oranges problem in an increasingly diverse industry. Would anyone seriously compare Mobage/ngmoco with EA/PopCap without a pile of provisos about differences in commercial models, user bases, territories and so on?

A Japanese mobile social company mostly monetising young Japanese adults is fundamentally different to an American cross-platform company mostly monetising American mothers. These are exactly the kind of companies that could be sitting, uncomfortably, alongside each other in a single chart.

THE HOLE PICTURE

UKIE’s digital chart comes from the right place but faces formidable challenges, some of which will prove insurmountable.

Traditional publishers may prefer Swiss cheese data to none at all, even where there’s far more hole than cheese, and there’s no doubt that these kinds of charts could be more reliable than the survey-based approaches currently misleading the markets, the dangers of which we have described before.

More fundamentally, do you still need to measure your company’s commercial success against others in the same field when the field’s so varied and the data’s so patchy?

For many, the answer is probably not.

Of course network games companies watch the competition’s moves in these continually changing markets, but the best performers improve by comparing themselves not against others but against themselves, continuously optimising performance versus their previous months or weeks.

With the real challenge being understanding their own commercial performance, as opposed to their competitors’, many companies may ask why they should share their data at all.