The Attention Index — An open source algorithm for ranking premium media

At Kaleida we’ve been developing a new method for measuring media — an algorithm that values quality and impact. We’re opening up the project with a Creative Commons license today so we can start collaborating with other orgs interested in using the data.

The Attention Index algorithm considers editorial decisions and social activity to derive attention scores.

The idea for Kaleida when we started the company a year ago was to reinvent the way media is measured.

The media ecosystem has been defined by reach, impressions, and clicks for about 20 years. From our experience much of the value that goes into producing quality journalism gets lost by those types of metrics, and the effect has been disastrous for the industry.

Yet it seems completely counterintuitive that media orgs would struggle given the massive opportunity the Internet offers. Over 3.5B people are connected now. Fast mobile networks reach 84% of the global population. And digital ad spending is expected to reach $224 billion globally in 2017.

If quality journalism outlets could demonstrate how much impact they have in the world then lucrative business models would surely find them.

So, we collected tons of data about the media and began analysing it. You can see our machine doing its magic throughout the day every day on, creating a map of interests in the world. The data shows what matters to whom, when, where and how much. That was the first phase of our plan to reinvent media measurement.

The second phase is beginning to unfold now — The Attention Index.

The Attention Index puts all this activity into context. It scores and ranks behaviours in relative terms so it’s easy to compare what’s working vs what’s not working. For example, it considers whether a publisher thinks something is particularly important and uses reader response to see if the public agrees.

We’ve started publishing raw data for people to download and explore. The May 2017 CSV includes 128,000 article headlines, URLs, editorial promotion data, social data and attention scores. Every month we’ll publish the data we’ve collected and scored with the algorithm.

The math and methodology is public, too. Kaleida CTO Graham Tackley has documented step-by-step how he applied various statistics and arrived at these results.

And, of course, everyone is invited to join the community and to contribute. The Attention Index is published with a Creative Commons license with support from Google’s DNI fund.

The Attention Index algo — Full exploration and sample data is available via a Jupyter Notebook.

There’s a lot more to do. We started by sharing our work with several media academics in the US and the UK. The data and the algorithm will require more scrutiny, more contributors, more data and more use cases.

Opening it up for public use will help us get that kind of engagement and, hopefully, make the Attention Index an industry standard.

Everyone we talk to in the media business is frustrated with many of the same things. Fighting to lead on metrics you don’t believe in and that don’t really serve your business goals is right near the top of the list.

The lower-end commodity metrics will always be useful, but it’s time to recognize things that matter and to talk about them with shared language.

Keep an eye on the project and be sure to get in touch if you want to get involved or if you have questions about it.