The useful convergence of data

I have only one prediction for 2008. I think we’re finally about to see the useful combination of the 4 W’s – Who, What, Where, and When.

Marc Davis has done some interesting research in this area at Yahoo!, and Bradley Horowitz articulated how he sees the future of this space unfolding in a BBC article in June ’07:

“We do a great job as a culture of “when”. Using GMT I can say this particular moment in time and we have a great consensus about what that means…We also do a very good job of “where” – with GPS we have latitude and longitude and can specify a precise location on the planet…The remaining two Ws – we are not doing a great job of.”

I’d argue that the social networks are now really honing in on “who”, and despite having few open standards for “what” data (other than UPC) there is no shortage of “what” data amongst all the “what” providers. Every product vendor has their own version of a product identifier or serial number (such as Amazon’s ASIN, for example).

We’ve seen a lot of online services solving problems in these areas either by isolating specific pieces of data or combining the data in specific ways. But nobody has yet integrated all 4 in a meaningful way.


Jeff Jarvis’ insightful post on social airlines starts to show how these concepts might form in all kinds of markets. When you’re traveling it makes a lot of sense to tap into “who” data to create compelling experiences that will benefit everyone:

  • At the simplest level, we could connect while in the air to set up shared cab rides once we land, saving passengers a fortune.
  • We can ask our fellow passengers who live in or frequently visit a destination for their recommendations for restaurants, things to do, ways to get around.
  • We can play games.
  • What if you chose to fly on one airline vs. another because you knew and liked the people better? What if the airline’s brand became its passengers?
  • Imagine if on this onboard social network, you could find people you want to meet – people in the same business going to the same conference, people of similar interests, future husbands and wives – and you can rendezvous in the lounge.
  • The airline can set up an auction marketplace for at least some of the seats: What’s it worth for you to fly to Berlin next Wednesday?

Carrying the theme to retail markets, you can imagine that you will walk into H&M and discover that one of your first-degree contacts recently bought the same shirt you were about to purchase. You buy a different one instead. Or people who usually buy the same hair conditioner as you at the Walgreen’s you’re in now are switching to a different hair conditioner this month. Though this wouldn’t help someone like me who has no hair to condition.

Similarly, you can imagine that marketing messages could actually become useful in addition to being relevant. If CostCo would tell me which of the products I often buy are on sale as I’m shopping, or which of the products I’m likely to need given what they know about how much I buy of what and when, then my loyalty there is going to shoot through the roof. They may even be able to identify that I’m likely buying milk elsewhere and give me a one-time coupon for CostCo milk.

Bradley sees it playing out on the phone, too:

“On my phone I see prices for a can of soup in my neighbourhood. It resolves not only that particular can of soup but knows who I am, where I am and where I live and helps me make an intelligent decision about whether or not it is a fair price.

It has to be transparent and it has to be easy because I am not going to invest a lot of effort or time to save 13 cents.”

It may be unrealistic to expect that this trend will explode in 2008, but I expect it to at least appear in a number of places and inspire future implementations as a result. What I’m sure we will see in 2008 is dramatic growth in the behind-the-scenes work that will make this happen, such as the development and customization of CRM-like systems.

Lots of companies have danced around these ideas for years, but I think the ideas and the technologies are finally ready to create something real, something very powerful.

Photo: SophieMuc

The problem with being popular (part 2)

One of the more interesting sciences, in my mind, is how information relevance is both determined, surfaced and then evolved.

In Fred Wilson’s recent Cautionary Techmeme Tale he argues that making news popular takes away its social context and therefore becomes meaningless. He found Techmeme more useful when its sources more closely resembled his network of friends:

“For years, I’ve been using curators to filter my web experience…Techmeme has been the killer social media curator for my world of tech blogs. Lore has it that it was created using Scoble’s OPML file. It doesn’t matter to me if that’s true or not, I love that story. Because my OPML file was unusable until I found Techmeme and after that I stopped reading feeds and started reading curated feeds.”

This feeds into a larger argument about why pop culture and the art of being or becoming popular can be a bad thing. Not long ago I was inspired by the movie “Good Night and Good Luck” to dive into this idea myself:

“The real problem with popularity-driven models is that they reduce both the breadth and depth of the sources, topics and viewpoints being expressed across a community. Popularity-driven models water down the value in those hard-to-find nuggets. They normalize coverage and create new power structures that interesting things have to fight through.”

This is exactly why personalization, recommendations and social media technologies really matter. They can solve this problem of creating conformist media consumption practices by creating relevance through networks of people rather than through networks of commercial institutions.

I haven’t used My Yahoo! as much as I’d like, but there is a simple function in it that I love which could ultimately create amazing benefits for people who want a human filter for the Internet. It’s called “Top Picks”.

“The Top Picks module automatically highlights stories from your page, based on the articles you have recently read on My Yahoo! The more stories you click on, the more you will see this module reflect your interests.”

Actually, the technology beneath it is not so ‘simple’ but the application of it here makes so much sense that it feels like it’s simple when you watch it work. It works by using implicit behaviors. I don’t have to tell it what I like. It learns.

If it could also show me what my social network is tapped into right now, then the experience would feel nearly complete.

Media researchers will note here that people need pop culture to feel connected to a greater whole. I believe that’s true, too. Television is an amazingly powerful community builder.

But I would gladly trade a powerful singular social voice tied together by networks of distribution ownership for a less unified but still loosely connected network of pop culture tied together by my personal activities and my social connections.

The business of network effects

The Internet platform business has some unique challenges. It’s very tempting to adopt known models to make sense of it, like the PC business, for example, and think of the Internet platform like an operating system.

The similarities are hard to deny, and who wouldn’t want to control the operating system of the Internet?

In 2005, Jason Kottke proposed a vision for the “WebOS” where users could control their experience with tools that leveraged a combination of local storage and a local server, networked services and rich clients.

“Applications developed for this hypothetical platform have some powerful advantages. Because they run in a Web browser, these applications are cross platform, just like Web apps such as Gmail, Basecamp, and Salesforce.com. You don’t need to be on a specific machine with a specific OS…you just need a browser + local Web server to access your favorite data and apps.”

Prior to that post, Nick Carr offered a view on the role of the browser that surely resonated with the OS perspective for the Internet:

“Forget the traditional user interface. The looming battle in the information technology business is over control of the utility interface…Control over the utility interface will provide an IT vendor with the kind of power that Microsoft has long held through its control of the PC user interface.”

He also responded later to Kottke’s vision saying that the reliance on local web and storage services on a user’s PC may be unnecessary:

“Your personal desktop, residing entirely on a distant server, will be easily accessible from any device wherever you go. Personal computing will have broken free of the personal computer.”

But the client layer is merely a piece of the much larger puzzle, in my opinon.

Dare Obasanjo more recently broke down the different ideas of what “Cloud OS” might mean:

“I think it is a good idea for people to have a clear idea of what they are talking about when they throw around terms like “cloud OS” or “cloud platform” so we don’t end up with another useless term like SOA which means a different thing to each person who talks about it. Below are the three main ideas people often identify as a “Web OS”, “cloud OS” or “cloud platform” and examples of companies executing on that vision.”

He defines them as follows:

  1. WIMP Desktop Environment Implemented as a Rich Internet Application (The YouOS Strategy)
  2. Platform for Building Web-based Applications (The Amazon Strategy)
  3. Web-based Applications and APIs for Integrating with Them (The Google Strategy)

The OS metaphor has lots of powerful implications for business models, as we’ve seen on the PC. The operating system in a PC controls all the connections from the application user experience through the filesystem down through the computer hardware itself out to the interaction with peripheral services. Being the omniscient hub makes the operating system a very effective taxman for every service in the stack. And from there, the revenue streams become very easy to enable and enforce.

But the OS metaphor implies a command-and-control dynamic that doesn’t really work in a global network controlled only by protocols.

Internet software and media businesses don’t have an equivilent choke point. There’s no single processor or function or service that controls the Internet experience. There’s no one technology or one company that owns distribution.

There are lots of stacks that do have choke points on the Internet. And there are choke points that have tremendous value and leverage. Some are built purely and intentionally on top of a distribution point such as the iPod on iTunes, for example.

But no single distribution center touches all the points in any stack. The Internet business is fundamentally made of data vectors, not operational stacks.

Jeremy Zawodny shed light on this concept for me using building construction analogies.

He noted that my building contractor doesn’t exclusively buy Makita or DeWalt or Ryobi tools, though some tools make more sense in bundles. He buys the tool that is best for the job and what he needs.

My contractor doesn’t employ plumbers, roofers and electricians himself. Rather he maintains a network of favorite providers who will serve different needs on different jobs.

He provides value to me as an experienced distribution and aggregation point, but I am not exclusively tied to using him for everything I want to do with my house, either.

Similarly, the Internet market is a network of services. The trick to understanding what the business model looks like is figuring out how to open and connect services in ways that add value to the business.

In a precient viewpoint from 2002 about the Internet platform business, Tim O’Reilly explained why a company that has a large and valuable data store should open it up to the wider network:

“If they don’t ride the horse in the direction it’s going, it will run away from them. The companies that “grasp the nettle firmly” (as my English mother likes to say) will reap the benefits of greater control over their future than those who simply wait for events to overtake them.

There are a number of ways for a company to get benefits out of providing data to remote programmers:

Revenue. The brute force approach imposes costs both on the company whose data is being spidered and on the company doing the spidering. A simple API that makes the operation faster and more efficient is worth money. What’s more, it opens up whole new markets. Amazon-powered library catalogs anyone?

Branding. A company that provides data to remote programmers can request branding as a condition of the service.

Platform lock in. As Microsoft has demonstrated time and time again, a platform strategy beats an application strategy every time. Once you become part of the platform that other applications rely on, you are a key part of the computing infrastructure, and very difficult to dislodge. The companies that knowingly take their data assets and make them indispensable to developers will cement their role as a key part of the computing infrastructure.

Goodwill. Especially in the fast-moving high-tech industry, the “coolness” factor can make a huge difference both in attracting customers and in attracting the best staff.”

That doesn’t clearly translate into traditional business models necessarily, but if you look at key business breakthroughs in the past, the picture today becomes more clear.

  1. The first breakthrough business model was based around page views. The domain created an Apple-like controlled container. Exposure to eyeballs was sold by the thousands per domain. All the software and content was owned and operated by the domain owner, except the user’s browser. All you needed was to get and keep eyeballs on your domain.
  2. The second breakthrough business model emerged out of innovations in distribution. By building a powerful distribution center and direct connections with the user experience, advertising could be sold both where people began their online experiences and at the various independent domain stacks where they landed. Inventory beget spending beget redistribution beget inventory…it started to look a lot like network effects as it matured.
  3. The third breakthrough business model seems to be a riff on its predecessors and looks less and less like an operating system. The next breakthrough is network effects.

Network EffectsNetwork effects happen when the value of the entire network increases with each node added to the network. The telephone is the classic example, where every telephone becomes more valuable with each new phone in the network.

This is in contrast to TVs which don’t care or even notice if more TVs plug in.

Recommendation engines are the ultimate network effect lubricator. The more people shop at Amazon, the better their recommendation engine gets…which, in turn, helps people buy more stuff at Amazon.

Network effects are built around unique and useful nodes with transparent and highly accessible connection points. Social networks are a good example because they use a person’s profile as a node and a person’s email address as a connection point.

Network effects can be built around other things like keyword-tagged URLs (del.icio.us), shared photos (flickr), songs played (last.fm), news items about locations (outside.in).

The contribution of each data point wherever that may happen makes the aggregate pool more valuable. And as long as there are obvious and open ways for those data points to talk to each other and other systems, then network effects are enabled.

Launching successful network effect businesses is no easy task. The value a participant can extract from the network must be higher than the cost of adding a node in the network. The network’s purpose and its output must be indespensible to the node creators.

Massively distributed network effects require some unique characteristics to form. Value not only has to build with each new node, but the value of each node needs to increase as it gets leveraged in other ways in the network.

For example, my email address has become an enabler around the Internet. Every site that requires a login is going to capture my email address. And as I build a relationship with those sites, my email address becomes increasingly important to me. Not only is having an email address adding value to the entire network of email addresses, but the value of my email address increases for me with each service that is able to leverage my investment in my email address.

Then the core services built around my email address start to increase in value, too.

For example, when I turned on my iPhone and discovered that my Yahoo! Address Book was automatically cooked right in without any manual importing, I suddenly realized that my Yahoo! Address Book has been a constant in my life ever since I got my first Yahoo! email address back in the ’90’s. I haven’t kept it current, but it has followed me from job to job in a way that Outlook has never been able to do.

My Yahoo! Address Book is becoming more and more valuable to me. And my iPhone is more compelling because of my investment in my email address and my address book.

Now, if the network was an operating system, there would be taxes to pay. Apple would have to pay a tax for accessing my address book, and I would have to pay a tax to keep my address book at Yahoo!. Nobody wins in that scenario.

User data needs to be open and accessible in meaningful ways, and revenue needs to be built as a result of the effects of having open data rather than as a margin-based cost-control business.

But Dare Obasanjo insightfully exposes the flaw in reducing openness around identity to individual control alone:

“One of the bitter truths about “Web 2.0” is that your data isn’t all that interesting, our data on the other hand is very interesting…A lot of “Web 2.0″ websites provide value to their users via wisdom of the crowds appproaches such as tagging or recommendations which are simply not possible with a single user’s data set or with a small set of users.”

Clearly, one of the most successful revenue-driving opportunities in the networked economy is advertising. It makes sense that it would be since so many of the most powerful network effects are built on people’s profiles and their relationships with other people. No wonder advertisers can’t spend enough money online to reach their targets.

It will be interesting to see how some of the clever startups leveraging network effects such as Wesabe think about advertising.

Wesabe have built network effects around people’s spending behavior. As you track your finances and pull in your personal banking data, Wesabe makes loose connections between your transactions and other people who have made similar transactions. Each new person and each new transaction creates more value in the aggregate pool. You then discover other people who have advice about spending in ways that are highly relevant to you.

I’ve been a fan of Netflix for a long time now, but when Wesabe showed me that lots of Netflix customers were switching to Blockbuster, I had to investigate and before long decided to switch, too. Wesabe knew to advise me based on my purchasing behavior which is a much stronger indicator of my interests than my reading behavior.

Advertisers should be drooling at the prospects of reaching people on Wesabe. No doubt Netflix should encourage their loyal subscribers to use Wesabe, too.

The many explicit clues about my interests I leave around the Internet — my listening behavior at last.fm, my information needs I express in del.icio.us, my address book relationships, my purchasing behavior in Wesabe — are all incredibly fruitful data points that advertisers want access to.

And with managed distribution, a powerful ad platform could form around these explicit behaviors that can be loosely connected everywhere I go.

Netflix could automatically find me while I’m reading a movie review on a friend’s blog or even at The New York Times and offer me a discount to re-subscribe. I’m sure they would love to pay lots of money for an ad that was so precisely targeted.

That blogger and The New York Times would be happy share revenue back to the ad platform provider who enabled such precise targeting that resulted in higher payouts overall.

And I might actually come back to Netflix if I saw that ad. Who knows, I might even start paying more attention to ads if they started to find me rather than interrupt me.

This is why the Internet looks less and less like an operating system to me. Network effects look different to me in the way people participate in them and extract value from them, the way data and technologies connect to them, and the way markets and revenue streams build off of them.

Operating systems are about command-and-control distribution points, whereas network effects are about joining vectors to create leverage.

I know little about the mathematical nuances of chaos theory, but it offers some relevant philosophical approaches to understanding what network effects are about. Wikipedia addresses how chaos theory affects organizational development:

“Most of the focus on chaos theory is primarily rooted in the underlying patterns found in an otherwise chaotic enviornment, more specifically, concepts such as self-organization, bifurcation and self-similarity…

Self-organization, as opposed to natural or social selection, is a dynamic change within the organization where system changes are made by recalculating, re-inventing and modifying its structure in order to adapt, survive, grow and develop. Self-organization is the result of re-invention and creative adaptation due to the introduction of, or being in a constant state of, perturbed equilibrium.”

Yes, my PC is often in a state of ‘perturbed equilibrium’ but not because it wants to be.

A start page on my own domain

With a quick copy and paste job using Kent Brewster’s Pipes Badger and a few widgets from services I use, I now have what is a mostly sufficient start page on my own domain that displays my various forms of online expression. Really interesting stuff here.

For my wishlist: a start page that learns

I had the pleasure of joining Rex Hammock for drinks last night in Potrero Hill while he was here for Macworld Expo.

Rex is tuned in to some interesting aspects of the online world, particularly through his site SmallBusiness.com which is becoming a useful and increasingly powerful wiki. I was amazed to hear that the contributions are no longer coming from his team. The community is making the site work and building it into a resource that matters.

We also talked about RSS and start pages. Rex shares my frustration that start pages are so dependent on custom configurations that the majority of the world will never do. Machine learning and recommendations technology is not new, and it seems like such an obvious direction for the start page to go…

Show me what the world looks like through a global lens, my networks' lenses and my own personal lens. Learn from both my explicit and implicit behaviors and then adjust.

Amazon knows how to use my shopping behavior to create compelling shopping experiences. Why can’t my news reading behavior be interpreted to create a better start page experience?

The Onion understands this, too:

Amazon Recommendations Understand Area Woman Better Than Husband

Pamela Meyers said that her husband, whose gift choices have never reflected any outward recognition of her desire to learn Spanish, nor of the fact that she looks terrible in orange, rarely, if ever, communicates with Meyers while away on any of his frequent business trips.

“I was having some tea from that Nebraska Cornhuskers mug Dean got me for Valentine’s Day, when a little emai from Amazon popped up out of the blue,” Meyer said. “Just completely out of the blue.”

“It was nice to know that on my birthday, someone or something was out there thinking about me, and what boxsed sets I wanted.”

Switching my default browser home page, again

I’ve probably changed my browser’s default home page about 10 times in the last year. Something about working here at Yahoo! has made me very picky about start pages.

The new Yahoo! home pageI most recently was using Netvibes which had a couple of really cool modules: a notes box that you could write in just by clicking in it and a sudoku puzzle that I would play on the train ride home. Unfortunately, Netvibes became way too slow for me. I found myself typing in a new URL before Netvibes came up every time I launched a browser window or clicked ‘home’.

I don’t think Netvibes is alone in learning the hard lesson of scaling personalization features. It’s clear that NewsAlloy is struggling under the weight of their usage, and Rojo recently rescued their ailing infrastructure, at least we hope, by adopting a new parent in MovableType.

Even more dramatic is the performance on Wizag. Wizag is one of the most promising start pages I’ve seen yet with its learning and categorization concepts. The design is awful and the speed is unusable, but those problems are easier to solve than developing really new and interesting algorithms. I’m hoping they figure these things out, because I would love to use it more.

Not too long ago I tried switching to Google’s Personalized page. I loved the integration with my phone. You can select modules from your personalized start page that will appear on the phone version. It’s really smart. And it made me try using Google Reader more. But Google Reader is just not the way I want to work with my feed sources, and I got too annoyed.

Why not use My Yahoo! as your browser home page, you ask? I use My Yahoo!, actually. At least weekly. But it shares a problem I have with all personalized start pages…I want my browser to open with something that I don’t know. I want it to lead me, sometimes just a little bit.

And I just learned when I switched to the new Yahoo! home page that I want big pictures, too.

The new Yahoo! home page is brilliant. It has everything I actually want just prior to starting a journey somewhere or even when I’m not sure where to start. I can see the most recent email messages without having to open the full email app. I can check out traffic in my neighborhood, send a quick IM, search and get to my feeds (on My Yahoo!) all from the same place with minimal effort.

But what I love most is that the Yahoo! home page shows me stuff that I don’t know. The top stories have huge impact. They’re inviting, and they make me want to click. And the pulse box always catches my attention with the Top 10 this and Top 10 that.

One of the proven rules in magazine cover selling at the newsstand is that people love top 10 lists. It’s true online, too.

We also learned at InfoWorld how powerful imagery can be when we studied people’s eye movements on a more image-driven home page. The results of that study are here.

No doubt, I’ll switch home pages again soon. I haven’t stuck with one page for more than a few months, but I also don’t remember being as pleased as I am with this page. The dust has settled from the launch earlier in the summer, and I have to agree with what most people in the industry said: The new Yahoo! home page rocks.