What is it that makes my favorite online services so compelling? I’m talking about the whole family of services that includes Dopplr, Wesabe, Twitter, Flickr, and del.icio.us among others.

I find it interesting that people don’t generally refer to any of these as “web sites”. They are “services”.

I was fortunate enough to spend some time with Dopplr’s Matt Biddulph and Matt Jones last week while in London where they described the architecture of what they’ve built in terms of connected data keys. The job of Dopplr, Mr. Jones said, was to “surface coincidence”.

I think that term slipped out accidentally, but I love it. What does it mean to “surface coincidence”?

It starts by enabling people to manufacture the circumstances by which coincidence becomes at least meaningful if not actually useful. Or, as Jon Udell put it years ago now when comparing Internet data signals to cellular biology:

“It looks like serendipity, and in a way it is, but it’s manufactured serendipity.”

All these services allow me to manage fragments of my life without requiring burdensome tasks. They all let me take my data wherever I want. They all enhance my data by connecting it to more data. They all make my data relevant in the context of a larger community.

When my life fragments are managed by an intelligent service, then that service can make observations about my data on my behalf.

Dopplr can show me when a distant friend will be near and vice versa. Twitter can show me what my friends are doing right now. Wesabe can show me what others have learned about saving money at the places where I spend my money. Among many other things Flickr can show me how to look differently at the things I see when I take photos. And del.icio.us can show me things that my friends are reading every day.

There are many many behaviors both implicit and explicit that could be managed using this formula or what is starting to look like a successful formula, anyhow. Someone could capture, manage and enhance the things that I find funny, the things I hate, the things at home I’m trying to get rid of, the things I accomplished at work today, the political issues I support, etc.

But just collecting, managing and enhancing my life fragments isn’t enough. And I think what Matt Jones said is a really important part of how you make data come to life.

You can make information accessible and even fun. You can make the vast pool feel manageable and usable. You can make people feel connected.

And when you can create meaning in people’s lives, you create deep loyalty. That loyalty can be the foundation of larger businesses powered by advertising or subscriptions or affiliate networks or whatever.

The result of surfacing coincidence is a meaningful action. And those actions are where business value is created.

Wikipedia defines coincidence as follows:

“Coincidence is the noteworthy alignment of two or more events or circumstances without obvious causal connection.”

This is, of course, similar and related to the definition of serendipity:

“Serendipity is the effect by which one accidentally discovers something fortunate, especially while looking for something else entirely.”

You might say that this is a criteria against which any new online service should be measured. Though it’s probably so core to getting things right that every other consideration in building a new online service needs to support it.

It’s probably THE criteria.

There was something really depressing about Web 2.0 Expo that I can’t quite put my finger on. Though when I woke up Monday morning after a weekend of working on my house it started to become more clear.

On Friday I prepared the work plan, rented a truck, and bought some new gear (loving the laser-guided cicular saw). On Saturday four of my friends came around to my house. We removed one wall and put up a new wall. On Sunday I hauled the junk to the dump, bought a bunch of sheetrock and more 2×4’s for next weekend’s job.

(By the way, great tip here, instead of hiring a garbage removal company for $700-1000, rent a 15 footer and just load all your trash directly into the truck bed. Drive it right into the dump yourself and push it out. The dumping cost for me was $75 plus the truck rental fee…which of course was super handy for getting the lumber, tools and sheetrock, too.)

Anyhow, now I have a 2 bedroom house where we once technically just had 1 bedroom. I also have a sore back and aching hands. Shredded skin on my fingers. A bruised elbow. Tired legs. I couldn’t be happier.

Struggling to get my body out of bed Monday morning, I realized I hadn’t thought about or even used the Internet for 3 days. I saved ideas by writing them down in a notebook with a pencil. I used the yellow pages to find things I needed. I contacted people using the telephone.

I wasn’t worrying about the scalability of the construction (only the joists over my workspace), optimizing the collaborative labor (except that they got coffee, food and beer), or marketing my property. I was simply building. With my hands (and a few borrowed ones).

I’ve argued over and over about how the Internet can change just about everything. And though I’m sure there are ways it could have helped me this weekend, there was something deeply satisfying about getting back to basics for a few days, particularly after losing the plot a bit last week.

Somehow the tone of the dialog in the Internet market has shifted away from the fundamentals, things like expanding the network or the concept of the network itself, building the tools and systems and data streams that help people accomplish things, creating the opportunities for new breakthroughs to emerge.

It’s a natural progression for a mature market to start optimizing for revenue gains as the platforms define themselves. I guess I’m just paranoid that the smell of instant fortune is wafting in the noses of sharks and leeches while the revenue models that they plan to exploit are emptier than they know. The spending arms race will surely follow where budgets won’t matter, hiring will get out of hand, and marketing messages will get silly.

But if the market crashes again or worse, violence or viruses erupt in our cities or the planet heats up, I’ll have my hammer ready for building things that people care about. That’s all I need. My trusty hammer. And my thermos. All I need is my thermos and my hammer, and maybe my chair…

A commenter in Mark Glaser’s recent post on MediaShift about the USA Today redesign sheds light on a problem that Internet companies seem to struggle with a lot.

“I think there may be a lesson to be learned in how to roll these things out. Most of the problems people are having are usability issues that it is nearly impossible for designers/developers who are in the weeds to notice.”

Similarly, Scott Karp asked the right question:

“Could it be that it’s really the social media revolutionaries who “don’t get it” when they assume that what the people want is to rise up against the media autocracy and take control, when in fact what most people want is to get high quality information from a reliable source?”

Unfortunately, even if you do the user research the recommendations of the studies often don’t fit into tight product release deadlines. And the studies often just support product direction rather than fully investigate a user need.

But the problem isn’t the research, it’s the product roadmap. In order to deliver a big punch in the market and cut through the noise, you need to be bold. And big changes that get noticed by big audiences require a lot of planning and complicated scheduling. Big changes are expensive on many levels.

But do you really need a big punch?

Most of my favorite online services tend to evolve organically as if responding to the way people are using the tools. Last.fm, for example, subtely rolls out new features that can occassionally have a significant impact on my usage. They had a pretty crappy web-based player for a long time. Of course, they upgraded it, as I knew they would, and I found it when it was relevant for me to look for it. There’s no amount of marketing they could have done to make me upgrade, and if they had done heavy marketing I might have actually been annoyed with them and considered a competitor.

The online media market is way too fickle to annoy your loyal customers.

But what about reaching new customers? Subtelty won’t win market share.

Admittedly, when you have a hammer everything looks like a nail, but the lessons of the web services market can be instructive. When you empower people to build businesses (or audiences) with your core offering, then you create a multiplier effect and reach all kinds of markets that you might never reach otherwise.

Winning market share in online media can happen by giving people the ability to distribute your offering for you, to create loyal customers for you out of their own customers, to build their own buzz for your product because they have an incentive for it to succeed.

Building the kind of passion required for a distributed customer model like this will never come from big bang marketing. It comes from fostering trustworthy relationships, establishing meaningful brands, proving tangible value, and responding quickly to market changes.

It’s not about noise. It’s about relationships.

I tend to agree with most online media insiders who appreciate the conceptual breakthrough for USA Today online and the balls to act on it, but I would be surprised if any of the positive comments in the blogosphere came from USA Today readers. And if USA Today damaged their relationship with their readers with this redesign, then they have made an incredibly costly mistake.

Online services need to roll out important new features constantly. But the days of hitting the market hard with a new product launch are fading. It works occassionally for major releases of things that are really new and require a reeducation of the market, like the iPhone. But fewer and fewer things fit into that category.

At the risk of invalidating everything I’ve said here by quoting a man who’s social and political beliefs go against just about everything I believe, Eric S. Raymond’s “The Cathedral and the Bazaar” included many astute observations about the way Linux development was able to scale so efficiently. Among the lessons is the classic “Release early and often” mantra:

“In the cathedral-builder view of programming, bugs and development problems are tricky, insidious, deep phenomena. It takes months of scrutiny by a dedicated few to develop confidence that you’ve winkled them all out. Thus the long release intervals, and the inevitable disappointment when long-awaited releases are not perfect.

In the bazaar view, on the other hand, you assume that bugs are generally shallow phenomena…or, at least, that they turn shallow pretty quickly when exposed to a thousand eager co-developers pounding on every single new release. Accordingly you release often in order to get more corrections, and as a beneficial side effect you have less to lose if an occasional botch gets out the door.”

Product Managers and Marketers need to bake these concepts into their thinking as well or risk missing the wider opportunity, the ultimate in marketing and distribution efficiency — customers as partners.

Photos: marble2, ccarlstead

I started messing around with Ruby on Rails for the first time on Sunday. This was after spending all day Saturday tearing down kitchen cupboards, tiled sinks and entire walls for a friend who is remodeling his house, so I got my fill of building last weekend whether real or virtual.

Photo: bruce grant

Trying to figure out how Ruby on Rails worked, I felt like I was remodeling my brain. It was as if I walked into Ikea with just a basic idea of what I wanted my new kitchen to look like and then walked out with design schematics and new appliances an hour later. I suddenly had confidence that I could create a really nice web site with a lot of functionality that was basically inaccessible to me before because of my limited programming background.

The “Ah hah!” moment came for me when I added two words to one of the scripts: “scaffold mydatabase”. When I refreshed my web site, I was adding, editing and deleting data in my database via a web interface. It all automatically just worked. Then literally 15 minutes later I had 2 databases talking to eachother.

It’s mindblowing how much power this environment gives to people who aren’t true coders.

I have a feeling I’ll get stuck and frustrated with what I’m trying to build. But I’m very hopeful Ruby on Rails will get me closer than I could with open source PHP tools. If nothing else, I’ll get a sense for this new trend.

Programming seems to have about a 3 year fashion cycle that also intersects with influxes of new ideas for web applications and a full cycle of students coming out of university. Now we’re at the early stages of a creative explosion on the Internet enabled by things like Rails, open APIs, storage solutions like S3, and JSON. And you can also wrap an idea in any number of different business models in even less time than it takes to build the product itself.

Maybe instead of LAMP (Linux, Apache, MySQL, PHP), we now have RASH (Rails, APIs, S3, Hosted).

There must be similar reactions to breakthroughs in the construction industry when things like cross-linked polyethylene (PEX) hit the market. Of course, construction suffers from bad naming as much as any other trade. Not everything can be as cool as a sawzall or funny pipe.

Photo: SimonD

I think everyone has had the experience of brainstorming a “great” dotcom business idea over drinks with friends. There were more than a few good and bad ideas thrown around the offices of The Industry Standard as the editors learned that the companies we were covering didn’t always deserve to make as much money as they did.

I had another one recently that was kind of interesting, though not really new in any way, so I thought I might just post it here and see if anything comes of it. I haven’t come up with a name for it, yet:

ProductX is a toy exchange network for parents of small children. It consists of a directory of toys available for members; a profile for each toy with ratings, comments, and user submitted videos; a queue for each member to track which toys will be shipped next; and a distribution system for receiving, storing and shipping toys. Anyone can become a member by contributing X toy(s) and paying a monthly fee. The toy contribution will be donated to charity. After joining, a member can select any 3 toys available from the directory. Members can only keep 3 toys at any one time and must return a toy to receive the next toy from this or her queue.

There was a variation on this theme that was also interesting that involved essentially ‘platformizing’ a swap system like this. That way you could focus member networks on specific types of products of similar value.

And just to put some perspective on my thinking here, these are some of the other ideas I’ve either heard, contributed to or invented myself (not telling which is which) and about when they were conceived:

• ThingThunk.com (2005). The ultimate product review directory. Inspired by Wikipedia, ThingThunk allows anyone to add a product, rate a product, comment, submit video, etc. It’s the ultimate shopping assistant where you can get advice from other people who have actually bought and used a particular product.
• Bioconomist.com (2005). The Industry Standard of the biotech industry. Insider analysis, heavy-hitting journalism, industry metrics, deal tracking and job listings.
• Makingamockery.com (2002). Jack Ass meets Gong Show online. People submit videos of themselves doing stupid things. Users vote for favorite stunts, but, more importantly, users can request stunts. Successfully performed requests win cash prizes and appear on the accompanying weekly TV show. (I’m now realizing as I type this that it’s more like Jack Ass meets Y! Answers.)
• PublishingPilot.com (2001). A UK-based CMS and hosting environment for small businesses to crank out web sites without having to know anything about anything. (I got some momentum behind this one including a somewhat high-powered board of directors before my personal bank account ran dry and MovableType appeared on the scene.)
• LargeLifeStyles.co.uk (2001). Fashion retailer for larger-sized women. (Actually, I pulled this one off using my father-in-law’s resources in textiles in London. We morphed it into a shoe retailer that performed mildly well for a few months until we stopped updating it with new product.)
• Sugarpimp.com (2000). Sponsored behavior modification. This idea arose out of my desire to keep a floater friend of mine unemployed and travelling so that I could continue to get his emails from strange places around the world. Then it hit me that you could create a market of sorts where you could get sponsors to fund all sorts of things. The obvious twist was that this would clearly turn into some kind of strange fetish/porn market.

And my personal favorite (courtesy of John Masterson)…

• MyBeltWithHooks (1999). An offline clothing personalization platform. The belt would be very expensive but any third-party developer could create hooks to attach to it. One example of a third-party hook is the real-world search engine, a small child or dog that would attach to your belt and go find things on command.