Today it’s easy to store and share my pictures, my favorite URLs, my thoughts and lots of other things online. There are a range of data repositories that allow me to do this kind of thing in different ways.

What still needs work is how I give trusted services access to much more private data — things like my current location, my spending behavior, access to my friends and family, etc.

To date, most services follow the premise that the looser the controls, the more fluidly data will travel. And that’s all that mattered when it was still hard to get data flowing.

Data flow is no longer an issue. Perhaps data flow has actually become too easy now. And therein lies the problem.

Clearly, blogging, RSS and feed readers drove a lot of the early thinking about syndication. Blogging enabled people to post content in a publicly accessible data repository somewhere for anyone to pull out without any privacy or permissioning controls. The further your content then syndicated, the better.

Wikis and community sites like Slashdot created a slightly more complex read/write dynamic against the central content repository that lots of people could access together. The permissioning model was essentially hierarchical where controls were kept in the hands of a smaller community.

Then Flickr broke ground with a new approach. They applied a user-centric friends and family relationship model to permissioning access to personal photos. Flickr opened up what was once considered private data and defaulted it to a public read-only permission status. But each individual still has a great deal of control over the data he or she contributes.

Similarly, del.icio.us made it possible to store and publicly address what had previously been private data. The nice twist here was the easy-to-understand URLs that allowed machines to consume, interpret and redistribute data stored in del.icio.us.

Where services like Facebook and Wesabe are now breaking ground again is in identifying a security model around highly sensitive data. Contact lists are very personal, but there aren’t many data sets more personal than my purchases and spending patterns.

Neat things can happen when I give machines access to my data, both the things I explicitly ‘own’ and my implicit behaviors. I want machines to act on my behalf and make my data more useful to me in a range of different contexts.

For example, I like the fact that Facebook slurps up my Twitter activity and shares it with my friends in the Facebook network. I don’t want to change my ’status’ on every service that shows status messages. Similarly, I like that Last.fm captures my listening behavior from iTunes and then uses that data to give back personal recommendations on a badge posted to my blog.

Allowing machines to automatically act on personal data on my bahalf is the right direction for things to go. But important questions need to be resolved.

For example, what happens to my data in all the places I’ve allowed it to appear when I change it? How do permissions pass from one service to another? How do I guarantee that a permission type I grant in one service means the same thing in another service? How do changes propagate? How does consent get revoked?

And even trickier than all that will be the methods for enforcing protection of privacy and penalties for breaking those permissions.

Until trust is measurable with explicit consentual triggers, loosely coupled networks that act on the data I wish to protect are going to struggle to talk to each other. Standards need to enable common sharing tactics. Responsibility needs to be clearly defined. And policies need to be enforceable.

Empowering a person to invest in storing and sharing the more sensitive data he or she owns is going to require a lot more than traditional read/write controls. But given the pace of change right now I suspect the answers will happen as the people behind these services work things out together before the industry taskforces, legal entities and blogosphere sort it out for them.

Jay Rosen has an interesting post on the failure of AssignmentZero, an effort to build a publicly funded crowdsourced news organization.

Among the many lessons, he keeps coming back to motivation and incentive.

“A well managed project correctly estimates what motivates people to join in, what the various rewards are for participants, and where the practical limits of their involvement lie.

…amateur production will never replace the system of paid correspondents. It only springs to life when people are motivated enough to self-assign and follow through.”

The idea wasn’t fundamentally broken, in my mind. Crowdsourced news is very powerful. As Derek Powazek said,

“At its best, crowdsourcing is about expanding the walls of the newsroom to the internet, giving an opportunity to people with real experience to share their expertise. This is a point that’s often lost on people who are just looking to make a quick buck on Web 2.0.”

More than anything else, I suspect that AssignmentZero failed because there weren’t any readers. Motivation wouldn’t have been a problem with a NYTimes-sized audience.

To date, I’ve never seen a better explanation of the motivations in collaborative online experiences than Yochai Benkler’s paper called Coase’s Penguin. One of my favorite excerpts from that is where he warns against paying for contributions from the community:

“An act of love drastically changes meaning when one person offers the other money at its end, and a dinner party guest who will take out a checkbook at the end of dinner instead of bringing flowers or a bottle of wine at the beginning will likely never be invited again.”

There are as many motivations as there are contributors in a shared media project. What holds them together is more art than science. Some of that art includes good timing and luck. But it also requires a unique kind of commitment and salesmanship from the leaders of the project.

I’ve begun to wonder if the tipping point happens when the confluence of the community size, the ROI to the contributors and the depth of the trust relationship with the company or the brand creates more value than the sum of the parts. Maybe the science of collaboration services can be found by quantifying the meaning of the relationships between those elements: size, cost, benefit and trust.

Or it could also be that the secret sauce inside the Craig Newmarks, Stewart Butterfields and Jimmy Waleses of the world is much more complicated and nuanced than anyone realizes.

I get Twitter now.

Until last week it seemed a bit silly to me, perhaps overhyped. But after using it to share updates of my son’s birth with friends and family members distributed across several time zones in near real-time, I’ve become a new fan of this fantastic tool.

Whereas I may have used email to announce his arrival before Twitter (something I also did after the fact), I was able to Twitter the experience of my son’s arrival throughout the day using my phone to simply send a little bit of info at a time via SMS.

Email would have been way too cumbersome for nearly live storytelling like this. Plus, the self-selective nature of it allowed some people to follow my posts who I probably wouldn’t have thought to email.

Flickr served a similar role for my daughter’s birth nearly 3 years ago, and it was invaluable to me again this time now that my mother and mother-in-law are both Flickr users finally. The photo-hungry grandparent is insatiable when it comes to newborns.

But Twitter adds a really nice new dimension to the way we share bits of our daily experience.

It was great knowing that my little brother in London and my older brother in Los Angeles were getting text messages on their phones as this major life event unfolded for me. Twitter made it feel like they were part of the experience, like bystanders, even if the details were as boring as where we ate dinner or what was on the TV in the hospital waiting room (Fresh Choice and Maury Povich, in case you’re interested).

Somehow I think the inability to share those inane details with the people we care about is exactly what makes people feel isolated in this modern distributed world. Well, maybe the world doesn’t need more meaningless data out there, but it certainly needs better ways to get the right data to the right people at the right time.

Twitter does just that.

The recent blog frenzy over hyperlocal media inspired me to have a look at Outside.in again.

It’s not just the high profile backers and the intense competitive set that make Outside.in worth a second look. There’s something very compelling in the way they are connecting data that seems like it matters.

My initial thought when it launched was that this idea had been done before too many times already. Topix.net appeared to be a dominant player in the local news space, not to mention similar but different kinds of local efforts at startups like Yelp and amongst all the big dotcoms.

And even from their strong position, Topix’s location-based news media aggregaton model was kind of, I don’t know, uninteresting. I’m not impressed with local media coverage these days, in general, so why would an aggregator of mediocre coverage be any more interesting than what I discover through my RSS reader?

But I think Outside.in starts to give some insight into how local media could be done right…how it could be more interesting and, more importantly, useful.

The light triggered for me when I read Jon Udell’s post on “the data finds the data”. He explains how data can be a vector through which otherwise unrelated people meet eachother, a theme that continues to resonate for me.

Media brands have traditionally been good at connecting the masses to eachother and to marketers. But the expectation of how directly people feel connected to other individuals by the media they share has changed.

Whereas the brand once provided a vector for connections, data has become the vehicle for people to meet people now. Zip code, for example, enables people to find people. So does marital status, date and time, school, music taste, work history. There are tons of data points that enable direct human-to-human discovery and interaction in ways that media brands could only accomplish in abstract ways in the past.

URLs can enable connections, too. Jon goes on to explain:

“On June 17 I bookmarked this item from Mike Caulfield… On June 19 I noticed that Jim Groom had responded to Mike’s post. Ten days later I noticed that Mike had become Jim’s new favorite blogger.

I don’t know whether Jim subscribes to my bookmark feed or not, but if he does, that would be the likely vector for this nice bit of manufactured serendipity. I’d been wanting to introduce Mike at KSC to Jim (and his innovative team) at UMW. It would be delightful to have accomplished that introduction by simply publishing a bookmark.”

Now, Outside.in allows me to post URLs much like one would do in Newsvine or Digg any number of other collaborative citizen media services. But Outside.in leverages the zip code data point as the topical vector rather than a set of predetermined one-size-fits-all categories. It then allows miscellaneous tagging to be the subservient navigational pivot.

Suddenly, I feel like I can have a real impact on the site if I submit something. If there’s anything near a critical mass of people in the 94107 zip code on Outside.in then it’s likely my neighbors will be influenced by my posts.

Fred Wilson of Union Square Ventures explains:

“They’ve built a platform that placebloggers can submit their content to. Their platform “tags” that content with a geocode — an address, zip code, or city — and that renders a new page for every location that has tagged content. If you visit outside.in/10010, you’ll find out what’s going on in the neigborhood around Union Square Ventures. If you visit outside.in/back_bay, you’ll see what’s going on in Boston’s Back Bay neighborhood.”

Again, the local online media model isn’t new. In fact, it’s old. CitySearch in the US and UpMyStreet in the UK proved years ago that a market does in fact exist in local media somehwere somehow, but the market always feels fragile and susceptible to ghost town syndrome.

Umair Haque explains why local is so hard:

“Why doesn’t Craigslist choose small towns? Because there isn’t enough liquidity in the market. Let me put that another way. In cities, there are enough buyers and sellers to make markets work – whether of used stuff, new stuff, events, etc, etc.

In smaller towns, there just isn’t enough supply or demand.”

If they commit to building essentially micro media brands based exclusively on location I suspect Outside.in will run itself into the ground spending money to establish critical mass in every neighborhood around the world.

Now that they have a nice micro media approach that seems to work they may need to start thinking about macro media. In order to reach the deep dark corners of the physical grid, they should connect people in larger contexts, too. Here’s an example of what I mean…

I’m remodeling the Potrero Hill shack we call a house right now. It’s all I talk about outside of work, actually. And I need to understand things like how to design a kitchen, ways to work through building permits, and who can supply materials and services locally for this job.

There must be kitchen design experts around the world I can learn from. Equally, I’m sure there is a guy around the corner from me who can give me some tips on local services. Will Architectural Digest or Home & Garden connect me to these different people? No. Will The San Francisco Chronicle connect us? No.

Craigslist won’t even connect us, because that site is so much about the transaction.

I need help both from people who can connect on my interest vector in addition to the more local geographic vector. Without fluid connections on both vectors, I’m no better off than I was with my handy RSS reader and my favorite search engine.

Looking at how they’ve decided to structure their data, it seems Outside.in could pull this off and connect my global affinities with my local activities pretty easily.

This post is way too long already (sorry), but it’s worth pointing out some of the other interesting things they’re doing if you care to read on.

Outside.in is also building automatic semantic links with the contributors’ own blogs. By including my zip code in a blog post, Outside.in automatically drinks up that post and adds it into the pool. They even re-tag my post with the correct geodata and offer GeoRSS feeds back out to the world.

Here are the instructions:

“Any piece of content that is tagged with a zip code will be assigned to the corresponding area within outside.in’s system. You can include the zip code as either a tag or a category, depending on your blogging platform.”

I love this.

30Boxes does something similar where I can tell it to collect my Upcoming data, and it automatically imports events as I tag them in Upcoming.

They are also recognizing local contributors and shining light on them with prominant links. I can see who the key bloggers are in my area and perhaps even get a sense of which ones matter, not just who posts the most. I’m guessing they will apply the “people who like this contributor also like this contributor” type of logic to personalize the experience for visitors at some point.

Now what gets me really excited is to think about the ad model that could happen in this environment of machine-driven semantic relationships.

If they can identify relevant blog posts from local contributors, then I’m sure they could identify local coupons from good sources of coupon feeds.

Let’s say I’m the national Ace Hardware marketing guy, and I publish a feed of coupons. I might be able to empower all my local Ace franchises and affiliates to publish their own coupons for their own areas and get highly relevant distribution on Outside.in. Or I could also run a national coupon feed with zip code tags cooked into each item.

To Umair’s point, that kind of marketing will only pay off in major metros where the markets are stronger.

To help address the inventory problem, Outside.in could then offer to sell ad inventory on their contributors’ web sites. As an Outside.in contributor, I would happily run Center Hardware coupons, my local Ace affiliate, on my blog posts that talk about my remodelling project if someone gave them to me in some automated way.

If they do something like this then they will be able to serve both the major metros and the smaller hot spots that you can never predict will grow. Plus, the incentives for the individuals in the smaller communities start feeding the wider ecosystem that lives on the Outside.in platform.

Outside.in would be pushing leverage out to the edge both in terms of participation as they already do and in terms of revenue generation, a fantastic combination of forces that few media companies have figured out, yet.

I realize there are lots of ‘what ifs’ in this assessment. The company has a lot of work to do before they breakthrough, and none of it is easy. The good news for them is that they have something pretty solid that works today despite a crowded market.

Regardless, knowing Fred Wilson, Esther Dyson, John Seely Brown and Steven Berlin Johnson are behind it, among others, no doubt they are going to be one to watch.

Sometimes I forget to step outside of our little bubble here and see how people use or in fact don’t use the Internet. When I get that chance I often wonder if anything I’m doing in my career actually matters to anyone.

Usually, however, I’m reminded that even though the Internet isn’t weaved into every aspect of everything, it has great potential in places you might not consider.

For example, I’ve been remodelling my house to make room for a new little roommate due to be delivered in September. I’m trying to do most of the work myself or with help from friends and neighbors. I’m trying to save money, but I also really enjoy it. It’s a fantastic way to reconnect with the things that matter…food, shelter, love and life.

Well, I made the mistake of working without permits fully aware that I probably should have them. It’s my natural inclination to run around bureaucracy whenever possible.

As luck would have it, just as the pile of demolition debris on the sidewalk outside my house was at its worst, a building inspector happened to drive by on his way to another job. He asked to see my permit to which I replied, “The boss isn’t here. Can you come back later?”

The building inspector just laughed. After pleading a bit and failing, I started making calls to get drawings and to sort out the permits.

It was at this moment I realized how much building planning and construction could benefit from the advances made in the Internet market the last few years. The part of construction that people hate most is the one that is perhaps the most important. And it is this part that the Internet is incredibly well-suited to improve.

Admittedly, the permit process was not actually that painful and relatively cheap, too. I have spent in total maybe 1 day dealing with permits and drawings, so far, with a bit more to come, I’m sure.

But the desired effect of permitting jobs is sorely underserved by its process.

At the end of the day what you want is the highest building quality possible. You want builders using proven methods with at least semi-predictable outcomes. You want to make sure nobody gets hurt. And you want incentives for people to share expertise and information.

Rather than be a gatekeeper, the city needs to be an enabler.

One of the brochures I read called “How to Obtain a Permit” includes a whitelist of project types. I’m apparently allowed to put down carpets and hang things on my walls without a permit. Glad to know that.

Strangely, after explaining all the ways the city asserts itself into the process, on the very last page of the brochure it then says, “Remember, we are here to assist you. If you have any questions about your project, please give us a call!” I didn’t meet one person in the 6 queues I waded through the first morning who wanted to help me. They were mostly bored out of their brains.

Instead, the city should be putting that brainpower to work finding ways to lubricate conversation and collaboration around solving building problems. If the building community was in fact a community powered by thoughtful city-employed engineers, then I would be much more interested in working with them. I might even become dependent on them.

For example, if they helped me organize, store, print and even share my plans, then I’d be more than happy to let them keep my most current drawings, the actual plans I’m using to build with. If they could connect me to licensed contractors and certified service providers, I’d gladly give them my budget.

As it stands, my incentive is to avoid them and hide information whenever possible.

Imagine if I was able to submit a simple SketchUp plan to a construction service marketplace. I could then sit back and watch architects and interior designers bid for the planning work. My friends in the network could recommend contractors. Tools and parts suppliers could offer me discounts knowing exactly what I needed for the job. I could rate everything that happens and contribute to the reputation of any node in the ecosystem.

Imagine how much more value would be created in the home buying market if a potential buyer could see all this data on a house that was for sale. I might be able to sell my home for a higher price if my remodel was done using highly reputable providers. There would be a financial incentive for me to document everything and to get the right certifications on the work.

Imagine lenders knowing that I’m an excellent remodeller based on my reputation and sales track record. I might be able to negotiate better terms for a loan or even solicit competing bids for my mortgage on the next house I want to invest in.

At every step in the process, there is a role for the city government to add value and thus become more relevant. Then the more I contribute, the more it knows about what’s happening. The more it knows, the more effective it can be in driving better standards and improving safety and legislating where necessary.

My mind spins at the possibilities in such a world. Of course, when you have a hammer everything looks like a nail. But it seems to me that the building permit and inspection business is broken in exactly the places that the Internet is more than capable of fixing.