Building communities from Twitter posts

So then I wondered whether there was a way to capture, prioritize and then syndicate the best Twitter posts into a ‘kiddie quote of the day’ or something like that. #

  1. Get the quotes: I ran some searches through Twitter Search and collected the RSS feeds from those results to create the pool of content to use for the project. In this case, I used ‘daughter said‘ and ‘son said‘. I put those feeds into Yahoo! Pipes and filtered out any posts with swear words. Then I had a basic RSS feed of quotes to work with.
  2. Prioritize the quotes: I’m not sure the best way to prioritize a collection of sources and content, but the group voting method may do what you want. Jon Udell has another approach for capturing trusted sources using Del.icio.us. For voting, there’s an open source Digg clone called Pligg. I set it up on a domain at Dreamhost (I called it KidTwits…Dreamhost has a one-click Pligg installer that works great) and then pumped the RSS feed I just made into it. In no time I had a view into all the Twitter posts which were wrapped in all the typical social media features I needed (voting, comments, RSS, bookmarking, etc.).
  3. Resyndicate the quotes to Twitter: While you might be able to draw people into the web site, it made more sense in this case to be present where interested people might be socializing already. First, I created a Twitter account called KidTwits. Then I took a feed from the web site and sent it through an auto-post utility called twitterfeed. Now the KidTwits Twitter account gets updated when new posts bubble up to the home page of kidtwits.com.
  4. Link everywhere possible: When building the feed into Pligg I made sure that the twitter ID of each post was captured. This then made it possible to “retweet” with their IDs intact. Thus, the source of the quote would then see the KidTwit posts in their Twitter replies. It works really well. People were showing up at the web site and replying to me on Twitter the same day I began the project. Again, I used Yahoo! Pipes to clean up and format the feed back out to Twitter to include the ‘RT’ and @userid prefix to each entry. I played around a bit before arriving at this format.

    I also included a Creative Commons copyright on all the pages of the web site to make sure the rights ownership issues were clear. #

  5. Improve post quality: Now, here’s where things have been failing for me. I can’t think of better search terms to capture the pool of quotes I want, but there are so many extraneous Twitter posts using those words that it seems like I’m getting between 5% and 10% accuracy. Not bad, but certainly not good enough. The good news is that it’s pretty easy to kill the posts you don’t want through the Pligg interfaces. I just don’t have the time or desire to maintain that.
  6. Optimize the site: I then did a bunch of the little things that wrapped up the project. I added Google Analytics tracking, created a simple logo and favicon, customized the Twitter background, and configured Pligg to import the Twitter Search pipe automatically.
There are several things I like and a few I dislike about this little project. #

  • I really like the fluidity of Twitter’s platform. It’s amazingly easy to capture and resyndicate Twitter posts.
  • I love the effects of the @reply mechanism. I can essentially notify anyone who gets their Twitter post listed on the home page of kidtwits.com without lifting a finger. And they get credit automatically for their post.
  • I already knew this, but Yahoo! Pipes is just brilliant. I can’t imagine I would have even considered this project without it.
  • Pligg is pretty good, too. It does everything I want it to do.
  • I would love to hand over the management of the voting and quality checks to someone else. Voting naturally invites gaming. At the end of the day, however, the quality control and community management function is what makes a community service interesting to people. You can’t automate everything.
  • I’m actually not a fan of voting approaches to prioritizing content. It will ultimately result in dumbing down the quality. That’s less of an issue for highly niched topics like this, though.
  • The rights issues are a little weird. This wouldn’t be a problem in forming a community whose purpose is noncommercial naturally. But I’m not sure the Twitterverse would respond well to aggregators that make money off their posts without their knowledge or consent. (To be clear, KidTwits is not and never will be a commercial project…it’s just a fun experiment.)
  • Auto-retweeting feels a bit wrong. I wouldn’t be surprised if the KidTwits account gets banned. But I have explicitly included the source and clearly labeled each Twitter post with ‘RT’ to be clear about what I’m doing. I’m not building traffic to my account, the web site, nor am I intentionally misrepresenting anything.
  • By adding “RT @userid” I’ve killed the first 10 or so characters of the post that I’m retweeting. This means the punchline is often dropped which kills the meaning of the retweeted post.
  • Some conversational Twitter posts get through which include @replies to another user. When the KidTwits retweet of that post goes out it’s very confusing.
The potential here, among other things, is in creating cohesive topical communities around what people are saying on Twitter. You can easily imagine thousands of communities forming in similar ways around highly focused interest areas. #

10 thoughts on “Building communities from Twitter posts”

  1. Promote the use of a hashtag to everyone you find and then include the hashtag. Over time, your hit rate will rise, or you will get enough hashtags that you can drop the search.

  2. Thanks for the write-up, Matt. A cool read.

    To get a higher quality content pull, perhaps search for “‘son said’ AND ‘RT'” so you end up with the subset that somebody else has already “voted” for. Guess it might shrink the pool too much, but it would take advantage of the RT = voting nature already emerging on Twitter.

    The #kidtwits hashtag idea isn’t bad either, but you’d look even more of the 140 chars.

    Thanks again for sharing.

    Thanks,
    Nate

  3. I love this idea – I’ve gotten many a chuckle from “kid twits”. May I suggest you try adding “toddler” to the mix? You’d pick up gems like this one:
    wryredhead Dog pooped in the corner where Xmas tree previously stood. Toddler cried: Christmas tree GONE! POOP! Pretty much sums today up for everyone.

    @sarahebourne

  4. Really cool, Matt. For a more automated approach and less coding, you could set something up with my SocialSite widgetmaker. (ping me for an invite code)

    You can set up custom Twitter searches, but also hit all the other major web2 feeds and API’s — flickr, youtube, delicious, etc.

    Right now, content comes back in widget. Soon, many more killer features. Hope you like. -Mike

    PS — I love what you did with the ratings, gotta ad something like that to my system

  5. Got to spend some more time on KidTwits tonight.

    Interesting to me that you don’t like the quality of the posts, is that because they’re not all necessarily funny? Because I think your system catches a lot of pretty good content, it seemed to match the description on the site quite well, “children’s best quotes and phrases every day from twitter.”

    Have you thought about pulling in blog posts? Maybe a Google Blog Search feed for exact phrase like “my son said”?

    Google Blogs is probably actually worse than Twitter in terms of quality, there may be several ways to extend the site/idea through other open communities, but it seems the search capability has to be really precise or people need to do a lot of curating for something this niche and yet abstract. Great post and very cool experiment, thanks Matt.

  6. Like the blog search idea, Mike. On a quick scan I haven’t found a good slice of blog post content that would work, but there’s probably a way to make that work somehow.

    The quality problems with Twitter seem to come in waves. It wouldn’t be much work to monitor what comes in, but somebody needs to go in there and kill the totally irrelevant stuff which is well over half the posts, perhaps as much as 60-70%. Then there’s another 20-30% that isn’t about things that kids have said. It just needs a human touch.

  7. Hi Matt,

    First of all, kidtwitts doesn’t work 🙁 You said: “I set it up on a domain at Dreamhost […] and then pumped the RSS feed I just made into it.” Could you explain how you did it, please? Congratulation for your post. It’s really inspiring 🙂 Thanks in advance

Comments are closed.