There’s been a lot of hype lately about “realtime search” using Twitter being a Google Killer. John Battelle talked about it in searchblog. Mike Arrington talked about it in TechCrunch.

There are two scenarios that have been talked about with Twitter and search: using Twitter to ask questions of the Twitterverse and using Twitter search to search Tweets.

In the first scenario, you send out a Tweet looking for information. An example is a Tweet by Om Malik on Feb. 2 at 7:02 p.m. “suggestion for great Indian restaurant in or around Palo alto. needs to be authentic”. Within a few minutes, he got a bunch of responses. (I recommended Amber India in Mountain View, which was a frequent recommendation.) By 8:19 p.m. Om was “eating at amber India in mountain view.”

Wow! Send out a query and you can get answers from real people right away. Who wouldn’t want that?

Not so fast.

This reminds me of latenight TV commercials for miracle diet drugs. You see pictures of people who have: lost 75 pounds in 8 weeks! lost 10 pounds overnight! gone from a size 24 to a size 6 while eating cake and sitting on the couch! You usually see a line of fine print that says “results not typical.”

That’s exactly the case here. You get atypical results when you have tens of thousands of followers as Om (23,000+) and Battelle (11,000+) do.

I posted a Tweet yesterday “Looking for good wineries in napa. Focus on ambiance and red wines.” To give the Tweet extra chance of success, I posted it using Twinkle, an app that adds a location layer to Twitter. Given the nature of my query, my friend network and location in the Bay Area, I expected success. At 200+ followers, my follower count is well above the median for Twitter.

Three hours later, I got one response from a friend. By that time I’d already decided which wineries I was going to visit. Hardly a Google killer.

Suppose for a moment that I just picked a tough query. What if I’d gotten a dozen responses?

Then the problem becomes how I decide which of those responses are better than the others. Many queries have qualitative components: “What’s a good winery in Napa?” “Is the Nikon D80 a good camera?” The value of the answer depends on my needs as well as the expertise of the answerer.

With a typical search result, you have a number of clues as to quality of the answerer. If something appears on the first page of Google, presumably a lot of people have found that resource valuable. If I get a page from dpreview, I can see that they’ve reviewed hundreds of other cameras, so they probably know what they’re talking about.

With Twitter answers, I get limited information about the source and limited content.

In some cases, this is OK. I did a Tweet a while ago wondering if it was sunny at the Beach Chalet in San Francisco. You’d have to be a jerk to lie about the answers, there’s not much expertise required to answer the question and the answer fits within 140 characters.

But a query like “Is the Nikon D80 a good camera?” is tougher. If @maryvale says “yes, absolutely” then that’s all I need. I know her, I love her photography and I know she knows a lot about cameras. That doesn’t hold true for most of my other followers. And it certainly doesn’t hold true for people I don’t know at all. Someone may say “D80 is a piece of crap” because they would never consider anything less than the $2,000+ D700 or because they aren’t very technically savvy. A Tweeted answer doesn’t provide that context.

Part 2: Challenges of searching Twitter

More on: Twitter, Google