Something really interesting is happening to Twitter. What some people may see as an ocean of facile, pointless babble, is in fact evolving into powerful tool for discovering real-time insight into public opinion, which can be used to predict things such as the success of upcoming movie releases, stock market movements and even election results.
The technology is still in its infancy, but the promise is very alluring and there are a number of good reasons that Twitter lends itself well to this kind of experiment.
Brevity
Tweets are short enough to be analysed by a computer with relative accuracy and speed. In a 500 article there’s plenty of room for ambiguity and flowery language, making it hard for any algorithm to extract meaning. But the 140 character limit forces people to keep their tweets short and to the point, making the analysis much easier.
Immediacy
People share their thoughts on Twitter quickly and without the same level of mental filtering that would go into a blog post. You use Twitter to say what’s on your mind right now, but for a blog post you spend some time mulling the topic over and refining your thoughts before you share them. Therefore, I would argue, Twitter not only provides a more instant picture of the world’s opinions, but also a more honest one.
Volume
Even if just 5% of Twitter’s 175 million+ registered users are active, that’s still a pretty substantial sample size to collect data from.
What this means is that with the right kind of algorithm, you can mine Twitter for sentiment/opinions on any given topic, such as the title of a new movie, and paint a fairly accurate picture of how the world feels about it (or even just learn that nobody’s talking about it at all). Fflick.com is already doing this to provide movie ratings based on the overall sentiment of Twitter conversations.
And, as with the stock market experiment, you can do interesting things like compare the general sentiment of Twitter users against things like index movements and identify correlations between the two. If you know that when Twitter users in London feel happy on Monday there’s an 80% chance that the market will rise on Wednesday, that’s clearly a very powerful insight.
Schrodinger’s tweet
But where is this taking us? Obviously if Twitter proved to be an accurate instrument for predicting stock market movements, that would soon enough change the way people trade stocks and it would only be a matter of time before the prediction no longer worked. If people know the market is going to fall tomorrow, why would they buy today? The very act of making the prediction changes the outcome.
And there’s the problem of confusing cause with correlation. Is nobody tweeting about the new George Clooney film because nobody wants to see it, or is nobody going to see the film because not enough people are talking about it?
It’s inevitable that as this technology develops we’re going to see new services making all sorts of promises about what kinds of things they can predict and how accurately they can predict them. The only prediction I’m prepared to make is that once this happens, as with everything else involved in social media, the market will consist of about 10% genuine innovation and 90% snake-oil. The hard part for marketers will be telling the difference between them.