Text mining can help forecast stock market movements (Image: Yoshikazu Tsuno/AFP/Getty Images)
Read more: “Seven technologies to disrupt the next decade“
For around 20 years, starting in the 1980s, the sought predictions from people considered knowledgeable. His experts, 280 of them, were the kind of folk who, in their work as TV pundits or government advisors, opined on matters such as the rise of China or security in the Middle East. As time passed, he checked their forecasts. The results were dismal. “Human beings who spend their lives studying the state of the world… are poorer forecasters than dart-throwing monkeys,” wrote .
Not so for a powerful new method of forecasting called “text mining”. It draws on the vast amount of data available online. By sampling the sentiments expressed in the torrent of blog posts, tweets and Facebook updates, you can gain unprecedented insights into the mood of the world and use it to predict what is to come.
Researchers have already developed powerful enough to improve forecasts of . Others have used Google search queries as a forecasting tool. Many searches for certain job-related terms, for example, indicates that unemployment is rising.
That’s just the beginning. Several companies are now archiving whole swathes of the internet in a bid to create more powerful forecasting. WiseWindow, based in Irvine, California, claims to monitor opinions expressed by over 77 million people on Facebook and other social media sites. The firm mines the data for clues to consumer sentiment and emerging trends, which companies buy in the hope of…
