Decisions made by businesses today can have a dramatic impact on their performance tomorrow – and certainly for a long time to come. Unfortunately, there is no way for an executive or board of directors to be sure that their decisions are the right ones. After all, no one can know what the future will bring, unless they happen to have a crystal ball that works better than everyone else's.
Definitive prophesy is one thing, but thanks to advanced predictive analytics, we can now get important insights into what to expect in the future, even if we do not have the services of an in-house psychic. A technology growing in popularity – more than half of large companies are already using it to some extent – predictive analytics parses huge amounts of data about current or past events, cultural memes, individuals, governments, or anything else, and makes a highly-educated guess as to what is going to happen.
Of course, the success of such predictions depends on the quality and accuracy of the data being analysed. But, given the large amount of material out there, and the advanced algorithms, web crawling can acquire the data to perform deep-dive analyses that will extract actionable insights about a large variety of issues and topics. Some examples will illustrate the point:
Big data predictive analytics to predict political outcomes
In 2015, polls predicted that the Likud party of Israeli Prime Minister Benjamin Netanyahu would lose the office he had held since 2009. According to surveys conducted by Israel's top pollsters, the Likud was set to lose key seats in Israel's parliament, leaving Netanyahu without enough seats to form a coalition.
The poll results were sufficient to set off alarm bells in the Likud; during the last days of the campaign, Netanyahu ran ads aimed at convincing voters of other right-wing parties to switch their votes to enable him to get enough seats to form a coalition. But at least one organisation – Israeli business daily publication Globes – came to a different conclusion, based on analysis of news stories, forums, blog posts, and other content. Taken altogether, the data provided what turned out to be a much more accurate analysis of the election results – and concluded that it would be Netanyahu's party that would win the election by a relatively wide margin.
Using blog posts to predict movement in the Chinese stock market
Stock prices are driven by many factors, from the shape of the economy, to computer-driven trading, to institutional investment decisions. And, it’s driven, at least in part, by investor sentiment. That sentiment is often out there for everyone to see – on blog posts, investment forums, comments on news articles, and other vehicles where investors express their opinions. It stands to reason that harnessing that sentiment could provide an indication on what direction a specific stock, or an entire market, will take.
There have been several efforts to test that theory on stock markets, mostly in the U.S. - but in a recent study, a group of American and Chinese researchers used predictive analytics techniques to determine how to invest in the Shanghai stock exchange. According to the group's research paper, whose data they collected data from microblogs, chat rooms, and web forums, they discovered that “data from these sites exhibit distinct characteristics in activity level, post length, and correlation with stock market behavior.”
The team “investigated several machine learning models to classify post sentiment in chat rooms, and achieved a performance similar to the state-of-the-art sentiment analysis result for short posts. We find that there is strong correlation and Granger causality between chat room post sentiment and stock price movement, indicating that post sentiments can be used to improve the prediction of stock price return over using the historic stock trading information alone.” According to the team, their method yielded a nearly 20 per cent profit on investments that were based on the analysis, compared to a 25 per cent loss for those who used “a passive buy-and-hold baseline strategy.”
Analytics to power a SaaS financial platform
SESAMm is a fintech startup based in France and Luxembourg. Its SaaS platform, which was launched in 2014, uses predictive analytics to make recommendations to professionals at the world's largest financial institutions – among them La Francaise, a large investment group in France, and Nikko Global Wrap, a major financial institution in Japan. The company is also currently in talks with some of the largest hedge funds in the US.
SESAMm provides analytics based on textual data from the web - such as social media, financial data and other sources, including current data, and data going back three years. The SESAMm platform sweeps online sites where investor sentiment is stored – blog posts, news articles, forums, etc. - and, using advanced analytics, builds a sentiment, emotion and opinion score, which their clients use to gauge investment opportunities in over 10,000 different assets, including listed companies, commodities and cryptocurrencies. According to SESAMm, traders and institutions have benefited greatly from the platform, thanks to the large amount of data that would usually have gone untapped and unanalysed.
A system like this could be of great benefit in a wide variety of areas, drawing data from publicly available as well as proprietary sources. Historical stock prices correlated with market conditions, world events, and other relevant factors, as compared to the situation today, could yield insights on where the market, or even the price of a particular stock, is going; data on weather systems and forecasts, atmospheric conditions, tides, or any other relevant information could give farmers a better idea of when and what to plant; information relating to work trends, transportation, accident rates, and public sentiment about routes could give insurance companies and edge in setting risk.
And so on; medicine, industry, marketing, and many other areas will benefit from predictive analytics. The future is, as they say, a closed book, but with big data and artificial intelligence, we can get a peek at some of highlights of that book – before it gets opened.
Guy Mor, Co-founder and CMO, Webhose
Image Credit: Totojang1977 / Shutterstock