hrishi.me

Naive Applications of AI: Sentiment Analysis in Trading

November 18, 2018 | 8 min read

atvi drop chart

The Announcement

Over the weekend of November 2nd, Activision:ATVI dropped 6.7% after Blizzard announced Diablo: Immortal, a mobile adaptation of their popular Diablo franchise. Blizzard are developing this game in partnership with NetEase, a Mobile Games developer based in China. NetEase is a strong player in the mobile space, having previously ported and launched some of the most popular mobile games in China and around the world.

But Blizzard and Diablo fans were not happy with the announcement, and were quite vocal about it. The reasons behind the fan uproar had little to do with the fact that Blizzard was making a mobile game. For those unfamiliar with video games in general, or with Blizzard and Diablo specifically, I’ll explain.

The background is a bit meandering and you can click here to skip to the analysis and discussion.

Fan backlash: background

At the time of BlizzCon 2018, Diablo III had not received any new content for a year. This led to fan speculation that Blizzard was working on a successor to Diablo III. When Blizzard released the schedule for BlizzCon 2018, they listed a panel named ‘Diablo: What’s Next?’ immediately after the opening ceremony.

The schedule all but confirmed fan speculation that Diablo 4 would be announced at BlizzCon 2018, aided in no small part by voices in video games journalism.

atvi schedule

When Diablo: Immortal was announced, the initial fan reaction was disappointment and disbelief that the game they were expecting was not being worked on. The announcement video on youtube received nearly half a million dislikes. During the ‘Diablo: What’s Next?’ panel, an incredulous fan asked if the announcement was an April Fool’s Joke. The panelists on stage, having sensed the disappointment in the crowd, asked if the fans didn’t have phones. This only made things worse.

Clearly, Blizzard as a company was excited about this new addition to their game franchise, and, equally, they had failed to anticipate the fan speculation and subsequent reaction. They saw nothing wrong with their announcement because it was truly the most exciting thing they had done in years. On stage, minutes after the BlizzCon opening, they couldn’t see why fans were upset, perhaps because they had their business-owner hats on, and not their customer-service hats.

When the online conversations settled, it became clear that fans were not upset because Blizzard was making a mobile game. Fans were upset mainly because Blizzard did not announce the game they were expecting to hear about. The negative sentiment was real, but it was about a PR mistake and had almost no prospective effect on the company’s outlook. These fans are waiting for Diablo 4, and they will buy it when it’s released.

But is it really that bad?
Wall Street analysts disagree

From an investor point of view, entering the mobile market, supported by an Asian mobile giant is a very good thing. Analysts are bullish, despite the recent disappointment from Call of Duty: Black Ops 4 sales:

Morgan Stanley’s Brian Nowak suggested “Diablo Immortal” could eventually have more than 200 million monthly active users, and that it could generate annual earnings of as much as $2.52 a share.

Wedbush, one of 20 sell-side firms that carry a buy-equivalent rating and also holds a Street-high 12 month price target of $100, estimated the game could contribute annual revenue of up to $300 million after its roll out, and that it “should expand the franchise’s audience to hundreds of millions of players.”

The partnership with NetEase, analysts added, “signals that Blizzard seeks success in western and eastern markets, with gameplay elements for core ethusiasts and the previously-uninitiated.”

“We expect Activision Blizzard to outpace its peers with its in-game monetization, and expect dramatic growth in its mobile business as it launches new titles based upon its successful PC and console games,” Wedbush wrote.

Source: Bloomberg (https://www.bloomberg.com/news/articles/2018-11-05/activision-analysts-see-china-growth-from-diablo-mobile-game)

Adding a whole new customer segment and entering a new marketplace, as well as a new market vertical, is not something that happens frequently, and is generally regarded very positively by shareholders.

So, how much influence did fan sentiment really have on the stock price?

Motley Fool explains the weekend shedding as ‘nerves’ related to the upcoming Q3 results. In my opinion, this does not fit with the all-time high on October 1st, just a month before, followed by drops linked specifically to misreported CoD first day sales, and subsequently when compound sales were reported.

atvi trend chart

I think The Fool’s analysis misses the smaller picture. There’s likely a better reason it happened on that day and not a day before or after. Let’s take a step toward the topic of this blog: Applications of sentiment analysis.

Sentiment analysis in trading

Sentiment analysis, sometimes also called ‘opinion mining’ is a process applied most commonly to social media, to identify and track public sentiment surrounding a brand or a topic. When it was invented, simple, knowledge-based algorithms were used to identify simple sentiments based on pre-defined phrases and keywords.

With the rising popularity of AI, Natural Language Processing (NLP) techniques have been applied to sentiment analysis, leading to improvements in automation, and an increased range of identifiable emotions. There is a lot of interest in applying AI to sentiment analysis, especially from academics. Applying sentiment analysis to stock market predictions is a popular academic exercise:

The interest in applying AI sentiment analysis to social media is not limited to academia. Sentiment analysis is already being applied to automated stock trading. It is popular enough that there are studies on the impact of social media sentiment on stock prices.

we find a significant dependence between the Twitter sentiment and abnormal returns during the peaks of Twitter volume. This is valid not only for the expected Twitter volume peaks (e.g., quarterly announcements), but also for peaks corresponding to less obvious events

Source: https://www.researchgate.net/publication/282049046_The_Effects_of_Twitter_Sentiment_on_Stock_Price_Returns

We have reached a point where social media sentiment affects stock prices directly, and is no longer just an indication of a stock’s future performance.

Theory-crafting

Considering all this, I now wonder if sentiment analysis was the larger driving force behind the price drop for ATVI? Based purely on social media sentiment, ATVI was certainly going to perform poorly. That is, until you read into the sentiment and understood that it was not only temporary, but also lacking any significant impact on the short term or long term value of the company.

Sentiment was negative not because a company made a poor decision, but because it announced a good decision on the wrong platform, and poorly.

How can a sentiment analysis AI effectively differentiate between a temporary, inconsequential trend of negative sentiment and the “real thing”? Is it even possible to do so? How can we solve this problem?

To be honest, I do not have the answers in this post.

When professional analysts fail to see the reasons behind the backlash, an AI has little hope of performing better in this highly subjective field.

What does this teach us?

Improving the AI is not the only option we have. So long as we understand how well the AI performs, and how frequently it makes mistakes, we can improve the process utilizing the AI to accommodate and overcome these shortcomings.

AI is cheap to implement today. We must remember that the consumption of public data, data which anyone can manipulate easily, must be done with caution. If our AI is using social media to make decisions about automatically buying and selling shares, then we can be sure that bad actors are also using AI to manipulate social media to take advantage of predictable changes in stock prices.

When public data-based systems are part of our core business, we have to ensure there are checks in place - either manual or automated - which validate surge inputs with human intervention, or by comparing against a collection of curated, filtered signal sources. I accept that this is easier said than done, but we cannot ignore something simply because it is difficult or expensive.

It is incumbent on us, the builders and integrators of AI into production workflows, to understand the limitations of the techniques we are using. 80% accuracy may be enough for an application that classifies objects in photographs. 90% accuracy may be enough to identify bots and spam. In most cases, less than 100% accuracy is probably acceptable.

But today’s applications of AI are not limited to Quality-of-life improvements. AI is increasingly being combined with the immediacy and speed of modern computing in applications where the AI attempts to be better and quicker than humans at decision making. Some decisions driven by AI affect people in a real way, in this case their bank accounts, but in other cases such as self-driving cars and medical applications, AI-driven decisions can affect their lives directly. It is in applications such as these that we must be careful.


Comments? Join the conversation on LinkedIn


Hrishikesh Desai

Written by Hrishikesh Desai.
You can follow me on LinkedIn