Deep Learning for Event-Driven Stock Prediction

Reading time:2019/3/30-2019/4/12 Theme:Deep learning; CNN; NLP

Abstract:

　　We propose a deep learning method for eventdriven stock market prediction. First, events are extracted from news text, and represented as dense vectors, trained using a novel neural tensor network. Second, a deep convolutional neural network is used to model both short-term and long-term influences of events on stock price movements. Experimental results show that our model can achieve nearly 6% improvements on S&P 500 index prediction and individual stock prediction, respectively, compared to state-of-the-art baseline methods. In addition, market simulation results show that our system is more capable of making profits than previously reported systems trained on S&P 500 stock historical data.

Main content:

First part:Introduction

　　1.As web information grows, recent work has applied Natural Language Processing (NLP) techniques to explore financial news for predicting market volatility.

　　2.Pioneering work mainly uses simple features from news documents, such as bags-of-words, noun phrases, and named entities.

　　3.One disadvantage of structured representations of events is that they lead to increased sparsity, which potentially limits the predictive power.

　　4.Ding et al. [2014] show that the performance of daily prediction is better than weekly and monthly prediction.

　　5.Despite the relatively weaker effects of long-term events, the volatility of stock markets is still affected by them. However, little previous work quantitively models combined short-term and long-term effects of events. To fill in this gap, we treat history news as daily event sequences, using a convolutional neural network (CNN) to perform semantic composition over the input event sequence, and a pooling layer to extract the most representative global features. Then a feedforward neural network is used to associate the global features with stock trends through a shared hidden layer and a output layer.

　　6.Experiments on large-scale financial news datasets from Reuters and Bloomberg show that event embeddings can effectively address the problem of event sparsity.

Second part: Neural Tensor Network for Learning Event Embeddings

1.Our goal is to automatically learn embeddings for structured event tuples E = (O1, P, O2), which draw more fundamental relations between events, even if they do not share the same action, actor or object

2.Our task is related to previous work on learning distributed representations of multi-relational data from knowledge bases [Bordes et al., 2011; Socher et al., 2013], which learns the embedding of (e1, R, e2), where e1 and e2 are named entities and R is the relation type. However, learning structured event embedding has two significant differences.

First, the number of relation types in knowledge bases is limited.

Second, the goal of relational database embedding is to be able to state whether two entities (e1, e2) are in a certain relation R. When R is symmetric, e1 and e2 have interchangeable roles.

Neural Tensor Network

　　3.As most event arguments consist of several words, we represent the actor, action and object as the average of its word embeddings, respectively, allowing the sharing of statistical strength between the words describing each component.

　　4.We extract more than 10 million events from Reuters financial news and Bloomberg financial news as the training data for event embeddings. The training algorithm repeats for N iterations over the training examples, which is a set of event tuples E = (O1, P, O2), extracted from the training corpus using the method in Section 2.1. In each iteration, the training procedure is shown in Algorithm 1.

Deep Prediction Model

　　1.We model long-term events as events over the past month, mid-term events as events over the past week, and short-term events as events on the past day of the stock price change. As shown in Figure 3, the prediction model learns the effect of these three different time spans on stock prices based on the framework of a CNN.

　　2.For our task, it is necessary to utilize all local features and predict stock price movements globally. Hence, we use a max pooling layer on top of the convolutional layer, which forces the network to retain only the most useful local features produced by the convolutional layer. Note that the convolution operation is only applied to the long-term and mid-term event embeddings, because the unit of timing is one day.

　　3.To correlate the feature vector V^C and stock prices, we use a feedforward neural network with one hidden layer and one output layer.

　　4.We use financial news from Reuters and Bloomberg over the period from October 2006 to November 2013, released by Ding et al. [2014] 1.Randinsky et al. [2012] and Ding et al. [2014] show that news titles are more useful for prediction compared to news contents. This paper extracts events only from news titles. We conduct our experiments on predicting the Standard & Poor’s 500 stock (S&P 500) index and its individual stocks, obtaining indices and prices from Yahoo Finance. Detail statistics of training, development (tuning) and test sets are shown in Table 1.

　　5.In contrast to the baselines, we use a neural tensor network to learn event embeddings for representing news documents, and build a prediction model based on a deep CNN.

　　6.Event embedding is useful for the task of stock market prediction. Given the same prediction model (CNN or NN), the event embeddings based methods (EB-NN and EB-CNN) achieve consistently better performance than the events-based methods (E-NN and E-CNN). This is likely due to the following reasons.

　　First, low-dimensional dense vector can effectively alleviate the problem of feature sparsity.

　　Second, we can learn deeper semantic relations between event embeddings, by modeling the semantic compositionality over word embeddings.

Market Simulation(a strategy)

　　7.We simulate real stock trading by following the strategy proposed by Lavrenko et al. [2000], which mimics the behavio of a daily trader who uses our model in a simple way. If the model indicates that an individual stock price will increase the next day, the fictitious trader will invest in $10,000 worth of that stock at the opening price. After a purchase, the trader will hold the stock for one day. During the holding time, if the stock can make a profit of 2% or more, the trader sells immediately. Otherwise, at the end of the day, the trader sells the stock at the closing price. The same strategy is used for shorting, if the model indicates that an individual stock price will decrease. If the trader can buy the stock at a price 1% lower than shorted, he/she buys the stock to cover. Otherwise, the trader buys the stock at the closing price.

　　8.we notice that if there is no news reported for an individual stock on the previous day, their models cannot predict the trend of the stock price movements on a day, because they do not leverage long-term and mid-term news. This does not hurt the evaluation results of accuracy and MCC, but can hurt the real profit.

　　9.Despite 50 years of studies from the fields of finance,computer science and other research communities, the debate continues over what kinds of information can be useful for stock market prediction. In Artificial Intelligence (AI), three sources of information has been the most exploited for algorithmic stock market prediction.

　　First, some prediction techniques leverage historical and time-series data.

　　Second,with advances of NLP techniques, various studies have found that financial news can dramatically affect the share price of a security.

　　Apart from events, sentiment is another perspective of deep semantic analysis of news documents.

Conclusion

　　We demonstrated that deep learning is useful for event-driven stock price movement prediction by proposing a novel neural tensor network for learning event embeddings, and using a deep convolutional neural network to model the combined influence of long-term events and short-term events on stock price movements. Experimental results showed that eventembeddings-based document representations are better than discrete events-based methods, and deep convolutional neural network can capture longer-term influence of news event than standard feedforward neural network. In market simulation, a simple greedy strategy allowed our model to yield more profit compared with previous work.

關於本篇文章的一些體會和總結：

1.這篇文章是哈爾濱工業大學本部劉挺團隊（主攻NLP）於2015年發表在頂級會議IJCAI(國際人工智能聯合會議)上關於“用深度學習在事件驅動股票預測中的運用”。其主要創新點在於將事件從新聞文本中提取出來，並用一個新的神經張量網絡訓練的密集向量表示。用到的數據為：美國500家上市公司的標準普爾500指數，利用自然語言處理NLP的事件抽取技術,將新聞轉化為結構信息（結構化表示），深度學習CNN進行預測，結果顯示相對最先進的預測方法，性能提高了6%。

2.using open information extraction(actor+action+object)

3.structured representation => increase sparsity => limited predictive power => event embeddings => dense vectors.

4.A novel neural tensor network:它可以學習事件參數的語義組成，方法是將他們相乘組合，而不是像標準的神經網絡那樣僅僅隱式的組合。

5.盡管長期事件影響相對減弱，但是股票市場的波動性仍然受其影響，而以往的定量模型很少將短期和長期事件的影響結合起來。
6.（news to sequences,CNN）在輸入事件序列上用CNN表征其語義組成，池化層來提取最有代表意義的全局特征，前饋神經網絡關聯通過共享隱層和輸出層的伴隨股票趨勢的全局特征。

7.CNN+long term history data:目標是自動學習結構化事件元組（在時間中聯系基礎，盡管沒有共同的action,actor和object）的嵌入。
8.Neural tensor network:input(word embedding),output(event embedding)

9.我們的任務與以前從知識庫中學習多關系數據的分布式表示有關。問題：關聯類型有限，確定兩個事件實體是否關聯。

10.由於大多數事件參數由幾個詞組成，因此我們分別將參與者，動作和對象表示為其詞嵌入的平均值，從而允許描述每個組件的詞之間的共享統計強度。

11.news titles are more useful for prediction compared to news contents.

12.experiment:S&P500index yahoo finance, ACC and MCC(馬修斯相關系數)，using bag of words, svm.

13.in the baseline:a standard feedforward neural network.in new model:a neural tensor network to learn event embedding and a deep CNN.

Contrast:WB-NN,WB-CNN,E-NN,EB-NN,EB-CNN.

14.實驗驗證部分用了市場模擬交易，模仿一個以簡單的方式使用文章中的模型的日常交易行為，如果模型表明某個股票價格將在第二天上漲，那麽虛構的交易者將以開盤價投資價值10000美元的該股票。（設定策略，設置閾值控制交易程度）單獨利用了長期和中期的新聞來預測某天的股價。

15.結論：文章提出了一種新的學習事件嵌入的神經張量網絡，並利用深卷積神經網絡對長期事件和短期事件對股票價格運動的聯合影響進行建模，證明了深度學習對事件驅動的股票價格運動預測是有用的。實驗結果表明，基於事件嵌入的文檔表示方法優於基於離散事件的方法，深度卷積神經網絡比標準前饋神經網絡能夠捕獲新聞事件的長期影響。在市場模擬中，一個簡單的貪婪策略允許我們的模型比以前的工作產生更多的利潤。

總結：這篇文章主要利用了NLP和CNN相關技術，做了長期中期股票新聞對美股指數和個股的影響，發表於2015年，當時神經網絡深度學習正處於迅猛發展期，所以比較新穎。文末作者也指出了，目前對股票價格預測的方法主要有：（1）利用時間序列中一段股票歷史數據；（2）利用自然語言處理技術，不同的研究發現金融新聞能夠戲劇性地影響股票價格。（3）對於新聞做深度情感處理也是另外一個方向。

由於作者並未提供源代碼和數據集，實驗沒法浮現結果，但是作者的思路十分清晰。接下來我也會在新聞上做進一步的研究。

IJCAI_論文-深度學習-Deep Learning for Event-Driven Stock Prediction