1. 程式人生 > >Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks-paper

Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks-paper

 

1 introduction 

Most models for distributed representations of phrases and sentences—that is, models where realvalued vectors are used to represent meaning—fall into one of three classes:

bag-of-words models-句子中的單詞的序列關係看不出來

sequence models

tree-structured models.-包含了句法語義

 

這篇文章介紹了將標準lstm改進為樹結構一般化過程,在序列lstm上可以表示出句子的含義a generalization of
區別: 

the standard LSTM composes -- hidden state from the input at the current time step and the hidden state of the LSTM unit in the previous time step,

the tree-structured LSTM, orTree-LSTM--composes its state from an input vector 

and the hidden states of arbitrarily many child units.

標準lstm是tree-lstm的一個特例,看做tree-lstm的每個內部節點只有一個孩子

 

 

斯坦福的sentiment treebank:

treebank的形式如下
(0 (1 You) (2 (3 can) (4 (5 (6 run) (7 (8 this) (9 code))) (10 (11 with) (12 (13 (14 our) (15 (16 trained) (17 model))) (18 (19 on) (20 (21 (22 text) (23 files)) (24 (25 with) (26 (27 the) (28 (29 following) (30 command)))))))))))
這是句子“You can run this code with our trained model on text files with the following command”經過stanford模型計算後得到的情感treebank形式。

每個括號中的第一個元素為規則的頭,比如對於左右兩邊都只有一個節點的規則:
(1 You): 1->You , 1表示的是NON-Terminal字元,You表示terminal字元,和標準的pennetreebank的區別是1代表的是這個節點的情感強度,分五個等級。

(0 (1 You) (2 (3 can)…) :
在這個規則裡,右邊有兩個節點,是一個標準的二叉樹,0-> 1, 2。