This is a brief summary of paper for me to study and arrange for Max-Margin Tensor Neural Network for Chinese Word Segmentation (Pei et al., ACL 2014) I read and studied.
This paper is a research ralted to chinese word segmentation task.
They said that the existing study of chinese word segmeantationa used manual feature engineerings, however the neural network model alleviate the burden of manual feature engineering.
The following is an example of feature engineering they explained.
so they propose an model to utilize neural network with tensor.
The reason that they use tensor as alternative of non-linear transformation
That is because non-linear transformation poorly models the complex interaction wihch is between tags and characters(e.g. context and target character).
Frist of all, before diving into their model let’s see conventional model for chines word segmenation.
The conventional model is as follows:
The model is simple, they use an window for context character embedding and then compute hidden layer with concatenation of the center and context character embedding.
Finally it evaluates score for each tag types.
They propose Max-margin Tensor Neural Network to model tag-tag, tag-character and character-character interactions in neural network as follows:
They regarded each slice of the tensor as capturing a specific type of tag-character interaction and character-character interaction.
The following is bilinear form with tensor on vector a.
For tensor operation, it is expensive and has the risk of overfitting so they proposed tesor factorization as follows:
The paper: Max-Margin Tensor Neural Network for Chinese Word Segmentation (Pei et al., ACL 2014)
Reference
- Paper
- How to use html for alert