This is a brief summary of paper for me to study and arrange it, A Convolutional Neural Network for Modelling Sentences (Kalchbrenner et al., ACL 2014) I read and studied.

This paper propose sentence embedding method by convoutionala neural network with dynamic max pooling operation.

Their method complements the Time-Delay Neural Network(TDNN), in particular, MAX-TDNN.

The architecture is the following:

Kalchbrenner et al., ACL 20146

As you can see figure above, their propose K-Max pooling which extract K highest valures in time sequence of p(i.e a sequence p).

The order of K-max pooling corresponds to their orignal order of p.

In the figure above, K-max pooling extracts K highest valures in time sequence of p.

The Dynamic K-max pooling is to choose k dynamically depending on the number of convolutional layer and K in final k-max pooling.

The folding is simple way to give rise to dependencies in differenc row in sentence maxtrix by summing the fow in the maxtrix element-wise.

Note(Abstract): The ability to accurately represent sentences is central to language understanding. They describe a convolutional architecture dubbed the Dynamic Convolutional Neural Network (DCNN) that they adopt for the semantic modelling of sentences. The network uses Dynamic k-Max Pooling, a global pooling operation over linear sequences. The network handles input sentences of varying length and induces a feature graph over the sentence that is capable of explicitly capturing short and long-range relations. The network does not rely on a parse tree and is easily applicable to any language.

Download URL:
The paper: A Convolutional Neural Network for Modelling Sentences (Kalchbrenner et al., ACL 2014)

A Convolutional Neural Network for Modelling Sentences

Title of paper - A Convolutional Neural Network for Modelling Sentences

A Convolutional Neural Network for Modelling Sentences

Title of paper - A Convolutional Neural Network for Modelling Sentences

Reference