This is a brief summary of paper for me to study and organize it, context2vec: Learning Generic Context Embedding with Bidirectional LSTM (Melamud et al., CoNLL 2016) I read and studied.

This paper shows how to embed sentential context into a fixed-length vector with Bidirectional LSTM.

As you can see the figure below, They used a Bidirectional LSTM, feeding one LSTM network with the sentence from left to right, and aother from right to left. The parameter of these two networks are completlely separate, including two separate sets fo left-to-right and right-to-left context word embedding.

So as to represent sentential context to a embedding, they concatenate each context vectors from left to right ant from right to left as shown in figure1 b.

Objective function is how likely the target word vector and sentential context vector is to a pair based on the CBOW method. They used negative sampling technique.

In their embedding, they used two special tokens, BOS and EOS signified as the end and beginning of a sentence.

Melamud et al., CoNLL 2016

Note(Abstract): Context representations are central to various NLP tasks, such as word sense disambiguation, named entity recognition, coreference resolution, and many more. In this work they present a neural model for efficiently learning a generic context embedding function from large corpora, using bidirectional LSTM. With a very simple application of their context representations, they manage to surpass or nearly reach state-of-the-art results on sentence completion, lexical substitution and word sense disambiguation tasks. They release their [code and pretrained models](http://u.cs.biu.ac.il/~nlp/resources/downloads/context2vec/), suggesting their method could be useful in a wide variety of NLP tasks.

Download URL:
The paper: context2vec- Learning Generic Context Embedding with Bidirectional LSTM (Melamud et al., CoNLL 2016)

Reference

Paper
- ACL Version: context2vec- Learning Generic Context Embedding with Bidirectional LSTM (Melamud et al., CoNLL 2016)
How to use html for alert
- how to use icon

context2vec- Learning Generic Context Embedding with Bidirectional LSTM

Title of paper - context2vec- Learning Generic Context Embedding with Bidirectional LSTM

context2vec- Learning Generic Context Embedding with Bidirectional LSTM

Title of paper - context2vec- Learning Generic Context Embedding with Bidirectional LSTM

Reference