This is a brief summary of paper for me to study and organize it, Linguistic Regularities in Sparse and Explicit Word Representations (Levy and Goldberg, CoNLL 2014) that I read and studied.

They demonstrated that the linguistic regularity in word embedding is also captured by the traditional distributed word representation, which is the co-occurence matrix with word and context word.

In other words, for word embedding by neural network, it has the property: words that appear in similar contexts will be close to each other in the projected space.

The property makes word embedding share semantic or syntactic properties or capture linguistic regularities and relational similarties.

Those properties also can be captured by the word-context co-occurence matrix.

To sump their saying up, The linguistic regularities apparent in neural embedding are not a consequence of the embedding process, but rather are well preserved by neural embedding.

In order to prove it, they suggest new objective function to eevaluate liguistic regularity using logarithm.

For detailed experiment analysis, you can found in Linguistic Regularities in Sparse and Explicit Word Representations (Levy and Goldberg, CoNLL 2014)

Note(Abstract): Recent work has shown that neural embedded word representations capture many relational similarities, which can be recovered by means of vector arithmetic in the embedded space. They show that Mikolov et al.’s method of ﬁrst adding and subtracting word vectors, and then searching for a word similar to the result, is equivalent to searching for a word that maximizes a linear combination of three pairwise word similarities. Based on this observation, they suggest an improved method of recovering relational similarities, improving the state-of-the-art results on two recent word-analogy datasets. Moreover, they demonstrate that analogy recovery is not restricted to neural word embeddings, and that a similar amount of relational similarities can be recovered from traditional distributional word representations.

Download URL:
The paper: Linguistic Regularities in Sparse and Explicit Word Representations (Levy and Goldberg, CoNLL 2014)

Reference

Paper
- CoNLL 2014 : Linguistic Regularities in Sparse and Explicit Word Representations (Levy and Goldberg, CoNLL 2014)
How to use html for alert
- how to use icon

Linguistic Regularities in Sparse and Explicit Word Representations

Title of paper - Linguistic Regularities in Sparse and Explicit Word Representations

Linguistic Regularities in Sparse and Explicit Word Representations

Title of paper - Linguistic Regularities in Sparse and Explicit Word Representations

Reference