This is a brief summary of paper for me to study and organize it, Supervised Learning of Universal Sentence Representations from Natural Language Inference Data (Conneau et al., EMNLP 2017) I read and studied.

This paper propose the method to encode a sentence in a vector training on NLI task corpus.

In other words, they said the sentence vector from supervised data (NLI) contain semantic information of input sentence than other model trained on unsupervised corpus.

First of all, befoer diving into model to encode sentence in a vector, let’s see the what NLI (natural language inference) task is :

Conneau et al., EMNLP 2017

As you can see above, Models can be trained on SNLI in two different way

1) sentence encodeind-based model that explicitly separate the encoding of the indivisual sentences 2) joint methods that allow to use encoding of both sentences (to use cross-features or attention from one sentence ot the other)

They used a variety of models to generate sentence vector.

First, Bi-LSTM max pooling network

Conneau et al., EMNLP 2017

Second, inner attention network

Conneau et al., EMNLP 2017

Third, Hieraachical ConvNet architecture.

Conneau et al., EMNLP 2017

Note(Abstract): Many modern NLP systems rely on word embeddings, previously trained in an unsupervised manner on large corpora, as base features. Efforts to obtain embeddings for larger chunks of text, such as sentences, have however not been so successful. Several attempts at learning unsupervised representations of sentences have not reached satisfactory enough performance to be widely adopted. In this paper, They show how universal sentence representations trained using the supervised data of the Stanford Natural Language Inference datasets can consistently outperform unsupervised methods like SkipThought vectors (Kiros et al., 2015) on a wide range of transfer tasks. Much like how computer vision uses ImageNet to obtain features, which can then be transferred to other tasks, our work tends to indicate the suitability of natural language inference for transfer learning to other NLP tasks.

Download URL:
The paper: Supervised Learning of Universal Sentence Representations from Natural Language Inference Data (Conneau et al., EMNLP 2017)

Supervised Learning of Universal Sentence Representations from Natural Language Inference Data

Title of paper - Supervised Learning of Universal Sentence Representations from Natural Language Inference Data

Supervised Learning of Universal Sentence Representations from Natural Language Inference Data

Title of paper - Supervised Learning of Universal Sentence Representations from Natural Language Inference Data

Reference