This is a brief summary of paper for me to study it and organize the material, Recent Trends in Deep Learning Based Natural Language Processing (Young et al., arXiv 2018).

This paper review significant deep learning related models and methods that have been employed for numerous NLP task.

The section comprised of this paper is as follows:

  • Distributed Representation

    a. Word Embedding

    b. Word2vec

    c. character Embeddings

    d. Contextualized Word Embeddings

  • Convolutional neural networks

    a. Basic CNN

    b. Applications

  • Recurrent Neural Networks

    a. Nedd for Recurrent Networks

    b. RNN Models

    c. Applications

    d. Attention mechanism

    e. Parallelized Attetion: The transformer

  • Recusive Neural Networks

    a. Basic model

    b. Applications

  • Deep Reinforced Models and Deep unsupervised learning

    a. Reinforcement Learning for sequence generation

    b. Unsupervised sentene representation Learning

    c. Deep Generative models

  • Memory-augmented Networks

  • Performance of different Models on different NLP tasks

    a. POS tagging

    b. parsing

    c. Named-Entity Recognition

    d. Semantic Role Labeling

    e. Sentiment Classification

    f. Machine Translation

    g. question Answering

    h. Dialogue Systems

    i. Contextual Embeddings

Reference