This paper’s title is GhostWriter: Using an LSTM for Automatic Rap Lyric Generation (Potash et al., EMNLP 2015) from EMNLP 2015

The model from this paper focus on generation of rap lyrics. the goal of Ghost Writer is to produce a system that can take a given artist’s lyrics and generate similar yet unique lyrics.

They said the contributiono of their paper like this:

  • They said “we present the ghostwriting problem of producing similar yet differenct lyrics”

  • They said “we present computational, quantitative evaluation methods for these twe aspects”

  • They said “we evaluate the performance of a Long Short-Term Memory(LSTM) vs n-gram model for this problem”

their goal to build model is the model doesn’t require templates/constrains to generate lyrics and the model is able to produce full verses, as opposed to single lies.

The model can alos imitate the lyrics style of a target artist by understanding the artist’s vocabulary and rhythmic style.

Fron now on, I summarized the argument against their model.

Let’s See how to use LSTM for generation of lyrics.

I think they used word embedding to a LSTM, they used on each word learn a LSTM’s new representation of word embedding with sequence information.

They hope LSTM can captures the rhythmic style of an artist by learning rhyme and meter patters.

They said the goal of their model is to not just generate lyrics, but generate the structure for the lyrics as well.

For their goal, They used some tokens like this :

”<endLine>”, “<endVerse>”

They said they expect the system will generate its own line break(<endLine>) and define when the generated verse ends(<endVerse>).

They expected one thing more from “", It is the system has a better chancd of understading rhyme schemes used by an artist.

Their dataset

They collected songs from The Original Hip-Hap (Rap) Lyrics Achive - OHHLA.com - Hip-Hop Since 1992

They also used a paticular artist, Fabolous. because his lyrics produced highest accuracy in the artist recognition experiments in (Hirjee and Brown, 2010a).

For performance, they used a comparable model which is markov model for lyrics generation based on N-gram method.

In the case of n-gram model, they used back-off to a smaller n-gram against the probability that the context has been never encountered in training data.

Their model’s initialization used “<startVerse>” token.

When producing lyrics with either the LSTM or baseline model. they said “we initialize with the “<strtVerse>” token.

They used python implementation of an LSTM from Jonathan Raiman.

Reference