Song Genius

Have you ever wondered what the lyrics in your favorite songs really mean?

Well thanks to the great people over at Genius, now you can find out!

Song Genius makes use of the Genius API to look up the lyrics you're pondering about and attempts to give you the meaning behind those words using the annotations provided and voted on by Genius users.

Getting data for slot type values

There were 3 slot types that required me to scrape the web for values. These were the Artist, Song, and Lyrics slots.

To get the information, I visited a couple web pages that had charts for the top music out right now and I ran some simple JavaScript in my browser to collect each artist/song name and put it into an array that I could then save as a line-separated file.

For the artists, I used a combination of kworb.net and Resident Advisor (for more of the electronic people).

For songs, I used the charts on kworb.net.

Getting lyrics was a little more complicated. I actually ended up using the Genius API for this. Artist and song names are already very variable as I can't predict what songs people are listening to, but lyrics are even more variable so I wanted to get a decent amount of coverage with the slot values. So, I took the top songs that I had already collected and ran a search for them on Genius, and if it was available then I further queried Genius for lyrics for that song that had annotations, and added all of those lyrics to an array.

var allLyrics = []; 
var getJSON = function(url, callback) { 
   var xhr = new XMLHttpRequest(); 
   xhr.open("get", url, true); 
   xhr.responseType = "json"; 
   xhr.onload = function() { 
     var status = xhr.status; 
     if (status == 200) { 
       callback(null, xhr.response); 
     } else { 
       callback(status); 
     } 
   }; 
   xhr.send(); 
}; 
function getLyrics(songTitle) { 
 getJSON("http://api.genius.com/search?q="+songTitle.replace(' ', '%20')+"&access_token=" + token, function(err,data){ 
   getJSON("http://api.genius.com/referents?song_id="+data.response.hits[0].result.id+"&per_page=50&access_token=" + token, function(err,data){ 
     var lyrics = data.response.referents; 
     lyrics.forEach(lyric => { 
     	if (lyric.fragment[0] != "[") allLyrics.push(lyric.fragment); 
     }); 
     console.log(`Lyrics for ${songTitle} added`); 
   }); 
 }); 
} 
var index = 0; 
// I checked 50 songs at a time to avoid any rate limitations by the Genius API 
// (if there are any) 
var checkSongs = setInterval(function(){ 
	if(allLyrics.length > 40000 || index > songNames.length) { 
		clearInterval(checkSongs); 
	} 
	else { 
		songNames.slice(index, index + 50).forEach(song => getLyrics(song)); 
		index += 50; 
	} 
},6000); 

After removing duplicates and removing some results that weren't actually lyrics using regular expressions (lines that were clearly too long or had unusual characters) I ended up with 33,753 lines of lyrics from the top 1,388 songs out right now. To be honest, I was pretty impressed with myself, until I found out that it didn't really matter.

A point to bear in mind, though, is that after a couple hundred, the quality of recognition only improves slightly. The list of words you give for a custom slot is "advice". It isn't a hard-and-fast list. Alexa may return to you words not on your list. The more words you have, the wider and more arbitrary the input it will return. So, although you can submit 50,000 words, it is seldom profitable to do so.

I ended up not even being able to put all the lyrics into the model because there was a 600k character limit and I had over 1.2 million characters. Even trying to submit around 600k characters worth of lyrics was troublesome because the build process would always fail, so I ended up just using the top four thousand or so lyrics. I was a little bummed that I had all this great lyric data going to waste, but maybe that will open a door to a new project!

After that I wrote a Lambda function heavily using the sample Tide Pooler skill as a template. Please excuse the lack of optimizations and poor code styling; I wrote it in two nights so it was very rough and hacky (but it works).

Additional Project Documentation

To set up Song Genius yourself, compress the index.js and AlexaSkill.js files in the src/ folder into a .zip file and upload it as an AWS Lambda function. The speechAssets/ folder contains all the information needed to set up the actual skill using the Alexa Skills Kit.

The rough VUI is shown in the attachments below.

Alexa Usage

To use the skill, ask Song Genius questions like, "What did Kanye mean when he said, you should be honored by my lateness?"

Or if you don't know the artist, Song Genius will automatically try to find the song for you based on the lyrics.

If you try it, I hope it's helpful!

- Aurnik (aurnik@aurnik.com)