RoboSapien Whips Nae Nae

I wanted to add voice control to RoboSapien. At first, I was thinking, just create an app on the phone, but that's been done before! Then I saw the Amazon IOT Challenge. How about make an IOT Robosapien. The next generation - Internet of Toys! Connect RoboSapien to the Internet. That would be Awesome, don't you think?

That's what I did; I started looking into the Amazon IOT service, try their sample apps. Very interesting indeed. I like the way they presented the diagram. Here's what I want to do.... Replace RGB controller with Amazon Echo, Replace the LightBulb with RoboSapien... Amazon Echo sends the command, RoboSapien follows. That's it. 

If you enjoy this project and want to see more, please add respect and follow me. 

Amazon Echo ... echo ... echo ..

Last year, I started looking at Amazon Echo. I got one because I attended a hackathon and worked on an Amazon Echo. It was my first introduction to the AWS Lambda, and I was hooked. I like the idea where I can focus on my code and not the infrastructure.

WowWee RoboSapien

I've always liked the RoboSapien, I'm amazed on it's engineering and it's capabilities. Last year also, I started to research how to expand it's capabilities. I have RoboSapien X which doesn't have any bluetooth capabilities. It communicates through it's IR Remote but they gave you this dongle to connect it to your phone so you can use the app. I found out that there's a way to control the robot by playing WAV files. I saw it on WowWeeLabs github page. Here's the link

https://github.com/WowWeeLabs/RoboRemote-IR-Dongle-SDK

So I downloaded the WAV files and started playing it on my phone connected with IR Dongle to audio jack. The RoboSapien is ALIVE!

That brings me to an idea. What if I can connect it to a Raspberry Pi? Raspberry Pi has Audio jack, then I can attach it to the back of the RoboSapien. That's great news! No more heart bypass operation for the poor Robot. No soldering!

Back to Amazon IOT.

Here's a good tutorial how to connect AWS IOT to Raspberry Pi

https://github.com/sreid/aws-iot-raspberry-pi-how-to

The sample app is very helpful. Here's the link

https://github.com/aws/aws-iot-device-sdk-js

I patterned my code with this thing example

https://github.com/aws/aws-iot-device-sdk-js/blob/master/examples/thing-example.js

At first, I tried running the example on my laptop. I opened two shell window, one is acting as mobile app (value updater), the other acting as a device (update receiver)

it worked! Send update RGB colors from one window, the other receives the updates. 

Now I tried it on the Raspberry Pi with a Wifi connection.

It also worked!

The idea is, instead of RGB colors, how about send COMMANDS like Walk, High Five, Dance, etc... etc... 

Then once the Command is received from Amazon IOT thru the "delta" event, play the correct WAV files.

Playing WAV files on Raspberry Pi in NodeJS.

I am new to NodeJS, I was able to update node on my raspberry pi with this link. 

https://github.com/rwaldron/johnny-five/wiki/Getting-started-with-Raspberry-Pi-node-red-and-IoT

I looked at some npm packages to play WAV files on raspberry pi, I am familiar with CylonJS. So I followed instructions on how to install CylonJS audio module.

http://cylonjs.com/documentation/platforms/audio/

 sudo apt-get install mpg123

you have to install mpg123. CylonJS-Audio only plays mp3, so I had to convert all the WAV files to MP3. I used Lame

sudo apt-get install lame

to convert all mp3, I used this command.

for f in *.wav; do lame -V 1 "$f" "${f%.wav}.mp3"; done

Here's the sample code on how I played MP3 in NodeJS

var Cylon = require('cylon');

Cylon.robot({
  connections: {
    audio: { adaptor: 'audio' }
  },

  devices: {
    audio: { driver: 'audio' }
  },

  work: function(my) {
    my.audio.on('playing', function(song){
      console.log('Playing this nice tune: "' + song + '"');
    });

    // You can pass a string with a full or relative path here,
    my.audio.play('./RoboSapien/43_highfive.mp3');
  }
}).start();

I connected the RoboSapien dongle to Raspberry Pi audio jack. I played the mp3, and it worked! Woohoo! I actually did High Five RoboSapien.

Echo Thingy. Connecting Amazon Echo to Amazon IOT. 

I couldn't find any sample code out there that connected Amazon Echo to Amazon IOT. Most of the examples are just connecting amazon Echo to a website. That's not going to work. I need to send MQTT messages to Amazon IOT.

At first, I tried going thru their sample app.

https://developer.amazon.com/public/solutions/alexa/alexa-skills-kit/docs/developing-an-alexa-skill-as-a-lambda-function

then I modified it to use these schemas

{
  "intents": [
    {
      "intent": "MyCommandIntent",
      "slots": [
        {
          "name": "Command",
          "type": "LIST_OF_COMMANDS"
        }
      ]
    },
    {
      "intent": "WhatsYourNameIntent"
    },
    {
      "intent": "DanceIntent"
    }
  ]
}

I added LIST_OF_COMMANDS

HIGH FIVE
WHISTLE
DANCE
WALK

Then added Sample Utterances

WhatsYourNameIntent what's your name
WhatsYourNameIntent who are you
WhatsYourNameIntent who you are
DanceIntent Do a Dance
MyCommandIntent to do the {Command}
MyCommandIntent to {Command} me
MyCommandIntent to {Command} with me
MyCommandIntent to {Command}
MyCommandIntent I want him to {Command}
MyCommandIntent I want her to {Command}
MyCommandIntent Do a {Command}

Ok, now that I have the commands set up in Alexa Skills Set. I followed the AWS Lambda instructions. I started playing with the  alexa-skills-kit-color-expert sample javascript code. I was able to test the utterances and the intent and receive the command, but how do I send it to Amazon IOT. 

alexa-skills-kit-color-expert code and the thing-examples.js. At the end of the day, when I receive an intent, I need to connect to the thingymajigger. but the thingymajigger requires aws-iot-device-sdk library. How do I put things together?

I found out you can upload a zip file of all the stuff you need. OK, I zipped up the js file together with node_modules and certs. Then run it.

I got error left and right, up and down. It's complaining about certificates. I took it as a good sign because It's running index.js, that's a start.

I was stumped, don't know what to do, it's complaining that it can't read the certificates. AWS Lambda can't read a text file, it's a permissions thing. A-ha, so I copied the contents of the file and shove it into a variable.

At first, I wasn't including the BEGIN and END tags to the file, so I was getting frustrated. I looked at RoboSapien, he's dead. Battery's dead. I mean. Need to change batteries.

Then I realized, I need to include the BEGIN Key and END Key.

var myPrivateKey = new Buffer('-----BEGIN RSA PRIVATE KEY-----\n' +
'-----END RSA PRIVATE KEY-----');

So I did all that to all the certs. I assigned the contents to a variable, instead of reading it from a file.

var thingOptions = {
            privateKey: myPrivateKey,
            clientCert: myClientCert,
            caCert: myCaCert,
            clientId: 'macbook',
                region: 'us-east-1'
            };

That seemed to work. I used this test message to AWS Lambda. 

{
  "session": {
    "new": false,
    "sessionId": "session1234",
    "attributes": {},
    "user": {
      "userId": null
    },
    "application": {
      "applicationId": "amzn1.echo-sdk-ams.app.[unique-value-here]"
    }
  },
  "version": "1.0",
  "request": {
    "intent": {
      "slots": {
        "Command": {
          "name": "Command",
          "value": "WALK"
        }
      },
      "name": "MyCommandIntent"
    },
    "type": "IntentRequest",
    "requestId": "request5678"
  }
}

I got this in the log

START RequestId: 66122e48-bc93-11e5-b828-1f058216a32e Version: $LATEST2016-01-16T20:54:56.983Z	66122e48-bc93-11e5-b828-1f058216a32e	event.session.application.applicationId=amzn1.echo-sdk-ams.app.[unique-value-here]2016-01-16T20:54:56.984Z	66122e48-bc93-11e5-b828-1f058216a32e	onIntent requestId=request5678, sessionId=session12342016-01-16T20:54:58.072Z	66122e48-bc93-11e5-b828-1f058216a32e	connected to things instance, registering thing nameEND RequestId: 66122e48-bc93-11e5-b828-1f058216a32eREPORT RequestId: 66122e48-bc93-11e5-b828-1f058216a32e	Duration: 1136.99 ms	Billed Duration: 1200 ms 	Memory Size: 128 MB	Max Memory Used: 34 MB	

The most important thing is it connected to the thingymajigger. Cool! Then I looked at my device shadow, I got the value I expected!

{
  "desired": {
    "name": "WALK"
  },
  "delta": {
    "name": "WALK"
  }
}

Now What?

Next thing is to make the project in raspberry pi to receive messages from AWS IOT. I just reused the same template just to test if I'm going to receive data from Amazon Echo. 

https://github.com/aws/aws-iot-device-sdk-js/blob/master/examples/thing-example.js

Wait for it... Wait for it... then got this message

connected to things instance, registering thing name
DEVICE:get timeout on pi_1: {}
DEVICE:delta on pi_1: {"timestamp":1452977698,"state":{"name":"WALK"},"metadata":{"name":{"timestamp":1452977698}}}
Command: WALK

I got the WALK command. I felt like walking in the clouds. 

That Cylon Thing.

Next stop, combining the thing-example.js code with cylon-audio js sample code.

Here's my idea. When I receive a Delta event, play the corresponding mp3 command, then acknowledge the command by sending "NONE" command AWS IOT. 

var audioFileName = '';
var Cylon = require('cylon');
Cylon.robot({
  name: 'robosapien',
  connections: {
    audio: { adaptor: 'audio' }
  },
  commands: function() {
    return {
      doCommand: this.doCommand
    }
  },
  devices: {
    audio: { driver: 'audio' }
  },
  work: function(my) {
    my.audio.on('playing', function(song){
      console.log('Playing this command: "' + song + '"');
    });
         setInterval(function() {
                if (audioFileName != '')
                {
                        console.log('playing:' + audioFileName);
                        my.audio.play('./RoboSapien/' + audioFileName);
                        audioFileName = '';
                }
         }, 500);
  },
  doCommand: function() {
        my.audio.play('./RoboSapien/' + audioFileName);
  }
});
Cylon.start();

I stored all the mp3 files to RoboSapien subfolder. Then, I created this audioFileName global variable. I detect if audioFileName is blank or not. If it has a filename then play that one. make it check every 500 ms.

thingShadows
     .on('delta', function(thingName, stateObject) {
         console.log(role+':delta on '+thingName+': '+
                     JSON.stringify(stateObject));
         commandValues=stateObject.state;
        console.log("Command: " + commandValues.name);
        if (commandValues.name != "NONE")
        {
                doCommand(commandValues.name);
                opFunction("NONE");
        }
     });

var doCommand = function(command) {
        switch (command) {
                case "WALK":
                console.log(command);
                audioFileName = '11_GO_FORWARD_0x86.mp3';
                break;
        default:
                console.log("Unknown Command:" + command);
}
};

Then when I receive a Delta command, that means somethingy changed, I get the command and assign the audioFileName variable.

To start I just type in the raspberry pi

> node index.js

It listens to Amazon IOT for commands. I can actually update the Shadow, and it would send to all the listeners.

Then I'll say:

Alexa, tell RoboSapien to do the Nae Nae

Here's the github repository

https://github.com/rlyle78/iot-echo-robosapien

Let's let him do the Nae Nae again!

Few lessons from this approach:

1. The IR Sensor must be pointed to RoboSapien. Line of Sight is important.

2. Sometimes Alexa would timeout. The AWS Lambda is timing out getting connection and sending update to AWS IOT.  I'm not sure why? It works on the first update...

It is still a work in progress. It's a journey but it's all good. 

I enjoy trying to challenge myself how technology pieces can go together. Please add respect if you like these projects and follow me.