Software apps and online services
When I first noticed the LEGO MINDSTORMS Voice Challenge contest I really wanted to enter, what fun to play with Alexa voice driven skills and MINDSTORMS EV3 using to activate LEGOs! My first thought was this should be a fun activity that kids will enjoy (and hopefully learn from as well). Since one project objective is to have a strong use of Voice Interaction and I have friends whose children struggle to learn their multiplication tables an idea was born: focus the skill portion of this project on learning mathematics times tables!
The next question was what should the LEGO MINDSTORMS portion of this project look like? My first thought was a tug of war where two cars powered by correct math quiz answers drive off in opposite directions until a winner is named. No, no good, I wanted a game that allows just one child to practice their math skills, so instead I thought of a LEGO MINDSTORMS powered truck that motors towards the student with each correct answer. Assuming enough questions are answered correctly the truck delivers a prize! Of course each child is unique and so the Mindstorms Math skill I created as part of this project allows for defaults to be tweaked, i.e. number of questions and difficulty level, this lead me to even more Voice Interaction challenges, but it came out fine. I also managed to allow the prizes to be tweaked, in the video you can see that I used LEGO mini-figs as prizes, but a parent could choose to stock the truck with candy or even money!
One additional feature I wanted for this skill was "visuals". When this skill runs on an Echo Show device, I use the Alexa Presentation Language (APL) and present each question in nice color graphics. The game can also be played on an Alexa Fire HD and there the questions are shown in plain text. I think the visuals complement the learning process and help a bit just like old fashioned flash cards.
Let's see Mindstorms Math and the Prize Truck Gadget in ActionNote to speed up the process I changed the default number of questions from 10 to 4 for this video.
Mindstorms Math -Modify Intentand Random Response DemoA user can change the default settings to make a student work more (or less) to earn their prize. The skill allows you to change the number of questions (default 10), the lowest number that will be used (default 1), highest number used (default 12), the focus row (default none), number of correct answers to get a prize (default 9). The focus row option is useful because some multiplication questions are easier than others (i.e. 1 times and 10 times are easy), therefore the focus row option allows a student to work on tougher questions.Note this video also demonstrates how the responses vary both the words Alexa uses to respond and the sounds she plays.
Creating your own version of this projectIn the next sections of this document I'll talk about the process I used for creating this skill. Hopefully, I'll include enough information so you can create your own version of the Mindstorms Math skill and the Prize Truck Alexa gadget.
Building the SkillFor this project I started by building the Alexa skill first, I left the exact LEGO design for later (though I had a rough idea all I really wanted to use from the EV3 was two motors and no sensors). I started by walking through the hackster.ioSetup and Mission 1 through 4 (found here). These tutorials were very good and they led me through the steps from flashing the Alexa Gadgets Toolkit and associated demo missions code onto the EV3 Intelligent Brick, to guiding me through gradually adding function to the Alexa skill portion of the missions. If you run thru these tutorials you'll find setting up the Prize truck an easy process.
Essential Parts for the Setup and Mission 1 through 4With some commenting out of the mission Python code I could get by with only the following parts to prove the EV3 would do my bidding: LEGO MINDSTORMS EV3 Intelligent Brick, MINDSTORMS Large Motor (and cord), microSD or microSDHC card (2GB to 32GB), chip adapter for your computer, a USB wireless adapter (I used a TP-Link Nano adapter), 6 AA batteries (ideally rechargeable, I used TENAVOLTS brand and they worked well for me). As you work through the missions, you'll have to comment out or delete references to any motors or sensors you don't have or the Python code will fail. If you struggle during Missions 1 through 4 don't forget to use the helpful discussion board. You may also gain some hints by checking out my other Alexa / EV3 based project, the Bobblehead Bobbler.
Alexa Skill - Mindstorms Math Before playing with LEGOs, I focused on my Alexa Skill. I modified the Mission-04 Node.js code, but made sure to support the Voice commands and IntentHandlers required for fun game play and for the Modify Defaults feature. I knew I could re-add the EV3 directives after all the game play was worked out. I kept in mind that since this is a game children will want to play, I wanted to support slow responses, the ability to ask for a question to be repeated and even the answer "I don't know". Good news, I think I managed this, the key was use of Session Attributes. Notice in the code I have the statement:
Util.putSessionAttribute(handlerInput, 'lastQuestion', question); This stores the last question the user was asked...then in the AnswerIntentHandler, if the user did not answer with a valid number, the skill can "remember" (and then re-ask) that question by using this statement:
var lastQuestion = "Sorry, I didn't get that, " + attributesManager.getSessionAttributes().lastQuestion;
Other Features of the MindstormsMath Alexa skill:
- FallBack: I added the AMAZON.FallbackIntent (part of the Alexa Skill Kit) so that when a child responds with a goofy answer at the wrong time, we can prompt them to try again rather than just fail.
- Varied Responses: Notice the
randomWrong()methods. These spice up the skill by responding with a random phrase (and sound). So instead of always just responding "Correct!" when a right answer is given, I randomly switch between "Awesome!", "Correct!", "Yes!!" and "Way to Go!".
- Sound in Responses: I usually respond with the short "Bling" sound, but 1 out of 4 responds with a Truck revving sound, just a little extra random fun. Here's the line that adds the sound to the response:
responseString = "Correct <audio src='soundbank://soundlibrary/vehicles/cars/cars_08'/>";You can find a list (and even play) over 2500 sounds clips in theAlexa Skills Kit Sound Library
- Emotion in Responses: For more variety and a more pleasant experience, I also used the Emotion feature. Alexa will emphasize words differently when the code looks like this:
responseString = `<amazon:emotion name="excited" intensity="high">Alright!</amazon:emotion>`;You can find more ways to emphasize speech using the Alexa Skills Kit Speech Synthesis Markup Language.
- Works Disconnected: The hackster.io mission code requires the EV3 be connected via Bluetooth in order to run, if Alexa can't find the EV3 Brick the sample skills just quit. However I made this code work even if the brick was not available, this was done in several places by checking for a valid "endpointId". Because the skill works disconnected, a student can more easily play/practice math even on the Echo Auto while riding in the car!
- Supports Graphics and Different Devices: This skill has multiple code paths because in addition to EV3 or not, I support Screen devices (Echo Show), plain text screen devices (Fire HD) and no screen devices (Echo Dot).
Setting Up the Alexa Skillfor Yourself:To add the Mindstorms Math skill, log in to your Alexa Developer console, as described in Mission 3. You'll notice that most of the action occurs in index.jsbut you'll need to copy up all the files I've attached to this project model.json, common.js, datasource.json, index.js, latest.json, package.json and util.js. Datasource.json is used to send information to the APL screens which are designed in lastest.json In addition to copying the files up to your Alexa Developer Console, you'll need to register this skill as an Alexa gadget as described in Mission 1. As part of this process you go to the Interfaces screen for your skill to turn on "Custom Interface Controller". For this skill since we are enabling the Echo Show devices and APL, you'll also want to turn on Alexa Presentation Language, and check all the screens supported, see the image below.
You may want to tweak some of the defaults in the index.js file. For example, you may want to ask more than 10 questions or only require 8 correct answers for a prize. These defaults are easily found near the top of the index.js file
- Follow the hackster.ioLEGO Voice Challenge setup instructions to get the Alexa toolkit enabled on your EV3 brick. A nice feature of this process is the software is all located on your microSD chip so there's no risk of accidentally flashing bad or broken software on the Mindstorms brick itself.
- In the setup section you downloaded a file named alexa-gadgets-mindstorms-2019.nnn....zip. For the Mindstorms Math project I've simply attached the mindstorms-math.py to this project, you'll need to edit the mindstorms-math.ini file as it must contains your unique amazonId and alexaGadgetSecret (as described in the Mission 4 tutorial). Use Visual Studio Code to send mindstorms-math.py and mindstorms-math.ini over to your EV3 brick. The process is the same as described in Mission 4 except my files are mindstorms-math.pyand mindstorms-math.iniIf you're familiar with the Mission code this Python file is similar, however mindstorms-math.py handles "shake" and "prize" directives. One minor feature I added is when the EV3 (finally) connects to your Echo device, this line of code:
self.sound.speak('Alexa connected', espeak_opts='-a 200 -s 130', volume=100, play_type=0)causes the EV3 Brick itself to speak in an old school computer voice. Should your Echo device switch off or detach, the brick will again speak a message.
The EV3 is told what to do by the directives it receives from the Alexa skill. We send the following types of directives:
- 'shake' - used to move the drive motor
- 'prize' - used to cause the prize motor to deliver the prize
- 'prizeNONE' - used when we don't want the prize motor to run.
- For each of these directives the Alexa skill passes in direction ("forwards" or "backwards"), duration (time to run the motor) and speed (percent of 100% to run the motor). (Note you may need to change the speed or duration depending on how closely your build matches mine)
Pairing and Un-pairing the Prize Truck Alexa GadgetThere are instructions in Mission 1 that describe how to unpair a device. This will be required if you want to switch between say your Echo Dot to your new Echo Show :-). I had to do this early in the process so as a reminder to myself, another way than the described method to unpair is to open an SSH terminal to the EV3 brick and run the command
sudo python3 mission-01/mission-01.py --clear (You'll be prompted for a password and the default password is maker). Now if you simply run your mindstorms-math.py code it will put the brick into pairing mode for the new device. Should you wish to switch back and pair again to your Echo dot, I found that you'll need to forget the previous Bluetooth gadget using the Alexa app.
Buildingthe LEGO Technics Section of this ProjectNow for some additional fun: LEGOs! To document how to use this project I found a neat program called Studio by BrickLink Corporation. It is free and available here https://www.bricklink.com/v3/studio/download.page By the way, while working on this project I learned that this past November the LEGO Group acquired BrickLink Ltd the world’s largest online community of adult LEGO fans, so it makes me feel like the studio software is likely to be supported in the future.
Once you install BrickLink Studio you'll be able to use File -> Import to import the PrizeTruckFullBuildFinal.io model. From there you can view in 360 degrees, zoom in and out, see the Bill Of Materials (which I have also attached here), go step by step through the build and even modify the color or design or build instructions for yourself. Studio is a very powerful application and supports a very complete database of LEGO parts. Please excuse the color scheme of the truck bed, I had to go with the colors I had available and decided to be true to those colors even in the Studio app :-). There was one LEGO part I couldn't find, you'll notice in the Studio design model the EV3 brick is attached to the truck by two gray Technic beams, while in the videos it is attached by two gray curved halves of a Technic dump truck bed (#42088 Cherry Picker Truck to be exact). Another difference is the conveyor belt shown in the Studio design doesn't have a belt, but that piece is generally sold as one complete unit (Technic Part 92715C01).
Prize Truck GutsVideo showing the truck insides including the Conveyor Belt and Gear Box.
How to Wire the EV3 BrickOne more detail missing from the Studio design is how to connect the cables for the EV3. Motors get power from the wires connected at the top of the EV3 Brick, ports A - D. I used C for the motor that drives the vehicle and B for the motor that powers the prize conveyor belt.
Other TweaksHere's a short video taken after I changed to run the conveyor belt "after" moving forwards. This is how the attached code works. Note the defaults have been changed for this video to only ask 2 questions and only require 1 correct answer to deliver a prize, normally the default is 10 questions.
At almost the project deadline, I happened to find a Technics Racer set at a rummage sale. I used the larger wheels for an alternate wheel design that I think looks a bit nicer, so I'm including another Studio design model file for this design...here's the Prize Truck with larger wheels:
FinalThoughtsThanks for checking out this Alexa LEGO MINDSTORMS Voice Challenge project. I enjoyed learning while playing and encourage you to try creating an Alexa enabled gadget with LEGO MINDSTORMS, it's fun!