Control an NXT with Leap Motion

Unlike other posts, this post merely shows a project overview. Details on replicating this project will be released in subsequent posts with their own video instructions. In this way, you can pick and choose smaller tutorials to follow and build your own project. For example, you may choose to use my Leap Motion setup to control an Ollie robot by Sphero, or use my NXT control script and replace the Leap Motion controls with an Xbox or Wii remote. Read more about this project below or scroll to the bottom to watch it in action!

Overview

The Leap Motion device uses two cameras and infrared LEDs to track hand movements in the 3D space above (or in front) of it. This project uses a Node-Red module to communicate with the Leap software and sends hand location data over WebSockets to a Python script that controls a Lego NXT robot over Bluetooth or USB. Basically, you move the robot by moving your hand around over the Leap Motion, no wires attached.

Hardware

• Leap Motion device
• Lego NXT robot (Any robot with treads will work, including the basic shooter-bot build.)
• Desktop or laptop to run Leap software
• Raspberry Pi with Bluetooth(Optional but recommended)
• Bluetooth dongle (If you aren't using a Raspberry Pi and your computer doesn't have Bluetooth built-in)

Software

• Leap Motion software (leapd)
• Node-Red
• Python

Video

The top right corner of the video displays the Leap Motion Visualizer. This tool simply displays what the Leap Motion is currently seeing through the cameras as well as where it's currently tracking hands.
The top left corner displays the Node-Red flow and debugging output as well as the terminal running the Python script.
The bottom left displays video footage of the physical NXT robot driving in response to my hand input.

Click to watch and support on LBRY!

You can also support this project on YouTube and protect your privacy using BAT in the Brave Browser.

Written with excitment on StackEdit.