Robots and Vision

By jketreno
Adding artificial intelligence to a Roomba® through the addition of a MinnowBoard Max® and vision processing!
Building a robot that can follow faces and chase a ball doesn't have to be a huge undertaking. By combining a MinnowBoard Max*, a Roomba* vacuum cleaner, and a USB web camera, you can take a short cut that gets you up and running with a vision enabled robotic platform in an afternoon!
Robots
face-tracking

Introduction

I have always wanted to build a robot that could interact with its environment. However, I never had the time to wire motor controllers and other necessary components. By combining a MinnowBoard Max*, a Roomba* vacuum cleaner, and a USB web camera, I took a short cut that had me up and running with a vision enabled robotic platform in no time.

This recipe provides a step-by-step guide to building such a robot; one capable of performing face tracking and many other tasks. Once you have all of the necessary pieces, building the robot can be accomplished in an afternoon.

In addition to this guide, if you want to read more about the process I took for creating this robot, decisions made, or other posts related to things you can do with the robot, see the Journey section.

Ingredients

To get operational
MinnowBoard Max
Roomba
Any series 400+ or an iRobot Create
USB webcam
Supported by Linux and Video for Linux 2
Sparkfun FTDI Basic Breakout - 5V
8G+ microSD card or USB thumb drive
USB hub
(if using a USB thumb drive)
3 12” lengths of thin wire
(ideally of different colors)
AC/DC power converter for MBM
5V @ 2.5A in a 5.5mm x 2.1mm barrel plug [center is positive])
Double-sided sticky tape
For affixing the Minnowboard Max and web camera to the robot
Remote computer
With the ability to communicate with the Minnowboard Max over ethernet.
Optional materials
USB power charger capable of outputting 2.5A
For example, the RAVPower Element 10400mAh. That adapter has two USB output ports. If combined, their power output can reach 3A. You can combine the output using a USB 2.0 Type A male to dual male Y splitter, like this one from Sodial.
USB powered access point
For example, the TP-Link TL-WR702N. This can be wired to receive power from the Roomba’s data port (details in “Going Cordless”). Or you can use an access point that has an integrated battery and not worry about splicing together a cable. See, “Adding a Remote Monitoring to your Robot” for more details
7 or 8-pin mini-DIN Plug for building a Roomba-to-FTDI data cable
1’ 7-wire cable
8”x10” thin piece of wood or plastic
For mounting the electronics to the Roomba

Steps

5 total steps
1
If you do not have an operating system already installed on your MinnowBoard Max’s SD card, you will need to do so.
5
Plug the webcam into the second USB port on the Minnowboard Max.
1
Install software into MinnowBoard Max

If you do not have an operating system already installed on your MinnowBoard Max’s SD card, you will need to do so. I wrote down the steps I took to install Ubuntu on my MinnowBoard Max in the post “Installing Ubuntu on the MinnowBoard Max.”

To use the sample face tracking application, you need to have all of the necessary support libraries on your MinnowBoard Max. The following commands will install the necessary components on Ubuntu 14.04.

On the MinnowBoard Max (via ssh or a USB keyboard,) install the required system packages: sudo apt-get install libopencv-dev libopencv-core2.4 libopencv-calib3d24 \ libopencv-contrib24 libopencv-legacy24 npm git Once those packages are installed, you can download a Node.js face tracking example with the following commands: cd ~ git clone https://github.com/jketreno/robot.git cd robot npm install opencv npm install irobot npm install socket.io npm install mraa

Next, install a couple patches that reduce camera lag with OpenCV (you can read more about these patches in the post "Problems with camera lag!.") patch -p0 < patches/VideoCaptureWrap.h.patch patch -p0 < patches/VideoCaptureWrap.cc.patch npm rebuild --build-from-source opencv

Finally, to connect to the webcam, your user account on the MinnowBoard Max needs to be a member of the video group. To add your account to the group, run the following command: sudo gpasswd -a $(whoami) video

Tips & Tricks:
You can boot from a USB drive instead of a micro SD card, but to do so you will also need a USB hub to also plug in the web camera and USB-FTDI breakout board.
2
Remove Roomba faceplate
Female 7-pin mini-DIN jack

The Roomba ships with a 7-pin Mini-DIN connector which is the interface we will use to control the Roomba.

To access the 7-pin mini-DIN connector, you need to remove the Roomba faceplate. First remove the dust bin, and then lift the outer edge of the plastic faceplate to begin detaching it. The center part requires more force than the edges. For a video of removing the faceplate, see: https://www.youtube.com/watch?v=EaZibdOIeD0.

Once removed, you will see the DIN connector.

3
Connect the MinnowBoard Max to the Roomba
TX (pin 4), RX (pin 3), and Ground (pin 6) on the DIN port

The connection between the Minnowboard Max and Roomba DIN port is done by routing the TX/RX and GROUND lines from the Roomba to the FTDI breakout board.

Use three wires to connect the TX, RX, and Ground pins (labeled above) to the FTDI breakout board’s RXI, TXO, and GND port locations. The TX line from the Roomba is connected to the RXI port, the Roomba’s RX line connects to the FTDI TXI port, and the ground pin goes to ground. Once connected, it should look like the following:

Roomba connected to the FTDI breakout board (This photo also shows an optional wooden platform used to easily add or remove the MinnowBoard Max brain from the robot)

Use caution when putting the wires into the Roomba’s DIN port so you do not cross the wires. For a more permanent solution you can build a DIN plug. Using the wires is a reasonable first step to connect and test your robot. We recommend that you tape the wires to the Roomba to prevent them from moving or coming unplugged.

Connect FTDI breakout board to one of the Minnowboard Max’s two USB sockets

Tips & Tricks:
The MinnowBoard Max has a UART available on the low-speed header pins, however those pins are NOT 5V tolerant, and the Roomba outputs +5V. Accordingly, you cannot wire the Roomba directly to the MinnowBoard Max. You may be able to use a voltage shifter to achieve the same result as the FTDI breakout. For simplicity, however, this guide uses the FTDI board.
4
Mount the components onto the Roomba
Components loosely mounted on Roomba for testing (Note that the TX and RX lines aren’t connected in this photo)

Mount the components on the Roomba using double sided sticky tape. Use small amounts of tape for now; once you have tested that everything is working and you are happy with the position of the components, you can use more tape, or build a platform to mount the Minnowboard Max, web camera, and other components to.

5
Chasing faces

Plug the webcam into the second USB port on the Minnowboard Max.

The robot is now ready to detect and follow faces. Connect to the Minnowboard Max over ssh and launch the Node.js face tracking application. To do that, first create a screen session using the following commands: sudo apt-get install screen screen -S robot

The screen session is used so if you disconnect your Ethernet cable or close the ssh connection, the application you launch does not terminate. You can connect to that running session by running the screen -r robot command.

Now launch the facetrack application: cd ~/robot node facetrack.js

If the program does not come back saying READY, press the CLEAN button once on the Roomba. You may need to perform that action to reset the Roomba’s operating mode if the Roomba has gone to sleep or if it has encountered a safety condition (for example, if you picked it up.)

The light on the web camera should light up, indicating that communication with the robot is successful and the program has opened a connection to the web camera. It is now watching for faces!

Try moving your face in front of the camera. If it finds your face, the software will report various output messages to the console session, and the robot will rotate or move forward and backward to try to keep your face centered in its view.

You can terminate the software by pressing CTRL-C.

Before you disconnect the Ethernet cable and allow the Roomba to run around chasing you, it is a good idea to make sure all of your disk data has been written to the storage device. That way, if the MinnowBoard Max loses power, there is less chance of any disk corruption. You can sync your disk by running the sync command: sync.

You are now ready to disconnect your Ethernet cable. Have fun!