Intel RealSense Robotic Development Kit

By jketreno
From Zero to Hero: Getting up and running with the Intel RealSense Robotic Development Kit
Level: Beginner
1h
Getting the Intel RealSense Robotics Development Kit (RDK) up and running is a straightforward process.
RealSense
Robots
ROS

Introduction

Getting the Intel RealSense Robotics Development Kit (RDK) up and running is a straightforward process. This tutorial provides the links and instructions for the following: installing Ubuntu 16.04 LTS on the UP board; adding the open source linux-upboard kernel; installing the librealsense library for accessing the Intel RealSense camera; and installing the Robot Operating System (ROS) Kinetic Kame release.

If you are building a project that depends on Ubuntu 14.04 and ROS Indigo Igloo, you can follow the 14.04 instructions instead.

Updated: 2017-11-28 to provide links to Ubuntu 16.04.3 and updated rViz commands.

Updated: 2016-11-09 to provide steps for Ubuntu 16.04.1. Instructions for Ubuntu 14.04 moved here

Updated: 2016-08-25 to provide troubleshooting steps to boot from the USB flash drive.

Updated: 2016-08-15 to use upstream ros-indigo-realsense-camera package!

Ingredients

Others
Intel RealSense Robotic Development Kit
Intel RealSense Robotics Development Kit
HDMI to HDMI cable
USB Flash Drive (2G or larger)
The USB flash drive is used to install the operating system on the UP board.
USB Keyboard and USB Mouse
Ethernet cable
The ethernet cable is used to perform software updates and download the additional packages not provided with the base operating system.
Internet connected router/switch/hub
Desktop/Laptop computer
The desktop/laptop can run Windows, Linux, or OS X. It is used to download the OS image and write to the USB flash drive.

Steps

10 total steps
1
The RDK comes with the following items to use as the foundation for your robot.
2
Download a copy of Ubuntu 16.04 LTS desktop here.
3
Follow these instructions to create a bootable USB drive containing Ubuntu 16.04.
4
Boot the UP board from the USB flash drive
5
Now that you have Ubuntu installed, update the system to have the latest software.
6
Since the release of the upboard kernel, additional patches have been made available to enable more RealSense cameras.
7
Now that you have Ubuntu installed, with kernel source packages enabled, install the ROS framework release &q
8
The following is a based on the steps provided under Getting Started in the
9
RViz can be run from a remote computer or on the UP board. In either instance, roscore needs to be running on the UP board.
10
Congratulations! Your system is now setup.
1
What's in the box?

The RDK comes with the following items to use as the foundation for your robot.


What's in the box

  • Intel RealSense R200 Camera
  • UP board 4GB + 32GB eMMC
  • USB 3.0 Type A Female to Micro USB Type B Male Adapter
  • USB 3.0 Type B Male to Micro USB Type B Male Cable
  • 5V 4A Power supply

 

2
Download Ubuntu 16.04.3

Download a copy of Ubuntu 16.04 LTS desktop here. The ISO file is ~1.5G.

If you are using Linux, you can download that file using wget: wget http://releases.ubuntu.com/16.04/ubuntu-16.04.3-desktop-amd64.iso

If it has been a while since this tutorial has been updated, Ubuntu may have moved to a newer version. You can visit the main 16.04 page here and look for the most recent desktop-amd64.iso.

3
Make a bootable USB drive

Follow these instructions to create a bootable USB drive containing Ubuntu 16.04. Once you have the operating system ready on the USB flash drive, move on to installing Ubuntu 16.04 on the UP board.

OS X

If you are using a MAC, follow Ubuntu's Create a USB Stick on OS X instructions to create a bootable Ubuntu image on the USB drive.

For Macintosh users, you may need to reformat your USB flash drive to MS-DOS FAT prior to creating the Ubuntu USB Boot Disk as described at the above URL.

Windows

We strongly recommended you follow Ubuntu's Create a USB Stick on Windows instructions to create a bootable Ubuntu image on the USB drive. Using other utilities may result in the install process incorrectly identifying USB media partway through installation.

Ubuntu Linux

For Ubuntu users, follow Ubuntu's Create a USB Stick on Ubuntu instructions to create a bootable Ubuntu image on the USB drive.

4
Install Ubuntu 16.04.3 on the UP board

Boot the UP board from the USB flash drive

Connecting the UP board

  1. Plug the USB keyboard and mouse into two of the four USB 2.0 ports.
  2. Plug the USB 3.0 OTG adapter into the UP board's USB 3.0 port. Do not plug your RealSense camera into the adapter yet; it needs to be unplugged while the udev rules are being configured in a later step.
  3. Plug an HDMI cable into the UP board's HDMI port and connect the other end of the cable to your monitor.
  4. Plug the USB flash drive into one of the remaining USB 2.0 ports.
  5. Plug one end of the 5V power supply into an outlet and the other end into the UP board.

Prior to plugging in the 5V power supply, make sure the red and black wire for clock battery is not blocking the CPU fan. The fan should spin freely, and immediately begin spinning once the UP board is plugged in.

Your UP board will now boot. After a few seconds, the UP logo should appear on your display. The system will then begin booting from the Ubuntu installer on the USB flash drive. You will be presented with the GRUB bootloader. To install Ubuntu onto the UP board, select the 2nd option "Install Ubuntu."

If you have already installed an operating system on your UP board, you will need to boot into the Aptio Setup Utility, and instruct the system to boot from the USB flash drive instead of the internal eMMC flash. To do this, reboot your system and when the UP logo is displayed, press the DELETE key on your keyboard.

The default password for the system is emtpy, so just press ENTER when prompted.

Once in the Aptio Setup Utility, use the right arrow on the keyboard to navigate to the Boot menu option. Use the down arrow to navigate to the Boot Option #1, press ENTER, and then select the "UEFI: ..." entry that corresponds to your USB flash drive.

You can then press F4 to Save and Exit. Select Yes and the system will restart, booting from your USB Flash drive.

You will be presented with the GRUB bootloader. To install Ubuntu onto the UP board, select the 2nd option "Install Ubuntu."

Installation options

During the installation you will be prompted to make several decisions. If you do not have a mouse, you can use the TAB key to navigate between options, and SPACEBAR to select your option. There isn't anything tricky you need to do during the installation; however, all of the steps are below for your information.

For my installation, I made the following choices:

Screen 1: Welcome

  • I selected the default "English"
  • => Continue

Screen 2: Preparing to install Ubuntu

  • I did not select "Download updates..."
  • I did not checkbox "Install third-party software..."
  • NOTE: I did not have an ethernet cable connected to the UP board during my installation. This tutorial will walk you through updating the OS after the initial installation completes.
  • => Continue

Screen 3: Installation type

  • I selected the default 'Erase disk and install Ubuntu'
  • => Install Now
  • Write the changes to disks?
    • => Continue
  • If you have already installed Ubuntu 14.04 LTS, the default selection will be "Install Ubuntu 16.04 LTS alongside Ubuntu 14.04 LTS". For a fresh 16.04 LTS installation, it is recommended to select "Erase disk and install Ubuntu." However, only do this if you know you do not need anything from the current 14.04 installation.

Screen 4: Where are you?

  • Select your location.
  • => Continue

Screen 5: Keyboard layout

  • I selected 'English (US)'
  • => Continue

Screen 6: Who are you?

  • Your name: robot
  • Your computer's name: robot
  • Pick a username: robot
  • Choose a password: robot (yes, it is a Short password)
  • Confirm your password: robot
  • I selected 'Log in automatically'
  • I did not select 'Encrypt my home folder'
  • => Continue

After automated installation completes

The installation program will then take several minutes to automatically install and configure the base Ubuntu Desktop. Once it is complete: Ubuntu 16.04 up and running

  1. Select 'Restart Now.' It will take several seconds to reboot.
  2. When prompted, remove the USB media and press ENTER to reboot.
  3. If you haven't already, now is a good time to plug in your ethernet cable to the UP board (with the other end of the cable connected to your router/switch/hub.)
  4. Once booted, you should see a message pop up in the upper right corner of the screen indicating your UP board is now connected to the Internet.

If you happen to be behind a firewall that requires a proxy to access the Internet, you will need to configure Ubuntu to use your proxy server. See this article on help.ubuntu.com for information on how to do this.

5
Update the installed Ubuntu software

Now that you have Ubuntu installed, update the system to have the latest software. Do this via the command line (press CTRL-ALT-T to bring up a new terminal.)

Once the terminal is launched, you can update the operating system by running the following:

sudo apt update sudo apt -y dist-upgrade
If you used the default user and password from this tutorial, your root password (when prompted by running "sudo") is "robot".

If you see the following when you try and run apt update:
E: Could not get lock /var/lib/apt/lists/lock - open (11: Resource temporarily unavailable)
E: Unable to lock directory /var/lib/apt/lists/
The operating system is already trying to perform an automatic system update. To work around this, you can kill the background instance of aptd via: sudo killall -9 aptd and then re-run the sudo apt update command. NOTE: You can find out which application has a lock on the file via the 'lsof' command: sudo lsof /var/lib/apt/lists/lock

Install the linux-upboard kernel

Per the instructions on the UP board Wiki you can install the latest linux-upboard kernel by running the following:

sudo add-apt-repository ppa:ubilinux/up sudo apt update sudo apt -y install linux-upboard sudo apt -y autoremove --purge 'linux-.*generic'
You now must reboot to ensure you are running with all the latest software: sudo reboot

Verify you are running the linux-upboard kernel:

uname -r

If the output indicates you are running a kernel with the string upboard in the name, you are ready to go!:

robot@robot:~$ uname -r
4.4.0.2-upboard
robot@robot:~$

The linux-upboard package enables features such as GPIO pins, and the kernel video capture format necessary for the Intel RealSense camera.

6
Provide kernel source access while installing the latest RealSense patches

Since the release of the upboard kernel, additional patches have been made available to enable more RealSense cameras. Prior to installing the ROS with RealSense support, your system needs to have access to the kernel source packages. The following will is adapted from the ROS librealrense: Installation Prerequisites and will update apt's source files in /etc/apt/sources.list.d appropriately.

wget -q -O - https://bit.ly/en_krnl_src | sudo /bin/bash

The above script will modify your existing apt sources.list files so the kernel sources can be found during the installation of the ros-kinetic-librealsense package later in the tutorial.

For the curious, the bit.ly URL above (provided so you don't have to type as much if you are manually transcribing this tutorial) re-directs to the following file hosted on githubusercontent.com:

https://raw.githubusercontent.com/IntelRealSense/librealsense/rosdebian/scripts/enable_kernel_sources.sh

At this point you are now ready to install ROS with Intel RealSense support.

7
Install ROS Kinetic Kame framework

Now that you have Ubuntu installed, with kernel source packages enabled, install the ROS framework release "Kinetic Kame" including the RealSense pacages.

If one isn't running already, launch a terminal shell.

You need to add the ROS repository to Ubuntu, update the package lists, and install the ROS base software:

Add ROS repository and update Ubuntu package list

sudo add-apt-repository http://packages.ros.org/ros/ubuntu sudo apt-key adv --keyserver hkp://ha.pool.sks-keyservers.net \ --recv-key 0xB01FA116 sudo apt update

Install ROS packages and update

sudo apt -y install ros-kinetic-desktop-full python-rosinstall ros-kinetic-realsense-camera sudo rosdep init rosdep update echo "source /opt/ros/kinetic/setup.bash" >> ~/.bashrc source ~/.bashrc

Adding setup.bash to the .bashrc file ensures that the ROS framework is available in the active environment.

The installation of the package ros-kinetic-librealsense will build a new kernel image with the latest RealSense patches applied. Once installed, you will need to reboot your computer to take advantage of that kernel. As before, you can reboot via:

sudo reboot Once the system has rebooted, you can verify the new kernel image via uname -r:

robot@robot:~$ uname -r
4.4.0-2-upboard
robot@robot:~$

For additional information, see http://wiki.ros.org/kinetic/Installation/Ubuntu.

The README for the realsense-camera node provides comprehensive documentation. For additional information about the capabilities and options, see that README.

8
Running the Intel RealSense R200 nodelet

The following is a based on the steps provided under Getting Started in the intel-ros/realsense README.md

The Intel RealSense ROS nodelet provides various topics and data to consumers. In order to view the data, we must first launch the nodes that will source those various data streams:

If you haven't already, plug in your camera.

roscore & roscd realsense_camera roslaunch realsense_camera r200_nodelet_rgbd.launch

With the camera node running in the background, you can now continue on to running RViz.

 

If you are having problems running the `roslaunch realsense_camera r200_nodelet_rgbd.launch` command, verify that your camera was detected by the UP board:

ls /dev/video*

You should see three video devices named `video0`, `video1`, and `video2`.

lsmod | grep -q uvcvideo && echo "Kernel module found."

You should see the message "Kernel module found."

If either of the above do not occur, verify your camera is securely plugged into the USB 3 socket. If it is, you may need to update the firmware on your camera. To do so, follow the steps at librealsense: Firmware.

9
Running RViz

RViz can be run from a remote computer or on the UP board. In either instance, roscore needs to be running on the UP board. If you see the message "RViz: waiting for master Could not contact ROS master at [http://localhost:113111], retrying..." then make sure roscore is running.

For full instructions on running RViz, see the RViz user guide.

If you would like to create or use your own launch files, the nodelet name for the R200 camera is R200Nodelet. See the sample launch files for examples of how to launch the nodelet.

roscd realsense_camera rosrun rviz rviz -d rviz/realsense_rgbd_pointcloud.rviz RViz rendering a pointcloud

Once RViz loads, you can use your mouse to move the view around. The projection from the RealSense camera may be behind the default viewpoint, so you may need to zoom out and pan around to find the pointcloud.

If RViz crashes or hangs while loading, make sure you have the r200_nodelet_rgbd.launch running in the background.

View using other commands:

For color stream:

rosrun image_view image_view image:=/camera/rgb/image_raw

For depth stream:

rosrun image_view image_view image:=/camera/depth/image_raw
10
Success!
Congratulations! Your system is now setup.

Comments

I am unable to run Intel Realsense. Everything is installed correctly. Instuction followed exactly The terminal said something similar to no plugged realsense. Here, Jean-Sebastien

Well then something went wrong. I installed it with success but during the installation I mentioned some error-messages that you probably missed when you did the installation in a rush. Let me list the steps for you that I did and that are different from the installation guide above. First: In step 7 "ADD REPOSITORY AND UPDATE UBUNTU PACKAGE LIST" the keyserver link was not reachable. So instead of sudo apt-key adv --keyserver hkp://ha.pool.sks-keyservers.net \ --recv-key 0xB01FA116 I used the following command with success: wget https://raw.githubusercontent.com/ros/rosdistro/master/ros.key -O - | sudo apt-key add - Second: In step 7 "INSTALL ROS PACKAGES AND UPDATE" I had to install the packages step-by-step instead of installing them all together. In addition I had to install the python-rosdep package that is not listed, but required. So instead of sudo apt -y install ros-kinetic-desktop-full python-rosinstall ros-kinetic-realsense-camera use the following commands: sudo apt install python-rosdep sudo apt -y install ros-kinetic-desktop-full sudo apt -y install python-rosinstall sudo apt -y install ros-kinetic-realsense-camera After that you can go on with the installation guide: sudo rosdep init ...and so on. I hope this helps for you too. Edit: It might be that you have to do the command rosdep update BEFORE the command sudo rosdep init Not sure, but give it a try if it still won't work.

Following the instruction above leads to the result: rviz shows me depth and rgb image, but doesn't show infrared (ir1 & ir2) images and no point cloud (grid is displayed).

We've updated the tutorial steps to point to the latest launch files provided by the ROS packages. Instead of r200_nodelet_default.launch use r200_nodelet_rgbd.launch and the rviz/realsense_rgbd_pointcloud.rviz configuration file for RViz. With those, you should now see the an RViz window with a PointCloud2 view, a greyscale depth view, and the RGB camera view.

I am not able to get Ubuntu 16.0.4.3 to install on the up-board. It will only boot from the UBS stick. When USB stick is unplugged, no boot option for Ubuntu is present in the Aptio Boot options menu. NO boot option at all and upon restart the board goes straight into UEFI shell mode. I'm stuck and need assistance.

Were you able to make progress? I just tried a 16.04.3 install on the UP board and it was able to successfully install and boot from the UP board storage. When you installed from the live USB stick, did the installation process give you any warnings or errors?