High Speed Obstacle Avoidance using Monocular Vision and Reinforcement Learning

Home | Publications | Stanford Range Image data | Monocular Depth Estimation | Make3D

We have enabled a radio-controlled car to drive autonomously for several minutes through a cluttered, wooded area before crashing. Robots that are too small to carry many sensors or that must be built cheaply could navigate with just one web-camera using this algorithm. The car gets ability to approximate distances from single still images using this monocular vision algorithm.

See our recent work on Miniature Aerial Vehicles: Obstacle Avoidance!

Videos showing the car driving autonomously (taken Mar 5, 2005):
Video 1 (27 MB, AVI) (Low resolution, MP4) (1.6 MB)
Video 2 (31 MB, AVI) (Low resolution, MP4) (2.1 MB)
Video 3 (4.4 MB, MOV)
Video 4 (138 MB, AVI)

Robot Car. In John Fowler's Cutting Edge (KTVU News). 5 pm, Dec 13, 2005.
The Life and Depth of Robots. In ThomasNet IMT, Jan 10, 2006.

Raw Data
README file for matching laser with image data
parseRangeData.m, createTrainingSet.m
Set 1: Real Images (114 MB), Laser 1D scans (6 MB)
Set 2: Real Images (610 MB), Laser 1D scans (10.5 MB)
Set 3: Synthetic Data, available on request
Stanford Range Image data

Code
ICML version of the vision code
(For full monocular vision code, click here.)

Publications:

High Speed Obstacle Avoidance using Monocular Vision and Reinforcement Learning, Jeff Michels, Ashutosh Saxena, Andrew Y. Ng. Proceedings of the Twenty-first International Conference on Machine Learning (ICML), 2005. [ps, pdf, ppt]

Learning Depth from Single Monocular Images, Ashutosh Saxena, Sung H. Chung, Andrew Y. Ng. In NIPS 18, 2005.

View from the car and predicted distance

A sequence of images showing the car avoiding a series of obstacles.