In this project, My goal is to write a software pipeline to detect vehicles in a video (start with the test_video.mp4 and later implement on full project_video.mp4).

The goals / steps of this project are the following:

• Perform a Histogram of Oriented Gradients (HOG) feature extraction on a labeled training set of images and train a classifier Linear SVM classifier
• Apply a color transform and append binned color features, as well as histograms of color, to my HOG feature vector.
• Normalize my features and randomize a selection for training and testing.
• Implement a sliding-window technique and use the trained classifier to search for vehicles in images.
• Run the pipeline on a video stream (start with the test_video.mp4 and later implement on full project_video.mp4) and create a heat map of recurring detections frame by frame to reject outliers and follow detected vehicles.
• Estimate a bounding box for vehicles detected.

Here are links to the labeled data for vehicle and non-vehicle examples to train your classifier. These example images come from a combination of the GTI vehicle image database, the KITTI vision benchmark suite, and examples extracted from the project video itself. You are welcome and encouraged to take advantage of the recently released Udacity labeled dataset to augment your training data.

• Vehicle_detection_clean.py - Source code for the project
• training_result.p - Training result
• test_video_out.mp4 project_video_out.mp4 - Video output

The code for this step is contained in lines 27 through 115 of the file called Vehicle_detection_clean.py.

I started by reading in all the vehicle and non-vehicle images. Here is an example of one of each of the vehicle and non-vehicle classes:

I then explored different color spaces and different skimage.hog() parameters (orientations, pixels_per_cell, and cells_per_block). I grabbed 1st images from each of the two classes and displayed them to get a feel for what the skimage.hog() output looks like.

Here is an example using the RGB color space and HOG parameters of orientations=9, pixels_per_cell=(8, 8) and cells_per_block=(2, 2):

The method extract_features extract the HOG and color based features according to the parameters and output a flattened array of these features.

I tried various combinations of parameters. Here is parameters and their tuning sets. Through the training process, best parameters pattern is selected.

The method Training_process shows the steps of tuning the parameters and getting the optimal result.

• 1. Load the tuning parameters
• 2. According to the parameters, extract HOG and color based features from training set
• 3. Divide training dataset to training set and validation set. Normalize training set.
• 4. Use Linear SVM to train.
• 5. Record results and select best result to write into pickle file training_reslt.p

At the end, following parameters are select as best fit pattern.

Since vehicles look larger if it is near our car. I define the large search windows with larger coordinate Y and small search windows with smaller Y. And we only care about the road part on the image instead of the sky and trees, so I only do search in the downside of the image where Y is larger than 400.

Here is the windows defined,

Ultimately I searched on two scales using YUV 3-channel HOG features plus spatially binned color and histograms of color in the feature vector, which provided a nice result. Here is the test image:

Here's a link to my video result project_video_output.mp4

test_video_output.mp4

I recorded the positions of positive detections in each frame of the video. From the positive detections I created a heatmap and then thresholded that map to identify vehicle positions. I then used scipy.ndimage.measurements.label() to identify individual blobs in the heatmap. I then assumed each blob corresponded to a vehicle. I constructed bounding boxes to cover the area of each blob detected.

Here's an example result showing the heatmap from a series of frames of video, the result of scipy.ndimage.measurements.label() and the bounding boxes then overlaid on the last frame of video:

To process the detection more smoothly and robust between frames. I create a class Vehicles. Using this class, the past 10 frames can be stored and added to the heatmap and then set the threshold to be 2 + len(vehicles_rec.prepos)//2. By this way, the suddenly changed false positive detection can be filtered out.

1. Parameters tuning. Many parameters need to be tuned, so I use a loop to do training and testing to get the best accuracy. But this causes overfitting problem. Test images are used to partly exclude overfitting patterns. Need to collect more data to have a better model. And deep learning method can also be used after having enough data.
2. False positive problem. There are false positives during testing. Heatmap threshold can exclude some but sometimes it can not. Then I restrict the detection area to filter. This method need to collaborate with the detection of current driving lane(left/center/right), since the driving lane decides the possible vehicle appearing area.
3. Vehicle tracking problem. Currently I just simply use historical position of vehicle to help do tracking and detection correcting. More advanced methods can be involved such as kalman filter.