Lane Detection

Lane Detection: Data and Methods

Data sets of videos and photographs of the road as seen through the front windshield of the vehicle are used to assess the lane detection algorithms developed in this project. Individual frames are extracted from the videos to be studied as still images, but the processed frames can then be reassembled into a video afterward. The final images or videos are the original color images with the identified edges or lines marked in blue. The block diagrams below summarize the two methods used:

Method 1: Edge Detection

Edge Detection with Sobel Filters:

To detect edges in the test image, it is convolved with Sobel kernels defined for vertical and horizontal edge detection. These kernels are defined as follows:

S_v = [1 0 -1          S_h = [1 2 1

       2 0 -2                 0 0 0

         1 0 -1]              -1 -2 -1]

The result of the convolution is the gradient calculated at each point in the image in each direction. To obtain the total gradient magnitude from both directions, the square root of the sum of the squared outputs is taken. Where there is high contrast in an image, there are large differences between the values between neighboring pixels; therefore, large gradients exist at those locations and edges are indicated.

Initial attempts using the Sobel filters for edge detection were performed on "ideal" images. "Ideal" images have clearly defined lane lines on the roadway and the road covers the majority of the image area. This set of figures comes from the self-driving car video taken on a clear dry day.

Video Link: https://drive.google.com/file/d/1yr56gurMpz44yYd8MEu30xHnqlxAcd1Z/view?usp=sharing

As seen in the filtered results, the lanes are clearly detected as the primary gradients in the image with the exception of some content at the top of the image where the other cars and sky are brighter than the background. In order to retain fewer marked edges than the original filter indicates, a threshold for the gradient magnitude is defined for the image. For initial results, this threshold is chosen by manually probing the data value on the edges that are known to be lane lines. All values below and above the range that defines that lanes are set to 0 (black) in the image. The threshold used in the case above is 1 to 1.7. While the threshold does well to define the lane lines in this example, a more sophisticated method may also improve the results.

With promising initial results, "non-ideal" images were tested to understand the shortcomings of the method. The two "non-ideal" cases are a photo taken at night and a photo taken on an overcast, rainy day. Additionally, the rainy image is taken on a roadway with poorly defined lane lines.

The figures below illustrate the process described applied to a photo taken at night. There are many light sources in this image (street lights, headlights from oncoming traffic, reflections on the windshield), and each of these areas of contrast is marked as an edge. Defined in the same manner as the initial testing, the threshold used to obtain the image on the bottom right is also 1 to 1.7. This simplistic method of applying a threshold does not eliminate the gradients that come from objects other than the lane lines.

The images below are the original and filtered versions of the rainy and overcast image. As observed in the figures, this method did not work well in this case to identify the lane lines on the road. There is not much contrast between the lane paint and the roadway, nor are there many other clear objects in this image. The threshold used for this example is 0.3 to 0.8. Adjusting the threshold slightly improved the visibility of some objects in the filtered image (lamp posts, for instance), but it did not help distinguish the lane lines.

The contrast was improved to see how that affected the results for this image. Using the "adapthisteq" command in MATLAB, which performs contrast-limited adaptive histogram equalization, the histogram of the intensity distribution in the image is changed to fit an exponential curve. The images below are the results of improving the contrast and filtering the image.

Improving the image contrast improves the filtering results and shows many more detected edges than the previous attempt. However it is still difficult to identify the lane lines and setting a threshold will also include many of the other edges in the image.

To complete the algorithm objectives above, we returned to the video with clear lane markings. For the first 100 frames of the video, the edges were detected and the threshold was defined. In the video below, it is observed that the blue overlay on the lane lines is only present in a segment of the overall image. This is due to a manual mask that was applied to only indicate the lane lines in the frames where other edges remained even after thresholding (at the horizon, for instance). To fill in the lanes that are originally just outlined edges in the algorithm, a color threshold was set for the yellow and white to become blue. The result is a compilation of the video segment that has blue indication in the lanes in front of the moving vehicle.

Method 2: Edge and Line Detection

Masking can help to reduce the irrelevant noise completely by setting a region of interest (ROI) for processing. Since the automated masking is too complex for cameras that are set at the same angle for long-time use, manual masking is applied. The unprocessed image is firstly processed into a uniform resolution of 960*540. For Lane detection, a trapezoid or rectangle is usually used for the basic shape of ROI. Trapezoid masking can filter the irrelevant or misleading edges more effectively, but rectangle masking gives more broad side views for ROI, which helps when identifying lanes around the sharp turns.