Lane and Vehicle Detection in ROS Using YOLO v2 Deep Learning Algorithm

This example uses:

Open Live Script

This example shows how to use deep convolutional neural networks inside a ROS enabled Simulink® model to perform lane and vehicle detection. In this example, you first read traffic video as the input and publish the frames assensor_msgs/Imagemessages to a topic on the ROS network. Then you detect vehicles, and the left and right lane boundaries corresponding to the ego vehicle in every frame, annotate the input image with the detections, and publish them to a topic in the ROS network. Finally, you generate CUDA® optimized code for the ROS node from the Simulink model for lane and vehicle detection.

This example uses the pretrained lane detection network from theLane Detection Optimized with GPU Coderexample of GPU Coder Toolbox™. For more information, seeLane Detection Optimized with GPU Coder.

This example also uses the pretrained vehicle detection network from theObject Detection Using YOLO v2 Deep Learningexample of Computer Vision Toolbox™. For more information, seeObject Detection Using YOLO v2 Deep Learning(Computer Vision Toolbox).

This example illustrates the following concepts:

Model the conversion of video frames to ROS image in Simulink.
Model the lane detection application in Simulink. First the traffic video is preprocessed by resizing to 227-by-227-by-3 and multiplying by a constant factor of 255. It is then fed to the pretrained network loaded in thePredictblock from Deep Learning Toolbox™. Finally, if the left and right lane boundaries are detected, the parabolic coefficients to model the trajectories of the lane boundaries are obtained.
Model the vehicle detection application in Simulink. The traffic video is processed by a pretrained YOLO v2 detector. This network detects vehicles in the video and outputs the coordinates of the bounding boxes for these vehicles and their confidence score.
Configure the Simulink model for CUDA ROS node generation on host platform.
Generate a CUDA executable for the Simulink model configured for ROS.

Third-Party Prerequisites

CUDA enabled NVIDIA® GPU.
NVIDIA CUDA toolkit and driver.
NVIDIA cuDNN library.
Environment variables for the compilers and libraries. For more information, seeThird-Party HardwareandSetting Up the Prerequisite Products.

Overview

This example uses three Simulink models, which are each represented as ROS nodes.. The following diagram illustrates their interaction. TherosImagePublisher点头e reads the raw video file and converts the frame data into a ROS image. ThelaneAndVechicleDetection点头e performs lane and vehicle detection and publishes the annotated image. TheannotateImageDisplay点头e displays the annotated images as a video stream.

Download Example Video

if~exist('./caltech_washington1.avi','file') url ='//www.tianjin-qmedu.com/supportfiles/gpucoder/media/caltech_washington1.avi';websave ('caltech_washington1.avi',url);end

Get Pretrained Lane and Vehicle Detection Networks

The function downloads thetrainedLaneNet.matandyolov2ResNet50VehicleExample.matfiles if they are not already present.

getVehicleDetectionAndLaneDetectionNetworks

Lane and Vehicle Detection Simulink Model

ThelaneAndVehicleDetectionmodel performs lane and vehicle detection as shown. When lane images are being published to the topic/rawlaneimgs, the model runs the detection algorithm on them and publishes the annotated images to a different topic. For a more detailed explanation of Lane Detection, Vehicle Detection or Annotation using Vehicle Bounding Boxes and Lane Trajectory subsystems, seeLane and Vehicle Detection in Simulink Using Deep Learning.

Run Simulation

Initialize the ROS master.

rosinit;

Open the Simulink models.

open_system('rosImagePublisher'); open_system('annotatedImageDisplay'); open_system('laneAndVehicleDetection');

Open the Configuration Parameters dialog box of thelaneAndVehicleDetectionmodel. In theSimulation Targetpane, selectGPU acceleration. In theDeep Learningsection, select the target library ascuDNN. Alternatively, you can set the parameters using the following commands.

set_param('laneAndVehicleDetection','GPUAcceleration','on'); set_param('laneAndVehicleDetection','SimDLTargetLibrary','cudnn'); set_param('laneAndVehicleDetection','DLTargetLibrary','cudnn');

First run the models,laneAndVehicleDetectionandannotatedImageDisplay, by selectingRunfrom the fromSimulationtab. Then, runrosImagePublisher, which starts publishing video frames to ROS network. Every new frame received byannotatedImageDisplayis updated in theVideo Viewerwindow.

Generate and Build Simulink Model

In theHardware Implementationpane, selectRobot Operating System (ROS)forHardware Boardand specifyDevice VendorandDevice Type.

In theCode Generationpane, select theLanguageasC++and enableGenerate GPU code.

set_param('laneAndVehicleDetection','TargetLang','C++'); set_param('laneAndVehicleDetection','GenerateGPUCode','CUDA');

In theCode Generation > Libraries > GPU Codepane, enablecuBLAS,cuSOLVERandcuFFT.

set_param('laneAndVehicleDetection','GPUcuBLAS','on'); set_param('laneAndVehicleDetection','GPUcuSOLVER','on'); set_param('laneAndVehicleDetection','GPUcuFFT','on');

To configure ROS node deployment on your local host machine, in theConnectsection of theROStab, set部署到Localhost. Click theBuild and Runbutton from ROS tab to deploy the node. At the end of the build process, you will see the ROS node running on local host machine. Then, runannotatedImageDisplaymodel, you should be able to view the annotated image video in theVideo Vieweroutput.