主要内容

使用虚幻引擎模拟的深度和语义分割可视化

This example shows how to visualize depth and semantic segmentation data captured from a camera sensor in a simulation environment. This environment is rendered using the Unreal Engine® from Epic Games®.

您可以使用深度可视化来验证传感器的深度估计算法。您可以使用语义分割可视化来分析用于从虚幻引擎环境中生成合成语义分割数据的分类方案。

Model Setup

此示例中使用的模型模拟了在城市场景中驾驶的车辆。

加载包含路点姿势的垫子文件。将时间戳添加到姿势,然后打开模型。

loadsmoothedPoses.mat; refPosesX = [linspace(0,20,1000)', smoothedPoses(:,1)]; refPosesY = [linspace(0,20,1000)', smoothedPoses(:,2)]; refPosesYaw = [linspace(0,20,1000)', smoothedPoses(:,3)]; open_system('DepthSemanticentation.slx')

深度Visualization

A depth map is a grayscale representation of camera sensor output. These maps visualize camera images in grayscale, with brighter pixels indicating objects that are farther away from the sensor. You can use depth maps to validate depth estimation algorithms for your sensors.

The深度模拟3D摄像机块的端口在0到1000米的范围内输出一个值的深度图。在此模型中,为了获得更好的可见性,饱和块将深度输出饱和到最大150米。然后,增益块将深度图缩放到[0,1]范围[0,1],以便对视频显示块可以看到灰度中的深度图。

语义分割可视化

语义细分describes the process of associating each pixel of an image with a class label, such asroad,建筑, ortraffic sign。In the 3D simulation environment, you generate synthetic semantic segmentation data according to a label classification scheme. You can then use these labels to train a neural network for automated driving applications, such as road segmentation. By visualizing the semantic segmentation data, you can verify your classification scheme.

TheLabelsport of the Simulation 3D Camera block outputs a set of labels for each pixel in the output camera image. Each label corresponds to an object class. For example, in the default classification scheme used by the block,1corresponds to buildings. A label of0refers to objects of an unknown class and appears as black. For a complete list of label IDs and their corresponding object descriptions, see theLabelsport description on theSimulation 3D Camerablock reference page.

MATLAB功能块使用Label2RGB功能将标签转换为RGB三重态的矩阵以进行可视化。colormap基于camvid数据集中使用的颜色,如示例所示Semantic Segmentation Using Deep Learning。颜色映射到默认3D仿真场景中使用的预定义标签ID。辅助功能sim3dColormapdefines the colormap. Inspect these colormap values.

openSIM3DCOLORMAP.M

Model Simulation

Run the model.

sim卡('DepthSemanticentation.slx');

When the simulation begins, it can take a few seconds for the visualization engine to initialize, especially when you are running it for the first time. TheAutoVrtlEnvwindow displays the scene from behind the ego vehicle. In this scene, the vehicle drives several blocks around the city. Because this example is mainly for illustrative purposes, the vehicle does not always follow the direction of traffic or the pattern of the changing traffic lights.

相机显示,深度显示和语义分割显示屏块显示了相机传感器的输出。

要更改输出深度数据的可视化范围,请尝试更新饱和度中的值并增益块。

要更改语义分割颜色,请尝试修改定义的颜色值sim3dColormapfunction. Alternatively, in theSIM3DLABEL2RGBMATLAB Function block, try replacing the input colormap with your own colormap or a predefined colormap. See结肠

See Also

||

相关话题