Preprocess Volumes for Deep Learning

Read Volumetric Data

Supported file formats for volumetric image data include MAT-files, Digital Imaging and Communications in Medicine (DICOM) files, and Neuroimaging Informatics Technology Initiative (NIfTI) files.

Read volumetric image data into anImageDatastore. Read volumetric pixel label data into aPixelLabelDatastore(Computer Vision Toolbox). For more information, seeDatastores for Deep Learning.

The table shows typical usages ofimageDatastoreandpixelLabelDatastorefor each of the supported file formats. When you create the datastore, specify theFileExtensionsname-value argument as the file extensions of your data. Specify theReadFcnproperty as a function handle that reads data of the file format. Thefilepathargument specifies the path to the files or folder containing image data. For pixel label images, the additionalclassNamesandpixelLabelIDarguments specify the mapping of voxel label values to class names.

Image File Format	Create Image Datastore or Pixel Label Datastore
MAT	volds = imageDatastore(filepath,..."FileExtensions",".mat","ReadFcn",@(x) fcn(x)); pxds = pixelLabelDatastore(filepath,classNames,pixelLabelID,..."FileExtensions",".mat","ReadFcn",@(x) fcn(x)); `fcn`is a custom function that reads data from a MAT file. For example, this code defines a function called`matRead`that loads volume data from the first variable of a MAT file. Save the function in a file called`matRead.m`. functiondata = matRead(filename) inp = load(filename); f = fields(inp); data = inp.(f{1});end
DICOM volume in single file	volds = imageDatastore(filepath,..."FileExtensions",".dcm","ReadFcn",@(x) dicomread(x)); pxds = pixelLabelDatastore(filepath,classNames,pixelLabelID,..."FileExtensions",".dcm","ReadFcn",@(x) dicomread(x)); For more information about reading DICOM files, see`dicomread`(Image Processing Toolbox).
DICOM volume in multiple files	Follow these steps. For an example, seeCreate Image Datastore Containing Single and Multi-File DICOM Volumes(Image Processing Toolbox). Aggregate the files into a single study by using the`dicomCollection`(Image Processing Toolbox)function. Read the DICOM data in the study by using the`dicomreadVolume`(Image Processing Toolbox)function. Write each volume as a MAT file. Create the`ImageDatastore`or`PixelLabelDatastore`from the collection of MAT files by following the procedure for MAT files.
NIfTI	volds = imageDatastore(filepath,..."FileExtensions",".nii","ReadFcn",@(x) niftiread(x)); pxds = pixelLabelDatastore(filepath,classNames,pixelLabelID,..."FileExtensions",".nii","ReadFcn",@(x) niftiread(x)); For more information about reading NIfTI files, see`niftiread`(Image Processing Toolbox).

Pair Image and Label Data

To associate volumetric image and label data for semantic segmentation, or two volumetric image datastores for regression, use arandomPatchExtractionDatastore(Image Processing Toolbox). A random patch extraction datastore extracts corresponding randomly-positioned patches from two datastores. Patching is a common technique to prevent running out of memory when training with arbitrarily large volumes. Specify a patch size that matches the input size of the network and, for memory efficiency, is smaller than the full size of the volume, such as 64-by-64-by-64 voxels.

You can also use thecombinefunction to associate two datastores. However, associating two datastores using arandomPatchExtractionDatastorehas some benefits overcombine.

randomPatchExtractionDatastoresupports parallel training, multi-GPU training, and prefetch reading. Specify parallel or multi-GPU training using theExecutionEnvironmentname-value argument oftrainingOptions. Specify prefetch reading using theDispatchInBackgroundname-value argument oftrainingOptions. Prefetch reading requires Parallel Computing Toolbox™.
randomPatchExtractionDatastoreinherently supports patch extraction. In contrast, to extract patches from aCombinedDatastore, you must define your own function that crops images into patches, and then use thetransformfunction to apply the cropping operations.
randomPatchExtractionDatastorecan generate several image patches from one test image. One-to-many patch extraction effectively increases the amount of available training data.

Preprocess Volumetric Data

Deep learning frequently requires the data to be preprocessed and augmented. For example, you may want to normalize image intensities, enhance image contrast, or add randomized affine transformations to prevent overfitting.

To preprocess volumetric data, use thetransformfunction.transformcreates an altered form of a datastore, called anunderlying datastore, by transforming the data read by the underlying datastore according to the set of operations you define in a custom function. Image Processing Toolbox™ provides several functions that accept volumetric input. For a full list of functions, see3-D Volumetric Image Processing(Image Processing Toolbox). You can also preprocess volumetric images using functions in MATLAB^®that work on multidimensional arrays.

The custom transformation function must accept data in the format returned by thereadfunction of the underlying datastore.

Underlying Datastore	Format of Input to Custom Transformation Function
`ImageDatastore`	The input to the custom transformation function depends on the`ReadSize`property. When`ReadSize`is 1, the transformation function must accept an integer array. The size of the array is consistent with the type of images in the`ImageDatastore`. For example, a grayscale image has sizem-by-n, a truecolor image has sizem-by-n-by-3, and a multispectral image withcchannels has sizem-by-n-by-c. When`ReadSize`is greater than 1, the transformation function must accept a cell array of image data corresponding to each image in the batch. For more information, see the`read`function of`ImageDatastore`.
`PixelLabelDatastore`	The input to the custom transformation function depends on the`ReadSize`property. When`ReadSize`is 1, the transformation function must accept a categorical matrix. When`ReadSize`is greater than 1, the transformation function must accept a cell array of categorical matrices. For more information, see the`read`(Computer Vision Toolbox)function of`PixelLabelDatastore`.
`RandomPatchExtractionDatastore`	The input to the custom transformation function must be a table with two columns. For more information, see the`read`(Image Processing Toolbox)function of`RandomPatchExtractionDatastore`.

Underlying Datastore

Format of Input to Custom Transformation Function

ImageDatastore

The input to the custom transformation function depends on theReadSizeproperty.

WhenReadSizeis 1, the transformation function must accept an integer array. The size of the array is consistent with the type of images in theImageDatastore. For example, a grayscale image has sizem-by-n, a truecolor image has sizem-by-n-by-3, and a multispectral image withcchannels has sizem-by-n-by-c.
WhenReadSizeis greater than 1, the transformation function must accept a cell array of image data corresponding to each image in the batch.

For more information, see thereadfunction ofImageDatastore.

PixelLabelDatastore

The input to the custom transformation function depends on theReadSizeproperty.

WhenReadSizeis 1, the transformation function must accept a categorical matrix.
WhenReadSizeis greater than 1, the transformation function must accept a cell array of categorical matrices.

For more information, see theread(Computer Vision Toolbox)function ofPixelLabelDatastore.

RandomPatchExtractionDatastore

The input to the custom transformation function must be a table with two columns.

For more information, see theread(Image Processing Toolbox)function ofRandomPatchExtractionDatastore.

Thetransformfunction must return data that matches the input size of the network. Thetransformfunction does not support one-to-many observation mappings.

To apply random affine transformations to volumetric data inRandomPatchExtractionDatastore, you must use thetransformfunction. TheDataAugmentationproperty of this datastore does not support volumetric data.

Examples

Transform Batch of Volumetric Data in Image Datastore

This example uses:

Open Live Script

This example shows how to transform volumetric data in an image datastore using a sample image preprocessing pipeline.

Specify a set of volumetric images saved at MAT files.

filepath = fullfile(matlabroot,"toolbox","images","imdata","mristack.mat"); files = [filepath; filepath; filepath];

Create an image datastore that stores multiple volumetric images. Specify that theReadSizeof the datastore is greater than 1. Specify a custom read function,matRead. This function is defined in the Supporting Functions section of this example.

volDS = imageDatastore(files,FileExtensions=".mat",...ReadSize=3,ReadFcn=@(x) matRead(x));

Specify the input size of the network.

inputSize = [128 128];

Preprocess the volumetric images involDSusing the custom preprocessing pipeline defined in thepreprocessVolumetricIMDSsupporting function.

dsTrain = transform(volDS,@(x) preprocessVolumetricIMDS(x,inputSize));

Read a batch of data.

minibatch = read(dsTrain)

minibatch=3×1 cell array{128x128x21 uint8} {128x128x21 uint8} {128x128x21 uint8}

Supporting Functions

ThematReadfunction loads volume data from the first variable of a MAT file.

functiondata = matRead(filename) inp = load(filename); f = fields(inp); data = inp.(f{1});end

ThepreprocessVolumetricIMDSfunction performs the desired transformations of data read from an underlying image datastore. Because the read size of the image datastore is greater than 1, the function must accept a cell array of image data. The function loops through each read image and transforms the data according to this preprocessing pipeline:

Randomly rotate the image about thez-axis.
Resize the volume to the size expected by the network.
Create a noisy version of the image with Gaussian noise.
Return the image in a cell array.

functionbatchOut = preprocessVolumetricIMDS(batchIn,inputSize) numRows = size(batchIn,1); batchOut = cell(numRows,1);foridx = 1:numRows% Perform randomized 90 degree rotation about the z-axisimRotated = imrotate3(batchIn{idx,1},90*(randi(4)-1),[0 0 1]);% Resize the volume to the size expected by the networkimResized = imresize(imRotated,inputSize);% Add zero-mean Gaussian noise with a normalized variance of 0.01imNoisy = imnoise(imResized,"gaussian",0.01);% Return the preprocessed databatchOut(idx) = {imNoisy};endend

Transform Volumetric Data in Random Patch Extraction Datastore

This example uses:

Open Live Script

This example shows how to transform pairs of volumetric data in a random patch extraction datastore using a sample image preprocessing pipeline.

Specify two sets of volumetric images saved at MAT files. Each set contains five volumetric images.

dir = fullfile(matlabroot,"toolbox","images","imdata","BrainMRILabeled"); filesVol1 = fullfile(dir,"images"); filesVol2 = fullfile(dir,"labels");

Store each set of volumetric images in an image datastore. Specify a custom read function,matRead. This function is defined in the Supporting Functions section of this example. Use the defaultReadSizeof 1.

vol1DS = imageDatastore(filesVol1,FileExtensions=".mat",ReadFcn=@(x) matRead(x)); vol2DS = imageDatastore(filesVol2,FileExtensions=".mat",ReadFcn=@(x) matRead(x));

Specify the input size of the network.

inputSize = [128 128];

Create a random patch extraction datastore that extracts corresponding patches from the two datastores. Select three patches per image.

patchVolDS = randomPatchExtractionDatastore(vol1DS,vol2DS,inputSize,PatchesPerImage=3);

Preprocess the volumetric images inpatchVolDSusing the custom preprocessing pipeline defined in thepreprocessVolumetricPatchDSsupporting function.

dsTrain = transform(patchVolDS,@(x) preprocessVolumetricPatchDS(x));

Read a batch of data.

minibatch = read(dsTrain)

minibatch=15×2 tableInputImage ResponseImage ____________________ ___________________ {128x128x155 uint16} {128x128x155 uint8} {128x128x155 uint16} {128x128x155 uint8} {128x128x155 uint16} {128x128x155 uint8} {128x128x155 uint16} {128x128x155 uint8} {128x128x155 uint16} {128x128x155 uint8} {128x128x155 uint16} {128x128x155 uint8} {128x128x155 uint16} {128x128x155 uint8} {128x128x155 uint16} {128x128x155 uint8} {128x128x155 uint16} {128x128x155 uint8} {128x128x155 uint16} {128x128x155 uint8} {128x128x155 uint16} {128x128x155 uint8} {128x128x155 uint16} {128x128x155 uint8} {128x128x155 uint16} {128x128x155 uint8} {128x128x155 uint16} {128x128x155 uint8} {128x128x155 uint16} {128x128x155 uint8}

Supporting Functions

ThematReadfunction loads volume data from the first variable of a MAT file.

functiondata = matRead(filename) inp = load(filename); f = fields(inp); data = inp.(f{1});end

ThepreprocessVolumetricPatchDSfunction performs the desired transformations of data read from the underlying random patch extraction datastore. The function must accept a table. The function transforms the data according to this preprocessing pipeline:

Randomly select one of five augmentations.
Apply the same augmentation to the data in both columns of the table.
返回图像增强对表中。

functionbatchOut = preprocessVolumetricPatchDS(batchIn) numRows = size(batchIn,1); batchOut = batchIn;% 5对应:nil, rot90、fliplr flipud,rot90(fliplr)augType = {@(x) x,@rot90,@fliplr,@flipud,@(x) rot90(fliplr(x))};foridx = 1:numRows img = batchIn{idx,1}{1}; resp = batchIn{idx,2}{1}; rndIdx = randi(5,1); imgAug = augType{rndIdx}(img); respAug = augType{rndIdx}(resp); batchOut(idx,:) = {imgAug,respAug};endend

Related Examples

Create Image Datastore Containing Single and Multi-File DICOM Volumes(Image Processing Toolbox)
3-D Brain Tumor Segmentation Using Deep Learning