If Deep Learning Toolbox™ does not provide the layer you require for your task, then you can define your own custom layer using this example as a guide. For a list of built-in layers, seeList of Deep Learning Layers.
To define a custom deep learning layer, you can use the template provided in this example, which takes you through the following steps:
Name the layer — Give the layer a name so that you can use it in MATLAB®.
Declare the layer properties — Specify the properties of the layer including learnable parameters and state parameters.
Create a constructor function (optional) — Specify how to construct the layer and initialize its properties. If you do not specify a constructor function, then at creation, the software initializes theName
,Description
, andType
properties with[]
and sets the number of layer inputs and outputs to 1.
Create forward functions — Specify how data passes forward through the layer (forward propagation) at prediction time and at training time.
Create reset state function (optional) — Specify how to reset state parameters.
Create a backward function (optional) — Specify the derivatives of the loss with respect to the input data and the learnable parameters (backward propagation). If you do not specify a backward function, then the forward functions must supportdlarray
objects.
When defining the layer functions, you can usedlarray
objects.Usingdlarray
objects makes working with high dimensional data easier by allowing you to label the dimensions. For example, you can label which dimensions correspond to spatial, time, channel, and batch dimensions using the"S"
,"T"
,"C"
, and"B"
labels, respectively. For unspecified and other dimensions, use the"U"
label. Fordlarray
object functions that operate over particular dimensions, you can specify the dimension labels by formatting thedlarray
object directly, or by using theDataFormat
option.
Using formatteddlarray
objects in custom layers also allows you to define layers where the inputs and outputs have different formats, such as layers that permute, add, or remove dimensions. For example, you can define a layer that takes as input a mini-batch of images with format"SSCB"
(spatial, spatial, channel, batch) and output a mini-batch of sequences with format"CBT"
(channel, batch, time). Using formatteddlarray
objects also allows you to define layers that can operate on data with different input formats, for example, layers that support inputs with formats"SSCB"
(spatial, spatial, channel, batch) and"CBT"
(channel, batch, time).
dlarray
objects also enable support for automatic differentiation. This means that if your forward functions fully supportdlarray
objects, then defining the backward function is optional.
This example shows how to create aproject and reshapelayer, which is a layer commonly used in generative adversarial networks (GANs) that takes an array of noise with format"CB"
(channel, batch) and projects and reshapes it to a mini-batch of images with format"SSCB"
(spatial, spatial, channel, batch) using fully connected, reshape, and relabel operations.
Copy the intermediate layer template into a new file in MATLAB. This template outlines the structure of an intermediate layer class definition. It outlines:
The optionalproperties
blocks for the layer properties, learnable parameters, and state parameters.
The layer constructor function.
Thepredict
function and the optionalforward
function.
The optionalresetState
function for layers with state properties.
The optionalbackward
function.
classdefmyLayer < nnet.layer.Layer% & nnet.layer.Formattable (Optional)properties% (Optional) Layer properties.% Declare layer properties here.endproperties(Learnable)% (Optional) Layer learnable parameters.% Declare learnable parameters here.endproperties(State)% (Optional) Layer state parameters.% Declare state parameters here.endproperties(Learnable, State)% (Optional) Nested dlnetwork objects with both learnable% parameters and state.% Declare nested networks with learnable and state parameters here.endmethodsfunctionlayer = myLayer()% (Optional) Create a myLayer.% This function must have the same name as the class.% Define layer constructor function here.endfunction[Z,state] = predict(layer,X)% Forward input data through the layer at prediction time and% output the result and updated state.%% Inputs:% layer - Layer to forward propagate through% X - Input data% Outputs:% Z - Output of layer forward function% state - (Optional) Updated layer state.%% - For layers with multiple inputs, replace X with X1,...,XN,% where N is the number of inputs.% - For layers with multiple outputs, replace Z with% Z1,...,ZM, where M is the number of outputs.% - For layers with multiple state parameters, replace state% with state1,...,stateK, where K is the number of state% parameters.% Define layer predict function here.endfunction[Z,state,memory] = forward(layer,X)% (Optional) Forward input data through the layer at training% time and output the result, updated state, and a memory% value.%% Inputs:% layer - Layer to forward propagate through% X - Layer input data% Outputs:% Z - Output of layer forward function% state - (Optional) Updated layer state% memory - (Optional) Memory value for custom backward% function%% - For layers with multiple inputs, replace X with X1,...,XN,% where N is the number of inputs.% - For layers with multiple outputs, replace Z with% Z1,...,ZM, where M is the number of outputs.% - For layers with multiple state parameters, replace state% with state1,...,stateK, where K is the number of state% parameters.% Define layer forward function here.endfunctionlayer = resetState(layer)% (Optional) Reset layer state.% Define reset state function here.endfunction[dLdX,dLdW,dLdSin] = backward(layer,X,Z,dLdZ,dLdSout,memory)% (Optional) Backward propagate the derivative of the loss% function through the layer.%% Inputs:% layer - Layer to backward propagate through% X - Layer input data% Z - Layer output data% dLdZ - Derivative of loss with respect to layer% output% dLdSout - (Optional) Derivative of loss with respect% to state output% memory - Memory value from forward function% Outputs:% dLdX - Derivative of loss with respect to layer input% dLdW - (Optional) Derivative of loss with respect to% learnable parameter% dLdSin - (Optional) Derivative of loss with respect to% state input%% - For layers with state parameters, the backward syntax must% include both dLdSout and dLdSin, or neither.% - For layers with multiple inputs, replace X and dLdX with% X1,...,XN and dLdX1,...,dLdXN, respectively, where N is% the number of inputs.% - For layers with multiple outputs, replace Z and dlZ with% Z1,...,ZM and dLdZ,...,dLdZM, respectively, where M is the% number of outputs.% - For layers with multiple learnable parameters, replace% dLdW with dLdW1,...,dLdWP, where P is the number of% learnable parameters.% - For layers with multiple state parameters, replace dLdSin% and dLdSout with dLdSin1,...,dLdSinK and% dLdSout1,...dldSoutK, respectively, where K is the number% of state parameters.% Define layer backward function here.endendend
First, give the layer a name. In the first line of the class file, replace the existing namemyLayer
withprojectAndReshapeLayer
.
classdefprojectAndRehapeLayer < nnet.layer.Layer% & nnet.layer.Formattable (Optional)...end
If you do not specify a backward function, then the layer functions, by default, receiveunformatteddlarray
objects as input. To specify that the layer receivesformatteddlarray
objects as input and also outputs formatteddlarray
objects, also inherit from thennet.layer.Formattable
class when defining the custom layer.
Because a project and reshape layer outputs data with different dimensions as the input data, that is, it outputs data with added spatial dimensions, the layer must also inherit fromnnet.layer.Formattable
. This enables the layer to receive and output formatteddlarray
objects.
Next, specify to inherit from both thennet.layer.Layer
andnnet.layer.Formattable
superclasses.
classdefprojectAndRehapeLayer < nnet.layer.Layer & nnet.layer.Formattable...end
Next, rename themyLayer
constructor function (the first function in themethods
部分),具有相同的名称作为层.
methodsfunctionlayer = projectAndReshapeLayer() ...end...end
Save the layer class file in a new file namedprojectAndReshapeLayer.m
. The file name must match the layer name. To use the layer, you must save the file in the current folder or in a folder on the MATLAB path.
Declare the layer properties in theproperties
section and declare learnable parameters by listing them in theproperties (Learnable)
section.
By default, custom intermediate layers have these properties. Do not declare these properties in theproperties
section.
Property | Description |
---|---|
Name |
Layer name, specified as a character vector or a string scalar. ForLayer array input, thetrainNetwork ,assembleNetwork ,layerGraph , anddlnetwork functions automatically assign names to layers withName set to'' . |
Description |
One-line description of the layer, specified as a string scalar or a character vector. This description appears when the layer is displayed in a If you do not specify a layer description, then the software displays the layer class name. |
Type |
Type of the layer, specified as a character vector or a string scalar. The value of If you do not specify a layer type, then the software displays the layer class name. |
NumInputs |
Number of inputs of the layer, specified as a positive integer. If you do not specify this value, then the software automatically setsNumInputs to the number of names inInputNames . The default value is 1. |
InputNames |
输入层的名称,指定为一个细胞array of character vectors. If you do not specify this value andNumInputs is greater than 1, then the software automatically setsInputNames to{'in1',...,'inN'} , whereN is equal toNumInputs . The default value is{'in'} . |
NumOutputs |
Number of outputs of the layer, specified as a positive integer. If you do not specify this value, then the software automatically setsNumOutputs to the number of names inOutputNames . The default value is 1. |
OutputNames |
Output names of the layer, specified as a cell array of character vectors. If you do not specify this value andNumOutputs is greater than 1, then the software automatically setsOutputNames to{'out1',...,'outM'} , whereM is equal toNumOutputs . The default value is{'out'} . |
If the layer has no other properties, then you can omit theproperties
section.
Tip
If you are creating a layer with multiple inputs, then you must set either theNumInputs
orInputNames
properties in the layer constructor. If you are creating a layer with multiple outputs, then you must set either theNumOutputs
orOutputNames
properties in the layer constructor.For an example, seeDefine Custom Deep Learning Layer with Multiple Inputs.
A project and reshape layer requires an additional property that holds the layer output size. Specify a single property with nameOutputSize
in theproperties
section.
properties% Output sizeOutputSizeend
A project and reshape layer has two learnable parameters: the weights and the biases of the fully connect operation. Declare these learnable parameter in theproperties (Learnable)
section and call the parametersWeights
andBias
, respectively.
properties(Learnable)% Layer learnable parametersWeights Biasend
Create the function that constructs the layer and initializes the layer properties. Specify any variables required to create the layer as inputs to the constructor function.
The project and reshape layer constructor function requires two input arguments:
Layer output size
Number of channels
Layer name (optional, with default name''
)
In the constructor functionprojectAndReshapeLayer
, specify the two required input arguments namedoutputSize
andnumChannels
, and the optional arguments as name-value pairs with the nameNameValueArgs
. Add a comment to the top of the function that explains the syntax of the function.
functionlayer = projectAndReshapeLayer(outputSize,numChannels,NameValueArgs)% layer = projectAndReshapeLayer(outputSize,numChannels)% creates a projectAndReshapeLayer object that projects and% reshapes the input to the specified output size using and% specifies the number of input channels.%% layer = projectAndReshapeLayer(outputSize,numChannels,'Name',name)% also specifies the layer name....end
Parse the input arguments using anarguments
block. List the arguments in the same order as the function syntax and specify the default values. Then, extract the values from theNameValueArgs
input.
% Parse input arguments.argumentsoutputSize numChannels NameValueArgs.Name =''endname = NameValueArgs.Name;
Initialize the layer properties, including learnable parameters in the constructor function. Replace the comment% Layer constructor function goes here
with code that initializes the layer properties.
Set theName
property to the input argumentname
.
% Set layer name.layer.Name = name;
Give the layer a one-line description by setting theDescription
property of the layer. Set the description to describe the type of layer and its size.
% Set layer description.layer.Description ="Project and reshape layer with output size "+ join(string(outputSize));
Specify the type of the layer by setting theType
property. The value ofType
appears when the layer is displayed in aLayer
array.
% Set layer type.layer.Type ="Project and Reshape";
Set the layer propertyOutputSize
to the specified input value.
% Set output size.layer.OutputSize = outputSize;
A project and reshape layer applies a fully connect operation to project the input to batch of images. Initialize the weights using the Glorot initializer and initialize the bias with an array of zeros. The functionsinitializeGlorot
andinitializeZeros
are attached to the exampleTrain Generative Adversarial Network (GAN)as supporting files. To access these functions, open this example as a live script. For more information about initializing learnable parameters for deep learning operations, seeInitialize Learnable Parameters for Model Function.
% Initialize fully connect weights and bias.sz = [prod(outputSize) numChannels]; numOut = prod(outputSize); numIn = numChannels; layer.Weights = initializeGlorot(sz,numOut,numIn); layer.Bias = initializeZeros([prod(outputSize) 1]);
View the completed constructor function.
functionlayer = projectAndReshapeLayer(outputSize,numChannels,NameValueArgs)% layer = projectAndReshapeLayer(outputSize,numChannels)% creates a projectAndReshapeLayer object that projects and% reshapes the input to the specified output size using and% specifies the number of input channels.%% layer = projectAndReshapeLayer(outputSize,numChannels,'Name',name)% also specifies the layer name.% Parse input arguments.argumentsoutputSize numChannels NameValueArgs.Name ='';endname = NameValueArgs.Name;% Set layer name.layer.Name = name;% Set layer description.layer.Description = "Project and reshape layer with output size " + join(string(outputSize));% Set layer type.layer.Type ="Project and Reshape";% Set output size.layer.OutputSize = outputSize;% Initialize fully connect weights and bias.sz = [prod(outputSize) numChannels]; numOut = prod(outputSize); numIn = numChannels; layer.Weights = initializeGlorot(sz,numOut,numIn); layer.Bias = initializeZeros([prod(outputSize) 1]);end
With this constructor function, the commandprojectAndReshapeLayer([4 4 512],100,'Name','proj');
creates a project and reshape layer with name'proj'
that projects the input arrays with size 100 to a batch of 512 4-by-4 images.
Create the layer forward functions to use at prediction time and training time.
Create a function namedpredict
that propagates the data forward through the layer atprediction timeand outputs the result.
Thepredict
function syntax depends on the type of layer.
Z = predict(layer,X)
forwards the input dataX
through the layer and outputs the resultZ
, wherelayer
有一个single input, a single output.
[Z,state] = predict(layer,X)
also outputs the updated state parameterstate
, wherelayer
有一个single state parameter.
You can adjust the syntaxes for layers with multiple inputs, multiple outputs, or multiple state parameters:
For layers with multiple inputs, replaceX
withX1,...,XN
, whereN
is the number of inputs. TheNumInputs
property must matchN
.
For layers with multiple outputs, replaceZ
withZ1,...,ZM
, whereM
is the number of outputs. TheNumOutputs
property must matchM
.
For layers with multiple state parameters, replacestate
withstate1,...,stateK
, whereK
is the number of state parameters.
Tip
If the number of inputs to the layer can vary, then usevarargin
而不是X1,…,XN
. In this case,varargin
is a cell array of the inputs, wherevarargin{i}
corresponds toXi
.
If the number of outputs can vary, then usevarargout
而不是Z1,…,ZN
. In this case,varargout
is a cell array of the outputs, wherevarargout{j}
corresponds toZj
.
Tip
If the custom layer has adlnetwork
object for a learnable parameter, then in thepredict
function of the custom layer, use thepredict
function for thedlnetwork
. Using thedlnetwork
objectpredict
function ensures that the software uses the correct layer operations for prediction.
Because a project and reshape layer has only one input and one output, the syntax forpredict
for a project and reshape layer isZ = predict(layer,X)
.
By default, the layer usespredict
as the forward function at training time. To use a different forward function at training time, or retain a value required for a custom backward function, you must also create a function namedforward
.
The dimensions of the inputs depend on the type of data and the output of the connected layers:
Layer Input | Input Size | Observation Dimension |
---|---|---|
Feature vectors | c-by-N, whereccorresponds to the number of channels andNis the number of observations. | 2 |
2-D images | h-by-w-by-c-by-N, whereh,w, andccorrespond to the height, width, and number of channels of the images, respectively, andNis the number of observations. | 4 |
3-D images | h-by-w-by-d-by-c-by-N, whereh,w,d, andccorrespond to the height, width, depth, and number of channels of the 3-D images, respectively, andNis the number of observations. | 5 |
Vector sequences | c-by-N-by-S, wherecis the number of features of the sequences,Nis the number of observations, andSis the sequence length. | 2 |
2-D image sequences | h-by-w-by-c-by-N-by-S, whereh,w, andccorrespond to the height, width, and number of channels of the images, respectively,Nis the number of observations, andSis the sequence length. | 4 |
3-D image sequences | h-by-w-by-d-by-c-by-N-by-S, whereh,w,d, andccorrespond to the height, width, depth, and number of channels of the 3-D images, respectively,Nis the number of observations, andSis the sequence length. | 5 |
For layers that output sequences, the layers can output sequences of any length or output data with no time dimension. Note that when training a network that outputs sequences using thetrainNetwork
function, the lengths of the input and output sequences must match.
Because the custom layer inherits from thennet.layer.Formattable
class, the layer receives formatteddlarray
与标签对应的输出对象the previous layer.
Theforward
function propagates the data forward through the layer attraining timeand also outputs a memory value.
Theforward
function syntax depends on the type of layer:
Z = forward(layer,X)
forwards the input dataX
through the layer and outputs the resultZ
, wherelayer
有一个single input, a single output.
[Z,state] = forward(layer,X)
also outputs the updated state parameterstate
, wherelayer
有一个single state parameter.
[__,memory] = forward(layer,X)
also returns a memory value for a custombackward
function using any of the previous syntaxes. If the layer has both a customforward
function and a custombackward
function, then the forward function must return a memory value.
You can adjust the syntaxes for layers with multiple inputs, multiple outputs, or multiple state parameters:
For layers with multiple inputs, replaceX
withX1,...,XN
, whereN
is the number of inputs. TheNumInputs
property must matchN
.
For layers with multiple outputs, replaceZ
withZ1,...,ZM
, whereM
is the number of outputs. TheNumOutputs
property must matchM
.
For layers with multiple state parameters, replacestate
withstate1,...,stateK
, whereK
is the number of state parameters.
Tip
If the number of inputs to the layer can vary, then usevarargin
而不是X1,…,XN
. In this case,varargin
is a cell array of the inputs, wherevarargin{i}
corresponds toXi
.
If the number of outputs can vary, then usevarargout
而不是Z1,…,ZN
. In this case,varargout
is a cell array of the outputs, wherevarargout{j}
corresponds toZj
.
Tip
If the custom layer has adlnetwork
object for a learnable parameter, then in theforward
function of the custom layer, use theforward
function of thedlnetwork
object. Using thedlnetwork
objectforward
function ensures that the software uses the correct layer operations for training.
The project and reshape operation consists of a three operations:
Apply a fully connect operations with the learnable weights and biases.
Reshape the output to the specified output size.
Relabel the dimensions so that the output has format'SSCB'
(spatial, spatial, channel, batch)
Implement this operation in thepredict
function. The project and reshape layer does not require memory or a different forward function for training, so you can remove theforward
function from the class file. Add a comment to the top of the function that explains the syntaxes of the function.
functionZ = predict(layer, X)% Forward input data through the layer at prediction time and% output the result.%% Inputs:% layer - Layer to forward propagate through% X - Input data, specified as a formatted dlarray% with a 'C' and optionally a 'B' dimension.% Outputs:% Z - Output of layer forward function returned as% a formatted dlarray with format 'SSCB'.% Fully connect.weights = layer.Weights; bias = layer.Bias; X = fullyconnect(X,weights,bias);% Reshape.outputSize = layer.OutputSize; Z = reshape(X, outputSize(1), outputSize(2), outputSize(3), []);% Relabel.Z = dlarray(Z,'SSCB');end
Tip
If you preallocate arrays using functions such aszeros
, then you must ensure that the data types of these arrays are consistent with the layer function inputs. To create an array of zeros of the same data type as another array, use the"like"
option ofzeros
. For example, to initialize an array of zeros of sizesz
with the same data type as the arrayX
, useZ = zeros(sz,"like",X)
.
Because thepredict
function only uses functions that supportdlarray
objects, defining thebackward
function is optional. For a list of functions that supportdlarray
objects, seeList of Functions with dlarray Support.
View the completed layer class file.
classdefprojectAndReshapeLayer < nnet.layer.Layer & nnet.layer.Formattable% Example project and reshape layer.properties% Output sizeOutputSizeendproperties(Learnable)% Layer learnable parametersWeights Biasendmethodsfunctionlayer = projectAndReshapeLayer(outputSize,numChannels,NameValueArgs)% layer = projectAndReshapeLayer(outputSize,numChannels)% creates a projectAndReshapeLayer object that projects and% reshapes the input to the specified output size using and% specifies the number of input channels.%% layer = projectAndReshapeLayer(outputSize,numChannels,'Name',name)% also specifies the layer name.% Parse input arguments.argumentsoutputSize numChannels NameValueArgs.Name ='';endname = NameValueArgs.Name;% Set layer name.layer.Name = name;% Set layer description.layer.Description = "Project and reshape layer with output size " + join(string(outputSize));% Set layer type.layer.Type ="Project and Reshape";% Set output size.layer.OutputSize = outputSize;% Initialize fully connect weights and bias.sz = [prod(outputSize) numChannels]; numOut = prod(outputSize); numIn = numChannels; layer.Weights = initializeGlorot(sz,numOut,numIn); layer.Bias = initializeZeros([prod(outputSize) 1]);endfunctionZ = predict(layer, X)% Forward input data through the layer at prediction time and% output the result.%% Inputs:% layer - Layer to forward propagate through% X - Input data, specified as a formatted dlarray% with a 'C' and optionally a 'B' dimension.% Outputs:% Z - Output of layer forward function returned as% a formatted dlarray with format 'SSCB'.% Fully connect.weights = layer.Weights; bias = layer.Bias; X = fullyconnect(X,weights,bias);% Reshape.outputSize = layer.OutputSize; Z = reshape(X, outputSize(1), outputSize(2), outputSize(3), []);% Relabel.Z = dlarray(Z,'SSCB');endendend
If the layer forward functions fully supportdlarray
objects, then the layer is GPU compatible. Otherwise, to be GPU compatible, the layer functions must support inputs and return outputs of typegpuArray
(Parallel Computing Toolbox).
Many MATLAB built-in functions supportgpuArray
(Parallel Computing Toolbox)anddlarray
input arguments. For a list of functions that supportdlarray
objects, seeList of Functions with dlarray Support. For a list of functions that execute on a GPU, seeRun MATLAB Functions on a GPU(Parallel Computing Toolbox).To use a GPU for deep learning, you must also have a supported GPU device. For information on supported devices, seeGPU Support by Release(Parallel Computing Toolbox).For more information on working with GPUs in MATLAB, seeGPU Computing in MATLAB(Parallel Computing Toolbox).
In this example, the MATLAB functions used inpredict
all supportdlarray
objects, so the layer is GPU compatible.
Define the following generator network architecture for a GAN, which generates images from 1-by-1-by-100 arrays of random values:
This network:
Converts the random vectors of size 100 to 7-by-7-by-128 arrays using aproject and reshapelayer.
Upscales the resulting arrays to 64-by-64-by-3 arrays using a series of transposed convolution layers and ReLU layers.
Define this network architecture as a layer graph and specify the following network properties.
For the transposed convolution layers, specify 5-by-5 filters with a decreasing number of filters for each layer, a stride of 2, and cropping of the output on each edge.
For the final transposed convolution layer, specify three 5-by-5 filters corresponding to the three RGB channels of the generated images, and the output size of the previous layer.
At the end of the network, include a tanh layer.
To project and reshape the noise input, use the custom layerprojectAndReshapeLayer.
filterSize = 5; numFilters = 64; numLatentInputs = 100; projectionSize = [4 4 512]; layersG = [ featureInputLayer(numLatentInputs,'Normalization','none','Name','in') projectAndReshapeLayer(projectionSize,numLatentInputs,'Name','proj'); transposedConv2dLayer(filterSize,4*numFilters,'Name','tconv1') reluLayer('Name','relu1') transposedConv2dLayer(filterSize,2*numFilters,'Stride',2,'Cropping','same','Name','tconv2') reluLayer('Name','relu2') transposedConv2dLayer(filterSize,numFilters,'Stride',2,'Cropping','same','Name','tconv3') reluLayer('Name','relu3') transposedConv2dLayer(filterSize,3,'Stride',2,'Cropping','same','Name','tconv4') tanhLayer('Name','tanh')]; lgraphG = layerGraph(layersG);
To train the network with a custom training loop and enable automatic differentiation, convert the layer graph to adlnetwork
object.
dlnetG = dlnetwork(lgraphG);
dlarray
|functionLayer
|checkLayer
|setLearnRateFactor
|setL2Factor
|getLearnRateFactor
|getL2Factor
|findPlaceholderLayers
|replaceLayer
|assembleNetwork
|PlaceholderLayer