Deep Learning in Parallel and in the Cloud

扩大深度学习with multiple GPUs locally or in the cloud and train multiple networks interactively or in batch jobs

Train deep networks on multiple GPUs, clusters, and clouds, using Parallel Computing Toolbox™. Scale up deep learning with multiple GPUs locally or on clusters, and train multiple networks interactively or in batch jobs. To learn about options, seeScale Up Deep Learning in Parallel, on GPUs, and in the Cloud.

Topics

Scale Up Deep Learning in Parallel, on GPUs, and in the Cloud
Options for deep learning with MATLAB^®in parallel and using multiple GPUs, locally or in the cloud.
Deep Learning with MATLAB on Multiple GPUs
Speed up deep neural network training using multiple GPUs locally or in the cloud.
Deep Learning with Big Data
Train deep neural networks using large amounts of data.
Deep Learning in the Cloud
Access MATLAB in the cloud for deep learning.
Use Experiment Manager to Train Networks in Parallel
Run multiple simultaneous trials or one trial at a time on multiple workers.
Run Custom Training Loops on a GPU and in Parallel
Speed up custom training loops by running on a GPU, in parallel using multiple GPUs, or on a cluster.
Cloud AI Workflow Using the Deep Learning Container
An example workflow for training, importing data, and optimizing a deep neural network in the cloud using the Deep Learning Container.
- STEP 1:Train a Network on Amazon Web Services Using MATLAB Deep Learning Container
- STEP 2:Use Amazon S3 Buckets with MATLAB Deep Learning Container
- STEP 3:Use Experiment Manager in the Cloud with MATLAB Deep Learning Container

Featured Examples

Train Network in the Cloud Using Automatic Parallel Support

Train a convolutional neural network using MATLAB automatic support for parallel training. Deep learning training often takes hours or days. With parallel computing, you can speed up training using multiple graphical processing units (GPUs) locally or in a cluster in the cloud. If you have access to a machine with multiple GPUs, then you can complete this example on a local copy of the data. If you want to use more resources, then you can scale up deep learning training to the cloud. To learn more about your options for parallel training, see Scale Up Deep Learning in Parallel and in the Cloud. This example guides you through the steps to train a deep learning network in a cluster in the cloud using MATLAB automatic parallel support.

Open Live Script

Train Network Using Automatic Multi-GPU Support

Use multiple GPUs on your local machine for deep learning training using automatic parallel support. Training deep learning networks often takes hours or days. With parallel computing, you can speed up training using multiple GPUs. To learn more about options for parallel training, see Scale Up Deep Learning in Parallel and in the Cloud.

Open Live Script

Create a Deep Learning Experiment for Classification

Train a deep learning network for classification using Experiment Manager.

Open Example

Create a Deep Learning Experiment for Regression

Train a deep learning network for regression using Experiment Manager.

Open Example

Train Deep Learning Networks in Parallel

Run multiple deep learning experiments on your local machine. Using this example as a template, you can modify the network layers and training options to suit your specific application needs. You can use this approach with a single or multiple GPUs. If you have a single GPU, the networks train one after the other in the background. The approach in this example enables you to continue using MATLAB® while deep learning experiments are in progress.

Open Live Script

Use parfor to Train Multiple Deep Learning Networks

Use a parfor loop to perform a parameter sweep on a training option.

Open Live Script

Use parfeval to Train Multiple Deep Learning Networks

Use parfeval to perform a parameter sweep on the depth of the network architecture for a deep learning network and retrieve data during training.

Open Live Script

Upload Deep Learning Data to the Cloud

Upload data to an Amazon S3 bucket.

Open Live Script

Train Network in Parallel with Custom Training Loop

Set up a custom training loop to train a network in parallel. In this example, parallel workers train on portions of the overall mini-batch. If you have a GPU, then training happens on the GPU. During training, a DataQueue object sends training progress information back to the MATLAB client.

Open Live Script

Train Network Using Federated Learning

Train a network using federated learning. Federated learning is a technique that enables you to train a network in a distributed, decentralized way [1].

Open Live Script

Send Deep Learning Batch Job to Cluster

Send deep learning training batch jobs to a cluster so that you can continue working or close MATLAB during training.

Open Live Script

Train a Network on Amazon Web Services Using MATLAB Deep Learning Container

Train a deep learning model in the cloud using MATLAB® on an Amazon EC2® instance.

Open Live Script

Use Amazon S3 Buckets with MATLAB Deep Learning Container

Train your deep learning model with training data stored in an Amazon S3™ Bucket and save the trained model to the cloud.

Open Live Script

Use Experiment Manager in the Cloud with MATLAB Deep Learning Container

Fine tune your deep learning network by using Experiment Manager in the cloud. Make use of multiple NVIDIA® high-performance GPUs on an AWS® EC2 instance to run multiple experiments in parallel. Tune the hyperparameters of your network and try different network architectures. You can sweep through a range of hyperparameters automatically and save the results for each variation. Compare the results of your experiments to find the best network.

Open Live Script