audioFeatureExtractor
Description
audioFeatureExtractorencapsulates multiple audio feature extractors into a streamlined and modular implementation.
Creation
Description
aFE= audioFeatureExtractor()
aFE= audioFeatureExtractor(Name=Value)aFEusing one or more name-value arguments.
Properties
Object Functions
extract |
Extract audio features |
setExtractorParameters |
Set nondefault parameter values for individual feature extractors |
info |
Output mapping and individual feature extractor parameters |
generateMATLABFunction |
CreateMATLABfunction compatible with C/C++ code generation |
plotFeatures |
Plot extracted audio features |
Examples
Extract Multiple Audio Features
Read in an audio signal.
[audioIn,fs] = audioread("Counting-16-44p1-mono-15secs.wav");
Create anaudioFeatureExtractorobject that extracts the MFCC, delta MFCC, delta-delta MFCC, pitch, spectral centroid, zero-crossing rate, and short-time energy of the signal. Use a 30 ms analysis window with 20 ms overlap.
aFE = audioFeatureExtractor (...SampleRate=fs,...Window=hamming(round(0.03*fs),"periodic"),...OverlapLength=round(0.02*fs),...mfcc=true,...mfccDelta=true,...mfccDeltaDelta=true,...pitch=true,...spectralCentroid=true,...zerocrossrate=true,...shortTimeEnergy=true);
Callextractto extract the audio features from the audio signal.
features = extract(aFE,audioIn);
Useinfoto determine which column of the feature extraction matrix corresponds to the requested pitch extraction.
idx = info(aFE)
idx =struct with fields:mfcc: [1 2 3 4 5 6 7 8 9 10 11 12 13] mfccDelta: [14 15 16 17 18 19 20 21 22 23 24 25 26] mfccDeltaDelta: [27 28 29 30 31 32 33 34 35 36 37 38 39] spectralCentroid: 40 pitch: 41 zerocrossrate: 42 shortTimeEnergy: 43
Plot the detected pitch over time.
t = linspace(0,size(audioIn,1)/fs,size(features,1)); plot(t,features(:,idx.pitch)) title("Pitch") xlabel("Time (s)") ylabel("Frequency (Hz)")
Plot the zero-crossing rate over time.
plot(t,features(:,idx.zerocrossrate)) title("Zero-Crossing Rate") xlabel("Time (s)")
Plot the short-time energy over time.
plot(t,features(:,idx.shortTimeEnergy)) title("Short-Time Energy") xlabel("Time (s)")
Extract Features from Dataset
Create an audio datastore that points to audio samples included with Audio Toolbox®.
folder = fullfile(matlabroot,"toolbox","audio","samples"); ads = audioDatastore(folder);
Find all files that correspond to a sample rate of 44.1 kHz and thensubsetthe datastore.
keepFile = cellfun(@(x)contains(x,"44p1"),ads.Files); ads = subset(ads,keepFile);
Convert the data to atallarray.tallarrays are evaluated only when you request them explicitly usinggather. MATLAB® automatically optimizes the queued calculations by minimizing the number of passes through the data. If you have Parallel Computing Toolbox™, you can spread the calculations across multiple workers. The audio data is represented as anM-by-1 tall cell array, whereMis the number of files in the audio datastore.
adsTall = tall(ads)
Starting parallel pool (parpool) using the 'local' profile ... Connected to the parallel pool (number of workers: 6). adsTall = M×1 tall cell array { 539648×1 double} { 227497×1 double} { 8000×1 double} { 685056×1 double} { 882688×2 double} {1115760×2 double} { 505200×2 double} {3195904×2 double} : : : :
Create anaudioFeatureExtractorobject to extract the mel spectrum, Bark spectrum, ERB spectrum, and linear spectrum from each audio file. Use the default analysis window and overlap length for the spectrum extraction.
aFE = audioFeatureExtractor (SampleRate=44.1e3,...melSpectrum=true,...barkSpectrum=true,...erbSpectrum=true,...linearSpectrum=true);
Define acellfunfunction so that audio features are extracted from each cell of the tall array. Callgatherto evaluate the tall array.
specsTall = cellfun(@(x)extract(aFE,x),adsTall,UniformOutput=false); specs = gather(specsTall);
Evaluating tall expression using the Parallel Pool 'local': - Pass 1 of 1: Completed in 14 sec Evaluation completed in 14 sec
Thespecsvariable returned from gather is anumFiles-by-1 cell array, wherenumFilesis the number of files in the datastore. Each element of the cell array is anumHops-by-numFeatures-by-numChannelsarray, where the number of hops and number of channels depends on the length and number of channels of the audio file, and the number of features is the requested number of features from the audio data.
numFiles = numel(specs)
numFiles = 12
[numHops1,numFeaturesFile1,numChanelsFile1] = size(specs{1})
numHops1 = 1053
numFeaturesFile1 = 620
numChanelsFile1 = 1
[numHops2,numFeaturesFile2,numChanelsFile2] = size(specs{2})
numHops2 = 443
numFeaturesFile2 = 620
numChanelsFile2 = 1
Visualize Extracted Audio Features
UseplotFeaturesto visualize audio features extracted with anaudioFeatureExtractorobject.
Read in an audio signal from a file.
[audioIn,fs] = audioread("Counting-16-44p1-mono-15secs.wav");
Create anaudioFeatureExtractorobject that extracts the gammatone cepstral coefficients (GTCCs) and the delta of the GTCCs. Set theSampleRateproperty to the sample rate of the audio signal, and use the default values for the other properties.
afe = audioFeatureExtractor (SampleRate = fs, gtcc = true,gtccDelta=true);
Plot the features extracted from the audio signal.
plotFeatures(afe,audioIn)
Algorithms
TheaudioFeatureExtractorcreates a feature extraction pipeline based on your selected features. To reduce computations,audioFeatureExtractorreuses intermediary representations and outputs some intermediate representations as features.
例如,要创建一个对象that extracts the centroid of the Bark spectrum, the flux of the Bark spectrum, the pitch, the harmonic ratio, and the delta-delta of the MFCC, specify theaudioFeatureExtractoras follows.
aFE = audioFeatureExtractor (...SpectralDescriptorInput="barkSpectrum",...spectralCentroid=true,...spectralFlux=true,...pitch=true,...harmonicRatio=true,...mfccDeltaDelta=true)
aFE = audioFeatureExtractor with properties: Properties Window: [1024×1 double] OverlapLength: 512 SampleRate: 44100 FFTLength: [] SpectralDescriptorInput: 'barkSpectrum' Enabled Features mfccDeltaDelta, spectralCentroid, spectralFlux, pitch, harmonicRatio Disabled Features linearSpectrum, melSpectrum, barkSpectrum, erbSpectrum, mfcc, mfccDelta gtcc, gtccDelta, gtccDeltaDelta, spectralCrest, spectralDecrease, spectralEntropy spectralFlatness, spectralKurtosis, spectralRolloffPoint, spectralSkewness, spectralSlope, spectralSpread To extract a feature, set the corresponding property to true. For example, obj.mfcc = true, adds mfcc to the list of enabled features.
Note
BecauseaudioFeatureExtractorreuses intermediary representations, the features output fromaudioFeatureExtractormight not correspond with the default configuration of features output by corresponding individual feature extractors.
Extended Capabilities
Version History
Introduced in R2019b
You can also select a web site from the following list:
Americas
- América Latina(Español)
- Canada(English)
- United States(English)
Europe
- Belgium(English)
- Denmark(English)
- Deutschland(Deutsch)
- España(Español)
- Finland(English)
- France(Français)
- Ireland(English)
- Italia(Italiano)
- Luxembourg(English)
- Netherlands(English)
- Norway(English)
- Österreich(Deutsch)
- Portugal(English)
- Sweden(English)
- Switzerland
- United Kingdom(English)