Main Content

matlab.tall.blockMovingWindow

Apply moving window function and block reduction to padded blocks of data

Since R2019a

collapse all in page

Syntax

tA = matlab.tall.blockMovingWindow(windowfcn,blockfcn,window,tX)
[tA,tB,...] = matlab.tall.blockMovingWindow(windowfcn,blockfcn,window,tX,tY,...)
[___] = matlab.tall.blockMovingWindow(___,Name,Value)

Description

example

tA= matlab.tall.blockMovingWindow(windowfcn,blockfcn,window,tX)appliesblockfcnto complete windows of data andwindowfcnto incomplete windows of data near the edges.windowspecifies the size of the sliding window. The result contains the vertical concatenation of applyingblockfcnandwindowfcnto these windows of data.

example

[tA,tB,...] = matlab.tall.blockMovingWindow(windowfcn,blockfcn,window,tX,tY,...), wherewindowfcnandblockfcnare function handles that return multiple outputs, returns arraystA, tB, ..., each corresponding to one of the output arguments ofwindowfcnandblockfcn. The inputs towindowfcnandblockfcnare pieces of data from the argumentstX, tY, .... This syntax has these requirements:

  • windowfcnandblockfcnmust return the same number of outputs as were requested frommatlab.tall.blockMovingWindow.

  • Each output ofwindowfcnandblockfcnmust be the same type as the first data inputtX.

  • All outputstA,tB,...must have the same height.

example

[___] = matlab.tall.blockMovingWindow(___,Name,Value)specifies additional options with one or more name-value pair arguments using any of the previous syntaxes. For example, to adjust the step size between windows, you can specify'Stride'and a scalar. Or to change the treatment of endpoints where there are not enough elements to complete a window, you can specify'EndPoints'and a valid option ('shrink',“丢弃”, or a numeric padding value).

Examples

collapse all

Open Live Script

Usematlab.tall.blockMovingWindowto calculate the moving mean of airline arrival and departure delays.

Create a datastore for theairlinesmall.csvdata set and convert it into a tall array. The data contains information about arrival and departure times of US flights. Extract theArrDelayandDepDelayvariables, which are vectors of flight delays, to create a tall array containing the delays as separate columns.

varnames = {'ArrDelay','DepDelay'}; ds = tabularTextDatastore('airlinesmall.csv','TreatAsMissing','NA',...'SelectedVariableNames', varnames); tt = tall(ds); tX = [tt.ArrDelay tt.DepDelay]
tX = Mx2 tall double matrix 8 12 8 1 21 20 13 12 4 -1 59 63 3 -2 11 -1 : : : :

Usematlab.tall.blockMovingWindowto calculate the moving mean of the data in the first dimension with a window size of 10. Sincewindowfcnapplies only to single windows of data, you can use themeanfunction to reduce the windows of data down into a matrix with one row. Theblockfcnapplies to whole blocks of data, so use themovmeanfunction to calculate the mean of each full window of data in the blocks.

windowfcn = @(info,x) mean(x,1,'omitnan'); blockfcn = @(info,x) movmean(x,info.Window,1,'omitnan','EndPoints',“丢弃”); A = matlab.tall.blockMovingWindow(windowfcn, blockfcn, 10, tX)
A = MxNx... tall double array ? ? ? ... ? ? ? ... ? ? ? ... : : : : : :

Gather a portion of the results into memory.

gather(A(1:10,:))
Evaluating tall expression using the Local MATLAB Session: - Pass 1 of 2: Completed in 1.1 sec - Pass 2 of 2: Completed in 4.7 sec Evaluation completed in 6.5 sec
ans =10×210.8000 8.8000 18.8333 17.8333 16.5714 15.0000 15。8750 13.0000 14.4444 11.8889 13.2000 10.8000 14.0000 11.1000 13.5000 11.9000 15.3000 11.4000 19.7000 13.4000
Open Live Script

Calculate moving statistics on the variables of a table.

Load theoutages.csvdata set as a tall table. The data contains information about power outages.

T = tall(readtable('outages.csv'))
T = 1,468x6 tall table Region OutageTime Loss Customers RestorationTime Cause _____________ ________________ ______ __________ ________________ ___________________ {'SouthWest'} 2002-02-01 12:18 458.98 1.8202e+06 2002-02-07 16:50 {'winter storm' } {'SouthEast'} 2003-01-23 00:49 530.14 2.1204e+05 NaT {'winter storm' } {'SouthEast'} 2003-02-07 21:15 289.4 1.4294e+05 2003-02-17 08:14 {'winter storm' } {'West' } 2004-04-06 05:44 434.81 3.4037e+05 2004-04-06 06:10 {'equipment fault'} {'MidWest' } 2002-03-16 06:18 186.44 2.1275e+05 2002-03-18 23:23 {'severe storm' } {'West' } 2003-06-18 02:49 0 0 2003-06-18 10:54 {'attack' } {'West' } 2004-06-20 14:39 231.29 NaN 2004-06-20 19:16 {'equipment fault'} {'West' } 2002-06-06 19:28 311.86 NaN 2002-06-07 00:51 {'equipment fault'} : : : : : : : : : : : :

Usematlab.tall.blockMovingWindowto apply a moving-window function to blocks of the tall table. Specify these options:

  • blkstatsas the block function to operate on complete blocks of data (included at the end of the example as a local function).

  • A window size of 50 and a stride of 5.

  • EndPointsas“丢弃”to ignore incomplete windows of data. With this value, thewindowfcninput can be specified as empty[]since only complete windows of data are operated on.

  • The input table has six variables, but the two outputs are double-precision vectors. Specify scalar doubles as the value forOutputsLikeso that the function permits this change in data type and size.

[A, B] = matlab.tall.blockMovingWindow([], @blkstats, 50, T,'Stride', 5,...'EndPoints',“丢弃”,'OutputsLike', {1, 1});

Preview a few rows in the results.

[A,B] = gather(head(A),head(B))
Evaluating tall expression using the Local MATLAB Session: - Pass 1 of 2: Completed in 0.4 sec - Pass 2 of 2: Completed in 0.61 sec Evaluation completed in 1.5 sec
A =8×1254.0861 254.0861 340.3499 452.0191 464.8524 471.9737 464.8524 464.8524
B =8×1105× 1.3447 1.0779 1.4227 1.4509 1.2888 1.2888 1.2308 1.3722

Theblkstatsfunction calculates the moving median value of theLossandCustomerstable variables in the first dimension using the specified window size. The function applies theStridevalue to reduce the size of the output, and then it returns the results as two vectors.

function[out1, out2] = blkstats(info, t) a = movmedian([t.Loss t.Customers], info.Window, 1,'omitnan','EndPoints',“丢弃”); a = a(1:info.Stride:end, :); out1 = a(:,1); out2 = a(:,2);end

Input Arguments

collapse all

Function to apply to incomplete windows of data, specified as a function handle, anonymous function, or[].windowfcnis invoked once per incomplete window as the calculation moves over data in the tall dimension.matlab.tall.blockMovingWindowapplieswindowfcnonly when'EndPoints'has the default value of'shrink'. If you specify a different value for'EndPoints', then setwindowfcnto[].

Each output ofwindowfcnmust be the same type as the first data inputtX. You can use the'OutputsLike'option to return outputs of different data types.

The general functional signature ofwindowfcnis

[a, b, c, ...] = windowfcn(info, x, y, ...)
Theinfoinput is a structure provided bymatlab.tall.blockMovingWindowthat includes these fields:

  • Stride— Specified step size between windows (default: 1). Set this value with the'Stride'name-value pair.

  • Window— Specified window size. Set this value with thewindowinput argument.

windowfcnmust satisfy these requirements:

  1. Input Arguments— The inputs[x, y, z, ...]are blocks of data that fit in memory. The blocks are produced by extracting data from the respective tall array inputs[tX, tY, tZ, ...]. The inputs[x, y, z, ...]satisfy these properties:

    • All of the inputs[x, y, z, ...]have the same size in the first dimension.

    • The blocks of data in[x, y, z, ...]come from the same index in the tall dimension, assuming the tall array is nonsingleton in the tall dimension. For example, iftXandtYare nonsingleton in the tall dimension, then the first set of blocks might bex = tX(1:20000,:)andy = tY(1:20000,:).

    • When the first dimension of any of[tX, tY, tZ, ...]has a size of1,相应的块[x, y, z, ...]consists of all the data in that tall array.

    • Applyingwindowfcnmust result in a reduction of the input data to a scalar or a slice of an array of height 1.

      When the input is a matrix, N-D array, table, or timetable, applyingwindowfcnmust result in a reduction of the input data in each of its columns or variables.

  2. Output Arguments— The outputs[a, b, c, ...]are blocks that fit in memory to be sent to the respective outputs[tA, tB, tC, ...]. The outputs[a, b, c, ...]satisfy these properties:

    • All of the outputs[a, b, c, ...]must have the same size in the first dimension.

    • All of the outputs[a, b, c, ...]are vertically concatenated with the respective results of previous calls towindowfcn.

    • All of the outputs[a, b, c, ...]are sent to the same index in the first dimension in their respective destination output arrays.

  3. Functional Rules—windowfcnmust satisfy this functional rule:

    • F([inputs1; inputs2]) == [F(inputs1); F(inputs2)]: Applying the function to the concatenation of the inputs should be the same as applying the function to the inputs separately and then concatenating the results.

Example:A = matlab.tall.blockMovingWindow(@windowfcn, @blockfcn, 10, tX)

Example:A = matlab.tall.blockMovingWindow([], @blockfcn, 10, tX, 'EndPoints', 'discard')

Data Types:function_handle

Function to apply to blocks of data, specified as a function handle or anonymous function.blockfcnis applied to blocks of data that contain complete windows of data. Thus,blockfcnmust operate in a vectorized manner on entire blocks of data and return output that has the proper size for the specified window size and stride.

Each output ofblockfcnmust be the same type as the first data inputtX. You can use the'OutputsLike'option to return outputs of different data types.

matlab.tall.blockMovingWindowappliesblockfcnto blocks of data whenever the block contains only complete windows:

  • For middle blocks when'EndPoints'is set to'shrink'(default behavior). In this casewindowfcnoperates on the incomplete windows of data on the ends.

  • For all blocks when'EndPoints'is set to“丢弃”or a padding value.

The general functional signature ofblockfcnis

[a, b, c, ...] = blockfcn(info, bX, bY, bZ, ...)
Theinfoinput is a structure provided bymatlab.tall.blockMovingWindowthat includes these fields:

  • Stride— Specified step size between windows (default: 1). Set this value with the'Stride'name-value pair.

  • Window— Specified window size. Set this value with thewindowinput argument.

The blocks of databX, bY, bZ, ...thatmatlab.tall.blockMovingWindowprovides toblockfcnhave these properties:

  • The blocks contain only full-sized windows.blockfcndoes not have to define a behavior for incomplete windows of data.

  • The first window of data starts at the first element of the block. The last element of the last window is the last element of the block.

blockfcnmust satisfy these requirements:

  1. Input Arguments— The inputs[bX, bY, bZ, ...]are blocks of data that fit in memory. The blocks are produced by extracting data from the respective tall array inputs[tX, tY, tZ, ...]. The inputs[bX, bY, bZ, ...]satisfy these properties:

    • All of the inputs[bX, bY, bZ, ...]have the same size in the first dimension after any allowed expansion.

    • The blocks of data in[bX, bY, bZ, ...]come from the same index in the tall dimension, assuming the tall array is nonsingleton in the tall dimension. For example, iftXandtYare nonsingleton in the tall dimension, then the first set of blocks might bebX = tX(1:20000,:)andbY = tY(1:20000,:).

    • If the first dimension of any of the data inputs[tX, tY, tZ, ...]has a size of1, then the corresponding block[bX, bY, bZ, ...]consists of all the data in that tall array.

    • Applyingblockfcnmust result in a reduction of the input data such that the result has height equal to the number of windows in the block. You can useinfo.Windowandinfo.Strideto determine the number of windows in a block.

      If the input is a matrix, N-D array, table, or timetable, then applyingblockfcnmust result in a reduction of the input data in each of its columns or variables.

  2. Output Arguments— The outputs[a, b, c, ...]are blocks that fit in memory, to be sent to the respective outputs[tA, tB, tC, ...]. The outputs[a, b, c, ...]satisfy these properties:

    • All of the outputs[a, b, c, ...]must have the same size in the first dimension.

    • All of the outputs[a, b, c, ...]are vertically concatenated with the respective results of previous calls toblockfcn.

    • All of the outputs[a, b, c, ...]are sent to the same index in the first dimension in their respective destination output arrays.

  3. Functional Rules—blockfcnmust satisfy this functional rule:

    • F([inputs1; inputs2]) == [F(inputs1); F(inputs2)]: Applying the function to the concatenation of the inputs should be the same as applying the function to the inputs separately and then concatenating the results.

Example:A = matlab.tall.blockMovingWindow(@windowfcn, @blockfcn, 10, tX)

Example:A = matlab.tall.blockMovingWindow([], @blockfcn, 10, tX, 'EndPoints', 'discard')

Data Types:function_handle

窗口大小, specified as a positive integer scalar or a two-element row vector[NB NF].

  • Ifwindowis a scalar, then:

    • When the window size isodd, each window is centered on the corresponding element in the data.

      Illustration of a window size of three for a vector with six elements. There are six windows and the first and last windows have two elements such that each window is centered on the corresponding element in the data.

    • When the window size iseven, each window is centered about the current and previous elements.

      Illustration of a window size of four for a vector with six elements. The first window has two elements, the second has three elements, the next three windows have four elements, and the last window has three elements. Each window is centered about the current and previous elements.

  • Ifwindowis a vector[NB NF], then the window includes the previousNB元素,current element, and the nextNFelements of the inputs.

    Illustration of a window size of [2 2] for a vector with six elements. The first window has three elements, the second has four elements, the next two windows have five elements, the second-to-last window has four elements, and the last window has three elements. Each window includes two previous values (when possible), the current value, and the next two values (when possible).

By default, the window size is automatically truncated at the endpoints when not enough elements are available to fill the window. When the window is truncated in this manner, the function operates only on the elements that fill the window. You can change this behavior with theEndPointsname-value pair.

Data Types:single|double|int8|int16|int32|int64|uint8|uint16|uint32|uint64

Input arrays, specified as separate arguments of scalars, vectors, matrices, multidimensional arrays, tables, or timetables. The input arrays can be tall or in-memory arrays. The input arrays are used as inputs to the transform functionfcn. Each input arraytX,tY,...must have the same height.

Name-Value Arguments

Specify optional pairs of arguments asName1=Value1,...,NameN=ValueN, whereNameis the argument name andValueis the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Before R2021a, use commas to separate each name and value, and encloseNamein quotes.

Example:tA = matlab.tall.blockMovingWindow(@windowfcn, blockfcn, window, tX, 'Stride', 2)

Step size between windows, specified as the comma-separated pair consisting of'Stride'and a positive integer scalar. Afterfcnoperates on a window of data, the calculation advances by the'Stride'value before operating on the next window. Increasing the value of'Stride'from the default value of 1 is the same as reducing the size of the output by picking out every other element, or every third element, and so on.

By default, the value of'Stride'is1, so that each window is centered on each element in the input. For example, here is a moving sum calculation with a window size of 3 operating on the vector[1 2 3 4 5 6]':

插图的移动和与六个向量elements utilizing a stride value of 1. A total of six windows are used in the calculation, so the output has six elements.

If the value of'Stride'is2, then the calculation changes so that each window is centered on every second element in the input (1, 3, 5). The moving sum now returns three partial sums rather than six:

插图的移动和与六个向量elements utilizing a stride value of 2. A total of three windows are used in the calculation, so the output has three elements.

Data Types:single|double|int8|int16|int32|int64|uint8|uint16|uint32|uint64

Method to treat leading and trailing windows, specified as the comma-separated pair consisting of'EndPoints'and one of the values in the table.

At the beginning and end of a windowed calculation, the window of elements being operated on is incomplete. The'EndPoints'option specifies how to treat these incomplete windows.

'EndPoints'Value Description Example: Moving Sum

'shrink'

Shrink the window size near the endpoints of the input to include only existing elements.

插图的移动和与六个向量elements. Six windows are used in the moving sum, with the windows at the endpoints including two elements and interior windows including three elements.

“丢弃”

Do not output any results where the window does not completely overlap with existing elements.

插图的移动和与六个向量elements. Four windows are used in the moving sum, with all windows including three elements.

数字或逻辑填充的值

Substitute nonexisting elements with a specified numeric or logical value.

  • The padding value must have the same type astX.

  • The size of the padding value in the first dimension must be equal to 1, and the size in other dimensions must matchtX.

插图的移动和与六个向量elements. Six windows are used in the moving sum, with the windows at the endpoints including two elements plus a fill value. The interior windows have three elements.

Data Types:single|double|int8|int16|int32|int64|uint8|uint16|uint32|uint64|logical|char|string

Prototype of output arrays, specified as the comma-separated pair consisting of'OutputsLike'and a cell array containing prototype arrays. When you specify'OutputsLike', the output arraystA,tB,...returned bymatlab.tall.blockMovingWindow有相同的数据类型和属性s as the specified prototype arrays{PA,PB,...}. You must specify'OutputsLike'whenever the data type of an output array is different than that of the input array. If you specify'OutputsLike', then you must specify a prototype array for each output.

Example:tA = matlab.tall.blockMovingWindow(..., tX, 'OutputsLike', {int8(1)});, wheretXis a double-precision tall array, returnstAasint8instead ofdouble.

Data Types:cell

Output Arguments

collapse all

Output arrays, returned as scalars, vectors, matrices, or multidimensional arrays. If any input tomatlab.tall.blockMovingWindowis tall, then all output arguments are also tall. Otherwise, all output arguments are in-memory arrays.

  • The size and data type of the output arrays depend on the specified window functionswindowfcnandblockfcn.

  • The output arraystA,tB,...all have the same height, which depends on the value of'Stride'and'EndPoints'. By default the output arrays are the same size as the input arrays.

  • In general, the outputstA,tB,...must all have the same data type as the first data inputtX. However, you can specify'OutputsLike'to return different data types. In cases where the input arraystX, tY, ...are empty, or when'EndPoints'is“丢弃”and there are not enough elements to fill a full-sized window,matlab.tall.blockMovingWindowreturns empty outputs. The sizes of the empty outputs are based on the size of the input arraytX, or on the sizes of the prototype arrays provided to'OutputsLike', if specified.

Tips

  • Usematlab.tall.movingWindowfor simple sliding-window calculations.matlab.tall.blockMovingWindowis an advanced API designed to provide more flexibility to perform sliding-window calculations on tall arrays. As such, it is more complicated to use since the functions must accurately process blocks of data that contain many complete windows. However, with properly vectorized calculations, you can reduce the necessary number of function calls and improve performance.

Version History

Introduced in R2019a

See Also

|

Topics