Main Content

kfoldEdge

Classification edge for cross-validated classification model

collapse all in page

    Syntax

    E = kfoldEdge(CVMdl)
    E = kfoldEdge(CVMdl,Name,Value)

    Description

    example

    E= kfoldEdge(CVMdl)returns theclassification edgeobtained by the cross-validated classification modelCVMdl. For every fold,kfoldEdgecomputes the classification edge for validation-fold observations using a classifier trained on training-fold observations.CVMdl.XandCVMdl.Ycontain both sets of observations.

    E= kfoldEdge(CVMdl,Name,Value)returns the classification edge with additional options specified by one or more name-value arguments. For example, specify the folds to use or specify to compute the classification edge for each individual fold.

    Examples

    collapse all

    Open Live Script

    Compute thek-fold edge for a model trained on Fisher's iris data.

    Load Fisher's iris data set.

    loadfisheriris

    Train a classification tree classifier.

    tree = fitctree(meas,species);

    Cross-validate the classifier using 10-fold cross-validation.

    cvtree = crossval(tree);

    Compute thek-fold edge.

    edge = kfoldEdge(cvtree)
    edge = 0.8578
    Open Live Script

    Compute the k-fold edge for an ensemble trained on the Fisher iris data.

    Load the sample data set.

    loadfisheriris

    火车100 b的合奏oosted classification trees.

    t = templateTree('MaxNumSplits',1);% Weak learner template tree objectens = fitcensemble(meas,species,'Learners',t);

    Create a cross-validated ensemble fromensand find the classification edge.

    rng(10,'twister')% For reproducibilitycvens = crossval(ens); E = kfoldEdge(cvens)
    E = 3.2033

    Input Arguments

    collapse all

    Cross-validated partitioned classifier, specified as aClassificationPartitionedModel,ClassificationPartitionedEnsemble, orClassificationPartitionedGAMobject. You can create the object in two ways:

    • Pass a trained classification model listed in the following table to itscrossvalobject function.

    • Train a classification model using a function listed in the following table and specify one of the cross-validation name-value arguments for the function.

    Classification Model Function
    ClassificationDiscriminant fitcdiscr
    ClassificationEnsemble fitcensemble
    ClassificationGAM fitcgam
    ClassificationKNN fitcknn
    ClassificationNaiveBayes fitcnb
    ClassificationNeuralNetwork fitcnet
    ClassificationSVM fitcsvm
    ClassificationTree fitctree

    Name-Value Arguments

    Specify optional pairs of arguments asName1=Value1,...,NameN=ValueN, whereNameis the argument name andValueis the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

    Before R2021a, use commas to separate each name and value, and encloseNamein quotes.

    例子:kfoldEdge(CVMdl,'Folds',[1 2 3 5])specifies to use the first, second, third, and fifth folds to compute the classification edge, but to exclude the fourth fold.

    Fold indices to use, specified as a positive integer vector. The elements ofFoldsmust be within the range from1toCVMdl.KFold.

    The software uses only the folds specified inFolds.

    例子:'Folds',[1 4 10]

    Data Types:single|double

    Flag to include interaction terms of the model, specified astrueorfalse. This argument is valid only for a generalized additive model (GAM). That is, you can specify this argument only whenCVMdlisClassificationPartitionedGAM.

    The default value istrueif the models inCVMdl(CVMdl.Trained) contain interaction terms. The value must befalseif the models do not contain interaction terms.

    例子:'IncludeInteractions',false

    Data Types:logical

    Aggregation level for the output, specified as'average','individual', or'cumulative'.

    Value Description
    'average' The output is a scalar average over all folds.
    'individual' The output is a vector of lengthkcontaining one value per fold, wherekis the number of folds.
    'cumulative'

    Note

    如果you want to specify this value,CVMdlmust be aClassificationPartitionedEnsembleobject orClassificationPartitionedGAMobject.

    • 如果CVMdlisClassificationPartitionedEnsemble, then the output is a vector of lengthmin(CVMdl.NumTrainedPerFold). Each elementjis an average over all folds that the function obtains by using ensembles trained with weak learners1:j.

    • 如果CVMdlisClassificationPartitionedGAM, then the output value depends on theIncludeInteractionsvalue.

      • 如果IncludeInteractionsisfalse, thenLis a(1 + min(NumTrainedPerFold.PredictorTrees))-by-1 numeric column vector. The first element ofLis an average over all folds that is obtained only the intercept (constant) term. The(j + 1)th element ofLis an average obtained using the intercept term and the firstjpredictor trees per linear term.

      • 如果IncludeInteractionsistrue, thenLis a(1 + min(NumTrainedPerFold.InteractionTrees))-by-1 numeric column vector. The first element ofLis an average over all folds that is obtained using the intercept (constant) term and all predictor trees per linear term. The(j + 1)th element ofLis an average obtained using the intercept term, all predictor trees per linear term, and the firstjinteraction trees per interaction term.

    例子:'Mode','individual'

    Output Arguments

    collapse all

    Classification edge, returned as a numeric scalar or numeric column vector.

    • 如果Modeis'average', thenEis the average classification edge over all folds.

    • 如果Modeis'individual', thenEis ak-by-1 numeric column vector containing the classification edge for each fold, wherekis the number of folds.

    • 如果Modeis'cumulative'andCVMdlisClassificationPartitionedEnsemble, thenEis amin(CVMdl.NumTrainedPerFold)-by-1 numeric column vector. Each elementjis the average classification edge over all folds that the function obtains by using ensembles trained with weak learners1:j.

    • 如果Modeis'cumulative'andCVMdlisClassificationPartitionedGAM, then the output value depends on theIncludeInteractionsvalue.

      • 如果IncludeInteractionsisfalse, thenLis a(1 + min(NumTrainedPerFold.PredictorTrees))-by-1 numeric column vector. The first element ofLis the average classification edge over all folds that is obtained using only the intercept (constant) term. The(j + 1)th element ofLis the average edge obtained using the intercept term and the firstjpredictor trees per linear term.

      • 如果IncludeInteractionsistrue, thenLis a(1 + min(NumTrainedPerFold.InteractionTrees))-by-1 numeric column vector. The first element ofLis the average classification edge over all folds that is obtained using the intercept (constant) term and all predictor trees per linear term. The(j + 1)th element ofLis the average edge obtained using the intercept term, all predictor trees per linear term, and the firstjinteraction trees per interaction term.

    More About

    collapse all

    Classification Edge

    Theclassification edgeis the weighted mean of theclassification margins.

    One way to choose among multiple classifiers, for example to perform feature selection, is to choose the classifier that yields the greatest edge.

    Classification Margin

    Theclassification marginfor binary classification is, for each observation, the difference between the classification score for the true class and the classification score for the false class. Theclassification marginfor multiclass classification is the difference between the classification score for the true class and the maximal score for the false classes.

    如果the margins are on the same scale (that is, the score values are based on the same score transformation), then they serve as a classification confidence measure. Among multiple classifiers, those that yield greater margins are better.

    Algorithms

    kfoldEdgecomputes the classification edge as described in the correspondingedgeobject function. For a model-specific description, see the appropriateedgefunction reference page in the following table.

    Model Type edgeFunction
    Discriminant analysis classifier edge
    Ensemble classifier edge
    Generalized additive model classifier edge
    k-nearest neighbor classifier edge
    Naive Bayes classifier edge
    Neural network classifier edge
    Support vector machine classifier edge
    二叉决策树的多类分类 edge

    Extended Capabilities

    Version History

    Introduced in R2011a

    expand all

    Behavior changed in R2022a

    See Also

    ||||