Creating Discriminant Analysis Model

The model for discriminant analysis is:

Each class (Y) generates data (X) using a multivariate normal distribution. In other words, the model assumesXhas a Gaussian mixture distribution (gmdistribution).
- For linear discriminant analysis, the model has the same covariance matrix for each class; only the means vary.
- For quadratic discriminant analysis, both means and covariances of each class vary.

Under this modeling assumption,fitcdiscrinfers the mean and covariance parameters of each class.

For linear discriminant analysis, it computes the sample mean of each class. Then it computes the sample covariance by first subtracting the sample mean of each class from the observations of that class, and taking the empirical covariance matrix of the result.
For quadratic discriminant analysis, it computes the sample mean of each class. Then it computes the sample covariances by first subtracting the sample mean of each class from the observations of that class, and taking the empirical covariance matrix of each class.

Thefitmethod does not use prior probabilities or costs for fitting.

Weighted Observations

fitcdiscrconstructs weighted classifiers using the following scheme. SupposeMis anN-by-Kclass membership matrix:

M_nk= 1 if observationnis from classk
M_nk= 0 otherwise.

The estimate of the class mean for unweighted data is

${\hat{μ}}_{k} = \frac{\sum_{n = 1}^{N} M_{n k} x_{n}}{\sum_{n = 1}^{N} M_{n k}} .$

For weighted data with positive weightsw_n, the natural generalization is

${\hat{μ}}_{k} = \frac{\sum_{n = 1}^{N} M_{n k} w_{n} x_{n}}{\sum_{n = 1}^{N} M_{n k} w_{n}} .$

The unbiased estimate of the pooled-in covariance matrix for unweighted data is

$\hat{Σ} = \frac{\sum_{n = 1}^{N} \sum_{k = 1}^{K} M_{n k} (x_{n} - {\hat{μ}}_{k}) {(x_{n} - {\hat{μ}}_{k})}^{T}}{N - K} .$

For quadratic discriminant analysis,fitcdiscrusesK= 1.

For weighted data, assuming the weights sum to 1, the unbiased estimate of the pooled-in covariance matrix is

$\hat{Σ} = \frac{\sum_{n = 1}^{N} \sum_{k = 1}^{K} M_{n k} w_{n} (x_{n} - {\hat{μ}}_{k}) {(x_{n} - {\hat{μ}}_{k})}^{T}}{1 - \sum_{k = 1}^{K} \frac{W_{k}^{(2)}}{W_{k}}},$

where

$W_{k} = \sum_{n = 1}^{N} M_{n k} w_{n}$ is the sum of the weights for classk.
$W_{k}^{(2)} = \sum_{n = 1}^{N} M_{n k} w_{n}^{2}$ is the sum of squared weights per class.

Creating Discriminant Analysis Model

Weighted Observations

See Also

Functions

Objects

Related Topics

统计和机器学习的工具箱Documentation

万博1manbetx

Mastering Machine Learning: A Step-by-Step Guide with MATLAB