Implement Bayesian Linear Regression

Econometrics Toolbox™ includes a self-contained framework that allows you to implement Bayesian linear regression. The framework contains two groups of prior models for the regression coefficientsβ和干扰差异σ²:

Standard Bayesian linear regression prior models — The five prior model objects in this group range from the simple conjugate normal-inverse-gamma prior model through flexible prior models specified by draws from the prior distributions or a custom function. Although standard prior models can serve several purposes, they are best suited for posterior estimation, simulation (from the joint or, for most models, conditional posterior), and forecasting from the posterior predictive distribution.
Bayesian prior models for predictor variable selection — The models in this group can perform Bayesian lasso regression or stochastic search variable selection (SSVS). They are best suited for posterior estimation, during which the predictor selection algorithm occurs. The resulting posterior model is represented by draws from the Gibbs sampler (an empirical model object), and the estimation summary contains the results from the predictor selection algorithm. The estimation procedure does not remove insignificant or redundant variables for you, but the coefficients of a well-tuned model are close to zero. Therefore, as with standard models, you can simulate draws from the posterior distribution or forecast from the posterior predictive distribution without having to remove any variables.

The general workflow for estimating features of the posterior distributions, and then forecasting responses given new predictor data, is:

Usebayeslm创建一个先前的模型对象，该对象代表您对分布或适用于预测器选择的先前模型对象的假设。
将先前的模型对象以及预测变量和响应数据传递给estimate功能。默认，estimatereturns a model object that represents the posterior distribution.
Pass the model object representing the posterior distribution to为了ecast。

For details on the standard Bayesian linear regression workflow, seeWorkflow for Standard Bayesian Linear Regression Models, and for Bayesian predictor selection, seeWorkflow for Bayesian Predictor Selection。

Workflow for Standard Bayesian Linear Regression Models

The following procedure describes the workflow for estimating features of the posterior distributions, and then forecasting responses given predictor data.

Choose a joint prior distribution for (β,σ²）。Then, usingbayeslm，创建贝叶斯线性回归模型对象，该对象完全指定您对联合先验分布的信念。该表包含可用的先验模型对象。

Model Object	Joint Prior Distribution of (β,σ²)	When to Create
`conjugateblm`	π(β\|σ²）是高斯人的卑鄙`Mu`and covarianceσ²`V`。 π(σ²) is inverse gamma with shape`A`和比例`B`。	Create this model object when all of the following are true: You are fairly confident that the parameters have the corresponding joint prior, and thatβdepends onσ²。 You want to incorporate your prior knowledge of the prior mean and covariance ofβ以及形状和规模σ²。您需要边际和条件后期的分析形式，它们是两种类型的正常内脉冲偶联分布。
`SemiconJugateBlm`	π(β）是高斯人的卑鄙`Mu`and covariance`V`。 π(σ²) is inverse gamma with shape`A`和比例`B`。 βandσ²are independent.	Create this model object when all of the following are true: You are fairly confident that the parameters have the corresponding joint prior, and thatβandσ²are independent. You want to incorporate your prior knowledge of the prior mean and covariance ofβ以及形状和规模σ²。 You want analytical forms for the conditional posteriors, which are normal-inverse-gamma conjugate conditional distributions.
`diffuseblm`	$π (β, σ^{2}) \propto \frac{1}{σ^{2}} 。$	Create this model object when all of the following are true: You want the posterior to be influenced more by the information in the data than by the prior. The joint prior distribution is inversely proportional toσ²(Jeffreys非信息先验[2]）。 You want analytical forms for the marginal and conditional posteriors, which are the normal-inverse-gamma conjugate distributions for both types.
`customblm`	定制功能的功能句柄，该功能计算关节先验分布的日志。	Create this model object when you want to specify the log of the joint prior distribution. This specification allows for maximum flexibility.

Given the data, estimate features of the posterior distributions. The functions to use in this step depend on your analysis goals.

Function Goals

Function	Goals
`estimate`	Obtain a posterior model object for forecasting. Posterior model objects include: Estimates of the mean and covariance matrix of the marginal posteriorπ(β\|y,x) and the mean and variance ofπ(σ²\|y,x）。边缘后期及其参数值。分析解决方案可用于万博尤文图斯`conjugateblm`and`diffuseblm`prior models. For all other prior models,`estimate`必须使用蒙特卡洛采样。 95% equitailed, credible intervals. For nonanalytical posteriors, 95% equitailed, credible intervals are the 0.025-quantile and the 0.975-quantile of the retained Monte Carlo sample. Estimate the mean and covariance of the conditional distributionπ(β\|σ²,y,x), that is, implement linear regression withσ²固定。 Update an existing posterior distribution based on new data.
`模拟`	Approximate the expected value of a function of the parameters with respect to the joint posteriorπ(β,σ²\|y,x）。也就是说，绘制多个样本（β,σ²) from their joint posterior, apply a function to each draw, and then compute the average of the transformed draws. Draw from the conditional posterior distributionsπ(β\|σ²,y,x) andπ(σ²\|β,y,x）。This selection is convenient for running a Markov chain Monte Carlo (MCMC) sampler, such as a Gibbs sampler.

estimate

Obtain a posterior model object for forecasting. Posterior model objects include:
- Estimates of the mean and covariance matrix of the marginal posteriorπ(β|y,x) and the mean and variance ofπ(σ²|y,x）。
- 边缘后期及其参数值。分析解决方案可用于万博尤文图斯conjugateblmanddiffuseblmprior models. For all other prior models,estimate必须使用蒙特卡洛采样。
- 95% equitailed, credible intervals. For nonanalytical posteriors, 95% equitailed, credible intervals are the 0.025-quantile and the 0.975-quantile of the retained Monte Carlo sample.
Estimate the mean and covariance of the conditional distributionπ(β|σ²,y,x), that is, implement linear regression withσ²固定。
Update an existing posterior distribution based on new data.

模拟

Approximate the expected value of a function of the parameters with respect to the joint posteriorπ(β,σ²|y,x）。也就是说，绘制多个样本（β,σ²) from their joint posterior, apply a function to each draw, and then compute the average of the transformed draws.
Draw from the conditional posterior distributionsπ(β|σ²,y,x) andπ(σ²|β,y,x）。This selection is convenient for running a Markov chain Monte Carlo (MCMC) sampler, such as a Gibbs sampler.

If you have a custom prior model (customblmobject), then choose a Markov chain Monte Carlo (MCMC) sampler when you callestimateor模拟。该表包含支持的MCMC采样器的列表。万博1manbetx选择采样器后，首先尝试默认调谐参数值。

MCMC Sampler Specify Using Description

Hamiltonian Monte Carlo (HMC)

MCMC Sampler	Specify Using	Description
Hamiltonian Monte Carlo (HMC)	`'Sampler',"hmc"`	Because the HMC sampler tunes itself, and resulting samples mix well and converge to their stationary distribution more quickly, try this sampler first. To increase sampling speed, supply the gradient of the log PDF for all or some of the parameters.
随机步行大都市	`'Sampler',"metropolis"`	If the sample size is reasonably large and the prior does not dominate the likelihood, then try this sampler. Supported proposal distributions are multivariate normal and multivariatetdistributions. Tuning parameters include the distribution, its scale matrix, and its degrees of freedom.
Slice	`'Sampler',"slice"`(default)	To achieve adequate mixing and convergence, carefully tune the typical sampling-interval width. Values are application dependent.

'Sampler',"hmc"

Because the HMC sampler tunes itself, and resulting samples mix well and converge to their stationary distribution more quickly, try this sampler first.

To increase sampling speed, supply the gradient of the log PDF for all or some of the parameters.

随机步行大都市

'Sampler',"metropolis"

If the sample size is reasonably large and the prior does not dominate the likelihood, then try this sampler.

Supported proposal distributions are multivariate normal and multivariatetdistributions.

Tuning parameters include the distribution, its scale matrix, and its degrees of freedom.

Slice 'Sampler',"slice"(default) To achieve adequate mixing and convergence, carefully tune the typical sampling-interval width. Values are application dependent.

After estimating a nonanalytical posterior by using an MCMC sampler, inspect the posterior or conditional posterior draws for adequate mixing. For more details, see后验估计和仿真诊断。

If the quality of the samples is not satisfactory, then create a sampler options structure by usingsampleroptions, which allows you to specify the tuning parameter values that are appropriate for the sampler. For example, to specify a random walk Metropolis sampler that uses a multivariatetproposal distribution with 5 degrees of freedom, enter:

选项= SamplerOptions（'Sampler',"metropolis",'分配',“ MVT”,。。。'自由度',5)

After you create the sampler options structure, specify it when you callestimateor模拟通过使用'选项'名称-值对的论点。

Forecast responses given the new predictor data using为了ecast。The为了ecast功能构建体从后验预测分布进行预测。分析后验预测分布可用于conjugateblmanddiffuseblmprior models. For all other prior models,为了ecastresorts to Monte Carlo sampling. As with estimation and simulation, you can choose an MCMC sampler forcustomblmmodels. If为了ecastuses an MCMC sampler, you should inspect the posterior or conditional posterior draws for adequate mixing.

Workflow for Bayesian Predictor Selection

以下步骤描述了为线性回归模型执行贝叶斯预测器选择的工作流程，然后预测响应给定的预测数据。

Econometrics Toolbox supports two Bayesian predictor selection algorithms: Bayesian lasso regression and SSVS. Choose a predictor selection algorithm, which implies a joint prior distribution for (β,σ²）。Then, usingbayeslm，创建执行选定预测器选择算法的贝叶斯线性回归先验模型对象，并选择指定调谐参数值。该表包含用于预测器选择的可用先验模型对象。有关先前分布形式的详细信息，请参见Posterior Estimation and Inference。

Model Object Predictor Selection Algorithm Tuning Parameter When to Create

Model Object	Predictor Selection Algorithm	Tuning Parameter	When to Create
`MixConjugateBlm`	SSVS[1]	高斯混合物方差，由`'v'`名称-值对的论点。指定一个两列positive matrix. The first column contains the variances for the variable-inclusion component. 指定相对较大的值。 Values attribute higher probability to coefficients further away from 0. The second column contains the variances for the variable-exclusion component. 指定相对较小的值。 Values attribute higher probability to coefficients close to 0.	The prior variance ofβ是σ²。 You want posterior estimates of the probability that a variable is included in the model.
`mixsemiconjugateblm`	SSVS	Same as`MixConjugateBlm`	βandσ²是独立的，先验。 You want posterior estimates of the probability that a variable is included in the model.
`lassoblm`	贝叶斯拉索回归[3]	λ, specified by the`'Lambda'`名称-值对的论点。You can supply a positive scalar or vector. 较大的值表示更多的正则化，这意味着β密集约为零。 If predictor variables differ in scale by orders of magnitude, then supply a vector of shrinkages, where elements correspond to coefficients.	You want to force insignificant and redundant predictor variables to have coefficients with posterior mode of zero and fairly tight 95% posterior credible intervals around that mode.

MixConjugateBlm

SSVS[1]

高斯混合物方差，由'v'名称-值对的论点。指定一个两列positive matrix.
The first column contains the variances for the variable-inclusion component.
- 指定相对较大的值。
- Values attribute higher probability to coefficients further away from 0.
The second column contains the variances for the variable-exclusion component.
- 指定相对较小的值。
- Values attribute higher probability to coefficients close to 0.

The prior variance ofβ是σ²。
You want posterior estimates of the probability that a variable is included in the model.

mixsemiconjugateblm

SSVS

Same asMixConjugateBlm

βandσ²是独立的，先验。
You want posterior estimates of the probability that a variable is included in the model.

lassoblm

贝叶斯拉索回归[3]

λ, specified by the'Lambda'名称-值对的论点。You can supply a positive scalar or vector.
较大的值表示更多的正则化，这意味着β密集约为零。
If predictor variables differ in scale by orders of magnitude, then supply a vector of shrinkages, where elements correspond to coefficients.

You want to force insignificant and redundant predictor variables to have coefficients with posterior mode of zero and fairly tight 95% posterior credible intervals around that mode.

因为预测器选择算法可以对预测变量的不同尺度敏感（尤其是贝叶斯拉索回归），所以通过将数据传递到boxplot, or by estimating their means and standard deviations by usingmeanandstd, respectively.
对于SSVS先验模型，请使用使用系数的先验分布阴谋。The plots give you an idea about how the densities of the two Gaussian components balance. You can adjust the variances of a coefficient by using dot notation. For example, to attribute prior variances of 1000 and 0.1 to the variable-inclusion and variable-exclusion components, respectively, for coefficientj，进入：
```
priormdl.v（j,:) = [1000 0.1];
```
For more details on specifyingV, see[1]。
For lasso prior models, determine aregularization path，也就是，一系列值λto iterate through during posterior estimation. Values are data dependent.
- If the predictor variables have similar scales, specify a left boundary that forces most variables into the model (that is, attribute almost no penalty), specify a right boundary that forces almost all coefficients to 0, and specify enough values in between the boundaries to search the space adequately. By default, the software attributes a shrinkage of0.01to the intercept and1to all coefficients.
- 如果预测变量的尺度不同orders of magnitude, then you can rescale or standardize the data. However, these actions make interpreting the coefficients difficult. Instead, specify a regularization path and use it to create a matrix of repeated rows. For those variables whose scales are small or large, multiply the corresponding rows by the appropriate order of magnitude to magnify the penalty for small coefficients or reduce the penalty for large coefficients.
  For example, suppose a regression model has three predictors. The first two predictors have similar scales, but the third predictor has a scale that is 3 orders larger. Suppose the 100-element regularization path is in the 1-by-100 vector兰姆达。To create a matrix of shrinkage values, enter the following code:
  p = 3预测变量的％numcoeffs = p + 1;% Number of coefficients, intercept and predictors兰姆达Mat = repmat(Lambda,4,1); LambdaMat(4,:) = 1e3*LambdaMat(4,:);
  For more on specifyingλ, see[3]。
In a为了loop, pass the predictor and response data toestimate为了估计调谐参数每个值的后验分布的特征。对于每次迭代，将后验模型对象和估计摘要表存储在单元格数组中。estimateuses the Gibbs sampler to sample successively from the full conditionals (seeAnalytically Tractable Posteriors）。You can change aspects of the Gibbs sampler, such as the thinning factor, using the available options.
Posterior models (经验性模型对象）存储从完整条件的绘制等等。
Estimation summary tables are MATLAB^®tables that include:
- 平均值的估计（Mean) and covariance matrix (Covariances) of the marginal posteriorπ(β|y,x），以及均值和差异π(σ²|y,x）。
- 95% equitailed, credible intervals (CI95), which are the 0.025-quantile and the 0.975-quantile of the retained Monte Carlo sample.
- 后变量融合概率（Regime）仅适用于先验模型。您确定的阈值以下的值表明相应的预测器微不足道或冗余。
Although you estimate multiple posterior models, a good practice is to inspect the posterior draws for adequate mixing, particularly for models estimated using the boundaries of the tuning parameter values. For more details, see后验估计和仿真诊断。
Determine the best-performing posterior model. Two examples are:
- 选择最简单的模型,最大限度地减少的意思squared error (MSE, the means ofSigma2）在所有后验模型中。此方法很简单，但是产生的模型可能无法很好地概括。
- Choose the simplest model that minimizes forecast MSE.
  1. Partition the data into estimation and forecast samples.
  2. 对于所有选定的调整参数值，请使用估计样品数据估算后验分布。
  3. 使用预测样本和通过使用后验预测分布预测的响应来计算预测MSE。
预测响应给出了使用新的预测数据的响应为了ecast。The为了ecastfunction constructs forecasts from the posterior predictive distribution by implementing a Gibbs sampler. Inspect the draws for adequate mixing.

References

[1]George, E. I., and R. E. McCulloch. "Variable Selection Via Gibbs Sampling."Journal of the American Statistical Association。卷。88，第423号，1993年，第881–889页。

[2]Marin, J. M., and C. P. Robert.Bayesian Core: A Practical Approach to Computational Bayesian Statistics.New York: Springer Science+Business Media, LLC, 2007.

[3]Park, T., and G. Casella. "The Bayesian Lasso."Journal of the American Statistical Association。卷。103，第482号，2008年，第681–686页。

也可以看看

Functions

bayeslm|sampleroptions

对象

conjugateblm|SemiconJugateBlm|diffuseblm|customblm|经验性|MixConjugateBlm|mixsemiconjugateblm|lassoblm