3
$\begingroup$

Suppose we have the regression model $$y_t=x_t'\beta +\lambda_t\epsilon_t$$ with flat prior $p(\beta) \propto 1$ , i.i.d $\epsilon_t \sim N(0,1)$, $\lambda_t$ i.i.d and fixed regressors $x_t$.

Now suppose I am interested in the posterior $p(\beta| \lambda, y) $ where $y$ and $\lambda$ are the vectors with $y_t$ and $\lambda_t$ stacked over time. Because we are looking at the posterior of $\beta$ conditional on $\lambda$, it seems that we could look at usual the regression model with constant error variance $$\frac{y_t}{\lambda_t}=\frac{x_t}{\lambda_t}\beta+\epsilon_t$$ and make use of the known bayesian regression model result which then suggests that $p(\beta| \lambda, y)$ is normal with mean $(\sum_{t=1}^T\lambda^{-2}_tx_tx_t')^{-1}\sum_{t=1}^T\lambda^{-2}_tx_ty_t$ and covariance matrix $(\sum_{t=1}^T\lambda^{-2}_tx_tx_t')^{-1}$.

How can we rigorously justify this step?

$\endgroup$
3
$\begingroup$

Your model for the residuals, $\lambda_t\epsilon_t$ with $\epsilon_t$ i.i.d. $N(0,1)$ and the $\lambda_t$ known, corresponds to the following:

$$y_t = x'_t\beta + e_t$$

with $e_t$ having mean $0$ and covariance matrix $\Sigma$, with $\sigma_{ii}=\lambda_i^2$ and $\sigma_{ij} = 0, \space i\neq j$. The likelihood function corresponding to this formulation is that associated with a multivariate Normal distribution with known covariance matrix:

$$\mathcal{L}(\beta) \propto \exp\left(-{1\over 2}(y-x\beta)^T\Sigma^{-1}(y-x\beta)\right)$$

where $y-x\beta$ is the vector of errors.

Making use of the fact that the non-diagonal elements of $\Sigma$ are all $0$, we can rewrite the exponentiated term in the likelihood function as a sum:

$$\mathcal{L}(\beta) \propto \exp\left(-{1\over 2}\sum_{i=1}^n(y_i-x_i'\beta)\lambda_i^{-2}(y_i-x_i'\beta)\right)$$

which evidently can be rewritten in the form you conjecture is correct:

$$\mathcal{L}(\beta) \propto \exp\left(-{1\over 2}\sum_{i=1}^n\left({y_i \over \lambda_i}-{x_i\over \lambda_i}'\beta\right)^2\right)$$

As you've realized, this implies that the posterior is indeed Normal with the mean and covariance matrix given in your problem statement.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.