I am a little confused on incorporating parameter restrictions in the Bayesian linear regression setup.

Assume the multivariate regression

$$R = \iota\alpha+X\beta+U_R$$ where $R$ is a $T \times N$ matrix, $\alpha$ is the $N$ vector of intercept terms, $\iota$ is a vector of ones and $\beta$ is a $N \times K$ matrix of slope coefficients for the $T \times K$ matrix of explanatory variables. The error terms $U_R$ are assumed to be iid Gaussian such that for $t \in 1,\ldots,T$: $$u_t \sim N(0,\Sigma)$$ Textbook theory tells us that for the flat prior $$\pi(\alpha,\beta,\Sigma)\sim |\Sigma|^{-\frac{N+1}{2}}$$ we get the posterior distribution \begin{align} &\text{vec}\left(\Phi\right)|\Sigma, D \propto f_\text{MV}\left(\text{vec}\left(\hat{\Phi}\right), \Sigma\otimes\left({W}'W\right)^{-1}\right)\\ &\Sigma|D \propto f^K_\text{IW}\left(R'Q_{W}R, T-K-1\right) \end{align} where $\text{vec}\left(\cdot\right)$ denotes the vector formed by stacking the successive transformed rows of a matrix, $f_\text{MV}$ denotes the multivariate normal distribution and $f^K_\text{IW}$ denotes the $K$-dimensional inverted Wishart distribution, $\Phi = [\alpha, \beta]$, $W=[\iota, X]$, $\hat \Phi = (W'W)^{-1}W'R$, and $Q_W = I - W'(W'W)^{-1}W$.

Now assume I want to impose some parameter restrictions (for instance given by a theoretical model).

In other words, I want to draw inference about the following model: $$R = X\beta+U_R$$

The error terms $U_R$ are still assumed to be zero-mean iid Gaussian such that for $t \in 1,\ldots,T$.

My question: How does the posterior look like? Can I simply replace $\hat \Phi$ with the regression coefficient of regressing $R$ on $X$ (omitting the intercept), set $W=X$ and adjust the degrees of freedom? Or am I missing something?

  • 1
    $\begingroup$ I don't see this type of regression as anything other than a fixed effects model. Why use an improper flat prior and speculate at the posterior's density? You'd have to estimate that via MCMC any way you cut it. On the other hand, if you apply an inverse gamma prior on the error term, and normal priors on the fixed effects these comprise the conjugate prior and the posterior can be calculated by hand, or more efficiently with MCMC. Fitting a no-intercept model is a trivial extension of the linear model. $\endgroup$ – AdamO Dec 11 '17 at 20:00
  • $\begingroup$ Thanks @AdamO! Can you tell me why I'd need MCMC based on the flat improper prior? And why is this not the case for the Normal Inverse Wishart setup? $\endgroup$ – muffin1974 Dec 11 '17 at 20:29
  • 1
    $\begingroup$ The flat prior is not conjugate because the posterior is not a flat posterior. Obtaining posteriors when using non-conjugate priors is rarely, if ever, analytically tractable. The Wishart is the conjugate prior for the precision (inverse covariance). You can show this using Bayes theorem. $\endgroup$ – AdamO Dec 11 '17 at 20:36
  • $\begingroup$ Thanks for your reply. I get your point. So your statement is: If I'd use the Normal inverse Wishart prior, imposing zero intercept restrictions on $/alpha$ does not bring any additional burden? $\endgroup$ – muffin1974 Dec 11 '17 at 21:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.