1
$\begingroup$

I am trying to retrace the steps that Gary King and colleagues suggest in their article "Making the most of statistical analyses: Improving interpretation and presentation" to calculate quantities of interest for statistical models. To calculate predicted values they propose that we first simulate the uncertainty of the model parameters to ensure that both the fundamental and the model uncertainty are incorporated into predictions from the models. While I find it simple enough to simulate the different model parameter values by drawing from a multivariate normal distribution using the parameter values and the variance-covariance matrix, I am struggling to understand how we might do the same for the dispersion parameter for the distribution. In my case, I am trying to simulate a simple linear regression, so I would need to simulate the uncertainty of the dispersion parameter sigma^2. Is this possible? Am I missing something obvious?

$\endgroup$
2
$\begingroup$

In linear regression,

\begin{equation} y = X\beta + \epsilon \end{equation}

If the errors are independent and $\epsilon \sim N(0,\sigma^2)$ then:

\begin{equation} \frac{1}{\sigma^2}\sum_{i=1}^n\epsilon_i^2\sim\chi^2_n \end{equation}

The above follows from the definition of a $\chi^2$ variable. We never have the errors $\epsilon$, but we observe the residuals, $e$, so we instead work with:

\begin{equation} \frac{1}{\sigma^2}\sum_{i=1}^ne_i^2\sim\chi^2_{n-p} \end{equation}

where $p$ is the number of predictors including the intercept and $n-p$ is the degrees of freedom.

$\sum_{i=1}^ne_i^2$ is the familiar residual sum of squares, so:

\begin{equation} \frac{\textrm{RSS}}{\sigma^2}\sim\chi^2_{n-p} \end{equation}

It then follows that: $\textrm{RSS}=\sigma^2(n-p)$, since the expectation of $\chi^2_{n-p}$ is $n-p$.

All together, you can simulate $\sigma^2$ as the residual sum of squares divided by a $\chi^2$ variable with $n-p$ degrees of freedom:

\begin{equation} \frac{\hat\sigma^2(n-p)}{\chi^2_{n-p}} \end{equation}

I have not come across the Gary King paper in the question. But Gelman and Hill in Chapter 7 of their regression text, Data Analysis Using Regression and Multilevel/Hierarchical Models, combine multivariate normal for coefficients with chi-squared for the dispersion parameter.

$\endgroup$
  • $\begingroup$ Thank you so much for your thorough answer! (Also thank you for the reference - about time I picked up that book again...) $\endgroup$ – user2018396 Sep 23 '18 at 14:33
  • $\begingroup$ Sure, you're welcome. If you accept the answer, you can also mark it as answered. $\endgroup$ – Heteroskedastic Jim Sep 23 '18 at 14:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.