5
$\begingroup$

What I have:

  1. a linear model $y=a_0+a_1x$ with given parameter estimates,
  2. the number of values used for fitting the model,
  3. the Pearson R² value.

I need to estimate errors of prediction. I don't see a way to calculate it, but is there a way to at least get a rough estimate?

$\endgroup$
  • $\begingroup$ Are you interested in the theoretical aspects or in the practical aspects of doing so with some software? If so, what software do you use? $\endgroup$ – Erik Feb 12 '13 at 13:29
  • $\begingroup$ I use R, but I am hopeful, that I would be able to implement a solution myself. Hence, I am mainly interested in a theoretical solution, but would be also happy with R code. $\endgroup$ – Roland Feb 12 '13 at 15:04
  • $\begingroup$ If that's all you have, the problem is hopeless, because $R^2$ is invariant under changes of scale in the predictand. What other information is available to you? $\endgroup$ – whuber Feb 12 '13 at 17:49
  • $\begingroup$ @whuber That's what I thought and told the phd student. Unfortunately this really is all information, which has been published for this (empirical) model. You don't find much statistics in papers from soil science ... $\endgroup$ – Roland Feb 12 '13 at 18:21
  • 1
    $\begingroup$ It depends on what journals you read :-). The Dutch in particular have been doing a lot with applications of spatial statistics and geostatistics to soils, publishing in Geoderma and other places. BTW, check out Is R^2 useful or dangerous?. $\endgroup$ – whuber Feb 12 '13 at 19:48
1
$\begingroup$

If you know the model was fit by least squares, then we have:

$$\hat{a}_0=\overline{y}-\hat{a}_1\overline{x}\;\;\;\;\;var(\hat{a}_0)=\hat{\sigma}^2\frac{s_x^2+\overline{x}^2}{ns_x^2}$$ $$\hat{a}_1=r_{xy}\frac{s_y}{s_x}\;\;\;\;\;var(\hat{a}_1)=\hat{\sigma}^2\frac{1}{ns_x^2}$$ $$cov(\hat{a}_0,\hat{a}_1)=-\hat{\sigma}^2\frac{\overline{x}}{ns_x^2}$$ $$R^2=r_{xy}^2=1-\frac{(n-2)\hat{\sigma}^2}{ns_y^2}$$

Where $\overline{x}=\frac{1}{n}\sum_ix_i$ and $s_x^2=\frac{1}{n}\sum_i(x_i-\overline{x})^2$ and similarly for $\overline{y},s^2_y$ and $r_{xy}$ is the correlation between $x$ and $y$.

Now the forumal for the prediction error is:

$$mse(\hat{y})=\hat{\sigma}^2(1+\frac{1+z^2}{n})$$

Where $z=\frac{x_p-\overline{x}}{s_x}$ and $x_p$ is the predictor used. Hence you need to know $\hat{\sigma}^2,n,\overline{x},s_x$.

However, you need $s_y^2$ in order to rescale $R^2$ properly. One way to get around this, is to note that:

$$\hat{\sigma}^2=\frac{n}{n-2}s_y^2(1-R^2)=\frac{n}{n-2}\frac{\hat{a}_1^2s_x^2}{R^2}(1-R^2)$$

One rough approximation is to use $\hat{y}^2$ in place of $s_y^2$ to get $\hat{\sigma}^2\approx \frac{n}{n-2}\hat{y}^2(1-R^2)$. We can safely approximate $\hat{z}^2= 4$ provided $x_p$ is "typical" of the units used in the model fitting.

An alternative (better?) approach is to estimate $\hat{\overline{x}}$ and $\hat{s}_x^2$ from the predictors that you have available, say as $\hat{\overline{x}}=\frac{1}{n_p}\sum_{j}x_{pj}$ and $\hat{s}_x^2=\frac{1}{n_p}\sum_j(x_{pj}-\hat{\overline{x}})^2$ where $n_p$ is the number of observations you are producing predictions for with $x_{pj}$ being the predictor for the "jth" observation. Then you replace $\hat{z}_j=\frac{x_{pj}-\hat{\overline{x}}}{\hat{s}_x}$ and $\hat{\sigma}^2\approx \frac{n}{n-2}\hat{a}_1^2\hat{s}_x^2\frac{1-R^2}{R^2}$.

Note that if you add $\overline{x}$ and $s_x^2$ to your available information, then you have everything you need to know about the regression fit.

| cite | improve this answer | |
$\endgroup$
1
$\begingroup$

Okay, I'm sure the folks who know more than I do will correct me if I am off base here. Heck, maybe I'm misinterpreting what you mean when you say "errors of prediction". My interpretation is that you are asking if you can estimate the errors of the slope and of the intercept.

My intuition is that depending on how rough you are willing to accept... it isn't quite hopeless.

General stuff: $\sqrt{R^2}$ gives us the correlation between our predicted values $\hat{y}$ and $y$ and in fact (in the single predictor case) is synonymous with $\beta_{a_1}$. The estimation of the intercept (and intercept error) does not affect this value/correlation.

Solution 1: We know the standard error of a pearson product moment correlation transformed into a Fisher $Z_r$ is $\frac{1}{\sqrt{N-3}}$, so we can find the larger of those distances when we transform back to the scale of r and treat that as the standard error of $\beta$. Knowing the nature of whatever system $x$ is as well as the nature of system $y$ you might be able to speculate regarding the standard deviations and extrapolate a likely scenario for what the naturally scaled error of $a_1$ is.

Solution 2: One worst case scenario is that all of the rest of the variance is in the estimate of the slope. I'm admittedly stumped and this seems like a complex topic, but shesh it really does seem that you should be able to come up with an estimate of the standard error of the standardized slope given these constraints.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Translation: Is there really no set of crazy assumptions we can make to come up with a wild estimate of some kind? $\endgroup$ – russellpierce Feb 13 '13 at 9:13
  • $\begingroup$ The goal is not to give standard errors for the coefficients. My phd student actually uses the model to predict values and needs associated errors (standard errors of predictions) for error propagation. I believe, it would be possible to use a Monte-Carlo simulation to obtain an approximation, if we had the variance-covariance matrix, but standard errors of the coefficient estimates alone are probably not sufficient, or are they? $\endgroup$ – Roland Feb 13 '13 at 9:30
  • $\begingroup$ I'm sorry, I'm not clear on what you mean by error propagation... are you stacking models on top of models? Are your standard errors of predictions typically derived from the difference between $y$ and the model predicted y ($\hat{y}$), i.e. the residuals? $\endgroup$ – russellpierce Feb 13 '13 at 9:38
  • $\begingroup$ This is just a small part of (let's call it) a model framework being developed, so yes, there is another model which is on top of this model (and I see some fundamental problems with the whole approach, but was only asked to help deriving an error for the predictions of the model framework). Now you make me doubt terminology: I need $se(\hat{y_0})$, i.e. the standard errors you would use to construct a prediction interval. And I believe that I don't have enough information to calculate it, but wanted to be sure. Hence, my question. $\endgroup$ – Roland Feb 13 '13 at 10:05
  • $\begingroup$ Your terminology is probably fine. My comprehension is somewhat limited and I know that convention also varies between fields. I think what you are saying is that you want the standard error of the mean for $\hat{y}$. If so, then I think you are right, there just isn't enough information to even try. Getting the standard errors of the estimates (slope and intercept) might be a start, but my approach seems like it is at a dead loss to predict intercept error separate from slope error, and the answer would also vary as a function of the particular x values selected. $\endgroup$ – russellpierce Feb 13 '13 at 15:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.