I'm looking to use a multivariate regression for prediction, but making use of (possibly) superior estimates of variance for both the independent and extraneous variables.

My approach is to standardize the dependent and extraneous variables (by dividing their respective standard deviations derived from the full data history). Once I have the standardized regression coefficients, I use these, together with my separately sourced variance estimates, to get back to an equation that will be used for generating the predictions. Do I simply multiply each coefficient by my separately sourced variance estimate? But what about the dependent variable variance estimate?

As background info, my separate sourcing of variances estimates is because I believe I have better (more up-to-date) estimates than available from the full data history (let's say, because the variances aren't stable through time, I can estimate a more timely measure of variance using higher frequency data over a shorter, recent period).

  • Is this approach a sensible standard practice?
  • How to go from the standardized betas, to un-standardized ones using the high frequency variance estimates?

I read here (http://stats.stackexchange.com/questions/29781/when-should-you-center-your-data-when-should-you-standardize) about WHEN to standardize, but not clear to me if this covers the case of standardizing both dependent and extraneous variables.

Thank you!

  • $\begingroup$ Sounds a bit like the Phd thesis of Allan Birnbaum?, where an initial sample was used to estimate the variance, and then that variance estimate was used to test a difference in means in a subsequent sample (using just means from that sample). The purpose of this was to be able to choose a sample size to assure set power (which was now possible given the earlier variance estimate). $\endgroup$
    – phaneron
    Nov 21, 2012 at 13:08
  • $\begingroup$ No, it's a real world problem. $\endgroup$
    – Yugmorf
    Nov 21, 2012 at 14:10
  • $\begingroup$ Sorry, did not mean to suggest yours was not a real world problem but that Phd thesis may outline the solution or suggest a solution to someone who wants to answer your question. $\endgroup$
    – phaneron
    Nov 21, 2012 at 15:35
  • $\begingroup$ No problem. Thank you for correcting my spelling! $\endgroup$
    – Yugmorf
    Nov 22, 2012 at 3:56

2 Answers 2


In linear regression when both covariates and outcome are sampled together, the parameter being estimated can be thought of as $$ \mathbf{\beta}=\mathrm{Var}[\,\mathbf{X}\,]^{-1}\mathrm{Cov}[\,Y,\mathbf{X}\,], $$ which you can further break out into terms in the $\mathrm{Corr}[\,Y,X\,]$, $SD[\,Y\,]$ and $SD[\,X\,]$ if you like. In your setting it sounds like you have extra information on some of those terms, but not all.

Taking a Bayesian approach, incorporating extra information about e.g. the $SD[\,Y\,]$ term into your posterior about $\beta$ would be straightforward, assuming you can quantify what the extra data tells you about that term (or terms) via a likelihood. Any component parts of $\beta$ for which you don't have extra information would just be updated in the usual way.

If you don't want to use Bayes, you could construct an estimate of $\beta$ that made the best use of the available data for each component, and then multiply them all together to get some $\hat\beta$. Getting an appropriate standard error estimate for this $\hat\beta$ would be some work, but I imagine e.g. the bootstrap would work reasonably well.

I don't see the need to standardize here... but I may be missing something.


As pointed out by Gung and MansT (http://stats.stackexchange.com/questions/29781/when-should-you-center-your-data-when-should-you-standardize), scaling the variables by their standard deviation corresponds to inverse scaling of the estimated coefficients. Thus if i estimate the regression coefficients using scaled data, i can then use each variable's standard deviation (taken or measured in which ever way i please - not necessarily from the same data set used in estimating the standardized coefficients) to back out the underlying (pre-scaled) coefficients.

The following relationships were useful in helping me understand this process:

The correlation coefficient (r) is a scaled beta.

  r = Cov(x,y)/(sd(x)*sd(y))

b = Cov(x,y)/sd(x)^2

b = r * sd(y) / sd(x)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.