Multivariate regression estimation when the variables' variances are known a priori / sourced seperately

Question

I'm looking to use a multivariate regression for prediction, but making use of (possibly) superior estimates of variance for both the independent and extraneous variables.

My approach is to standardize the dependent and extraneous variables (by dividing their respective standard deviations derived from the full data history). Once I have the standardized regression coefficients, I use these, together with my separately sourced variance estimates, to get back to an equation that will be used for generating the predictions. Do I simply multiply each coefficient by my separately sourced variance estimate? But what about the dependent variable variance estimate?

As background info, my separate sourcing of variances estimates is because I believe I have better (more up-to-date) estimates than available from the full data history (let's say, because the variances aren't stable through time, I can estimate a more timely measure of variance using higher frequency data over a shorter, recent period).

• Is this approach a sensible standard practice?
• How to go from the standardized betas, to un-standardized ones using the high frequency variance estimates?

I read here (http://stats.stackexchange.com/questions/29781/when-should-you-center-your-data-when-should-you-standardize) about WHEN to standardize, but not clear to me if this covers the case of standardizing both dependent and extraneous variables.

Thank you!

Answer 1

In linear regression when both covariates and outcome are sampled together, the parameter being estimated can be thought of as $$ \mathbf{\beta}=\mathrm{Var}[\,\mathbf{X}\,]^{-1}\mathrm{Cov}[\,Y,\mathbf{X}\,], $$ which you can further break out into terms in the $\mathrm{Corr}[\,Y,X\,]$, $SD[\,Y\,]$ and $SD[\,X\,]$ if you like. In your setting it sounds like you have extra information on some of those terms, but not all.

Taking a Bayesian approach, incorporating extra information about e.g. the $SD[\,Y\,]$ term into your posterior about $\beta$ would be straightforward, assuming you can quantify what the extra data tells you about that term (or terms) via a likelihood. Any component parts of $\beta$ for which you don't have extra information would just be updated in the usual way.

If you don't want to use Bayes, you could construct an estimate of $\beta$ that made the best use of the available data for each component, and then multiply them all together to get some $\hat\beta$. Getting an appropriate standard error estimate for this $\hat\beta$ would be some work, but I imagine e.g. the bootstrap would work reasonably well.

I don't see the need to standardize here... but I may be missing something.

Answer 2

As pointed out by Gung and MansT (http://stats.stackexchange.com/questions/29781/when-should-you-center-your-data-when-should-you-standardize), scaling the variables by their standard deviation corresponds to inverse scaling of the estimated coefficients. Thus if i estimate the regression coefficients using scaled data, i can then use each variable's standard deviation (taken or measured in which ever way i please - not necessarily from the same data set used in estimating the standardized coefficients) to back out the underlying (pre-scaled) coefficients.

The following relationships were useful in helping me understand this process:

The correlation coefficient (r) is a scaled beta.

  r = Cov(x,y)/(sd(x)*sd(y))

b = Cov(x,y)/sd(x)^2

b = r * sd(y) / sd(x)