Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I'm looking to use a multivariate regression for prediction, but making use of (possibly) superior estimates of variance for both the independent and extraneous variables.

My approach is to standardize the dependent and extraneous variables (by dividing their respective standard deviations derived from the full data history). Once I have the standardized regression coefficients, I use these, together with my separately sourced variance estimates, to get back to an equation that will be used for generating the predictions. Do I simply multiply each coefficient by my separately sourced variance estimate? But what about the dependent variable variance estimate?

As background info, my separate sourcing of variances estimates is because I believe I have better (more up-to-date) estimates than available from the full data history (let's say, because the variances aren't stable through time, I can estimate a more timely measure of variance using higher frequency data over a shorter, recent period).

  • Is this approach a sensible standard practice?
  • How to go from the standardized betas, to un-standardized ones using the high frequency variance estimates?

I read here (http://stats.stackexchange.com/questions/29781/when-should-you-center-your-data-when-should-you-standardize) about WHEN to standardize, but not clear to me if this covers the case of standardizing both dependent and extraneous variables.

Thank you!

share|improve this question
    
Sounds a bit like the Phd thesis of Allan Birnbaum?, where an initial sample was used to estimate the variance, and then that variance estimate was used to test a difference in means in a subsequent sample (using just means from that sample). The purpose of this was to be able to choose a sample size to assure set power (which was now possible given the earlier variance estimate). –  phaneron Nov 21 '12 at 13:08
    
No, it's a real world problem. –  Yugmorf Nov 21 '12 at 14:10
    
Sorry, did not mean to suggest yours was not a real world problem but that Phd thesis may outline the solution or suggest a solution to someone who wants to answer your question. –  phaneron Nov 21 '12 at 15:35
    
No problem. Thank you for correcting my spelling! –  Yugmorf Nov 22 '12 at 3:56
add comment

2 Answers

In linear regression when both covariates and outcome are sampled together, the parameter being estimated can be thought of as $$ \mathbf{\beta}=\mathrm{Var}[\,\mathbf{X}\,]^{-1}\mathrm{Cov}[\,Y,\mathbf{X}\,], $$ which you can further break out into terms in the $\mathrm{Corr}[\,Y,X\,]$, $SD[\,Y\,]$ and $SD[\,X\,]$ if you like. In your setting it sounds like you have extra information on some of those terms, but not all.

Taking a Bayesian approach, incorporating extra information about e.g. the $SD[\,Y\,]$ term into your posterior about $\beta$ would be straightforward, assuming you can quantify what the extra data tells you about that term (or terms) via a likelihood. Any component parts of $\beta$ for which you don't have extra information would just be updated in the usual way.

If you don't want to use Bayes, you could construct an estimate of $\beta$ that made the best use of the available data for each component, and then multiply them all together to get some $\hat\beta$. Getting an appropriate standard error estimate for this $\hat\beta$ would be some work, but I imagine e.g. the bootstrap would work reasonably well.

I don't see the need to standardize here... but I may be missing something.

share|improve this answer
add comment

As pointed out by Gung and MansT (http://stats.stackexchange.com/questions/29781/when-should-you-center-your-data-when-should-you-standardize), scaling the variables by their standard deviation corresponds to inverse scaling of the estimated coefficients. Thus if i estimate the regression coefficients using scaled data, i can then use each variable's standard deviation (taken or measured in which ever way i please - not necessarily from the same data set used in estimating the standardized coefficients) to back out the underlying (pre-scaled) coefficients.

The following relationships were useful in helping me understand this process:

The correlation coefficient (r) is a scaled beta.

  r = Cov(x,y)/(sd(x)*sd(y))

b = Cov(x,y)/sd(x)^2

b = r * sd(y) / sd(x)

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.