# Correlated variables in a math model

Let's say you have 8 variables in a regression model. If some of them are correlated, what degree or percent of correlation should you considering removing some of the variables from the equation? How about covariance - what effect does that have?

Lastly, how many variables are too many to inform the model? 5, 15, 50, 500? The model I have is a profit/financial type model and is fairly straightforward - profit is the output.

• These are very difficult questions to answer without more context and knowing exactly what you want to do with your model. Correlation can cause problems by making some of the computational methods difficult (e.g. taking inverse of matrix) for doing regression. So again, whether or not you should remove variables depends on what you are trying to do with your model Oct 31, 2013 at 23:00
• Hi Sir, thanks for your response. Its nothing too fancy. Let me explain a bit more about my model. The model is simply a list of paramters or variables related to a NYSE stock market price or other financial piece of information. My goal is to predict stock prices and profits by varying the variables within the reg. equation. My stat skills are not great, so please enlighten me. Oct 31, 2013 at 23:18
• Except in respect of the emphasis on correlation/multicollinearity, this question is at the least a near-duplicate of stats.stackexchange.com/questions/74140 ... whether you want to remove variables at all depends on whether your goal is statistical inference or predictive power. Nov 1, 2013 at 0:05
• Hi, sir, can you explain, what is the difference between statistical inference and predictive power? Nov 2, 2013 at 0:40

This is called Multicollinearity in regression. Dropping is just one way to handle this situation and it is not the only remedy. Sometimes you standardize your independent variables, or get more data (if possible), dropping a term is another option, or even fitting a regression with the knowledge of having Multicollinearity! To assess how much they are correlated, you need to find a factor called Variance inflation factor (VIF), where $VIF(\hat{\beta_j})=\dfrac{1}{1-R^2_i}$ and $R^2_i$ is the coefficient of determination of the regression model, i.e. you replace your $Y$ (dependent variable) with $X_i$ and fit your regression model with all other independent variables. Then you find your $R^2$ as usual. The result is denoted by $R^2_i$ to emphasize the fact that you regressed $X_i$ against other independent variables. As a rule of thumb when $VIF(\hat{\beta_j})>5$ (or in some references $VIF(\hat{\beta_j})>10$), we say that multicollinearity is high.