1
$\begingroup$

I'm working on a project where I'm analyzing how improvements in players' skills are associated with changes in their values. Specifically to see if there is a correlation between point changes in certain skills and percent changes in their value.

I used .corr() and p-values (<.01) for those calculated correlation coefficients to find a set of skills that have a correlation coefficient > .5 (moderate to high correlations). So this would be correlations for each individual skill to percent change in value.

I then decided to explore the data set with LinearRegression() from scikit-learn and found regression coefficients that are totally different for those same skill variables correlation coefficients I have found (in that they're negative and much smaller, ie correlation coefficient for attacking: 0.51, regression coefficient for attacking: -0.079).

I'm new to this, but does that seem plausible? Or did I possibly make a mistake in calculations? It doesn't make sense for a positive correlation to have a negative regression coefficient.

Improve this question
$\endgroup$
5
  • $\begingroup$ Did you calculate correlations separately between each pair of variables? And did you do the same for the regression, or did you then enter all independent variables at once? $\endgroup$
    – Ruben van Bergen
    Sep 23, 2020 at 21:10
  • $\begingroup$ For a while (and possibly still), sklearn automatically used regularization. Check out if that is still the case. If it is, perhaps try statsmodels. $\endgroup$
    – Dave
    Sep 23, 2020 at 21:13
  • $\begingroup$ Have you seen stats.stackexchange.com/questions/1580 and related threads on changes in regression coefficients? Moreover, there's never any reason to suppose multiple regression coefficients are even remotely similar to correlation coefficients unless (at the very least) all the variables have been standardized, which means shifting and rescaling them to have common variances. $\endgroup$
    – whuber ♦
    Sep 23, 2020 at 21:25
  • $\begingroup$ There is no reason to assume even the coefficients of simple regression will be the same as multiple regression due to issues like moderation. The same is true of correlation, a bi variate relationship, versus multiple regression. $\endgroup$
    – user54285
    Sep 23, 2020 at 22:02
  • $\begingroup$ (+1): This is a topic worthy of review, see researchgate.net/post/… $\endgroup$
    – AJKOER
    Sep 23, 2020 at 22:36

1 Answer 1

Reset to default
1
$\begingroup$

I would advise you to standardized your variables by subtracting the mean and dividing the result by the standard deviation.

Now, run the regression and look at the coefficient of each new standardized variable, where the new standardized regression coefficient is also known as the beta-coefficient. Per a source:

For simple linear regression with orthogonal predictors, the standardized regression coefficient equals the correlation between the independent and dependent variables.

So, if you explanatory variables are independent (orthogonal), your beta coefficients are completely consistent with computed Pearson correlation in the new variables.

My recommendation, given the question: "Does it make sense for correlation coefficients to be vastly different from regression coefficients?", your predictor variables are likely not independent. As such, consider constructing orthogonal variables using PCA (see, for example, this reference). The cited source reports, to quote:

The study shows that regression analysis and principal component analysis (PCA) use few explanatory variables to explain variations in a dependent variable and are therefore efficient tools for assessing turmeric yield depending on the set objective.

In the current context then, the regression standardized coefficients would agree with Pearson's correlations in the new variable constructs. However, this may come at a price of being able to readily interpret the meaning of the new variable constructs.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.