Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 6 down vote favorite
1

I received this elementary question by email:

In a regression equation am I correct in thinking that if the beta value is positive the dependent variable has increased in response to greater use of the independent variable, and if negative the dependent variable has decreased in response to an increase in the independent variable - similar to the way you read correlations?

share|improve this question
do we have to answer this ?? – ayush biyani May 17 '11 at 5:28
@Jeromy, by beta weights you mean the linear regression coefficients? – mpiktas May 17 '11 at 5:29
1  
@mp Conventionally, betas are the coefficients when all variables have been standardized. (That should instantly make them recognizable as partial correlations, answering the question... :-) – whuber♦ May 17 '11 at 5:30
3  
@ayush I realise it is an elementary question, so feel free to not answer it yourself. However, I think the site can benefit from having questions at various levels of difficulty; and I'd like to add my own answer after giving others a chance to answer that picks up on a few general issues. – Jeromy Anglim May 17 '11 at 5:41
4  
@whuber. good point. Being a stats consultant in psychology, I sometimes get questions by email that are fairly elementary. My ideal situation is to encourage such students to post directly here. In general, I prefer to answer these questions on this site rather than send an email response to the student. That way, my response can be an ongoing resource for the internet, and others may come up with an even better response. – Jeromy Anglim May 17 '11 at 6:05
show 1 more comment

2 Answers

up vote 6 down vote

In explaining the meaning of regression coefficient I found that the following explanation very useful. Suppose we have the regression

$$Y=a+bX$$

Say $X$ changes by $\Delta X$ and $Y$ changes by $\Delta Y$. Since we have the linear relationship we have

$$Y+\Delta Y= a+ b(X+\Delta X)$$

Since $Y=a+bX$ we get that

$$\Delta Y = b \Delta X.$$

Having this is easy to see that if $b$ positive, then positive change in $X$ will result in positive change in $Y$. If $b$ is negative then positive change in $X$ will result in negative change in $Y$.

Note: I treated this question as a pedagogical one, i.e. provide simple explanation.

Note 2: As pointed out by @whuber this explanation has an important assumption that the relationship holds for all possible values of $X$ and $Y$. In reality this is a very restricting assumption, on the other hand the the explanation is valid for small values of $\Delta X$, since Taylor theorem says that relationships which can be expressed as differentiable functions (and this is a reasonable assumption to make) are linear locally.

share|improve this answer
...assuming the behavior is truly linear across the entire range of $X$ values! (A more cautious answer might couch the same idea in terms of average changes and also avoid any hint of suggesting the relationship is causal.) – whuber♦ May 17 '11 at 5:44
@whuber, I knew that putting the word best was not a wise choice:) Thanks for your comment, I'll try to rephrase the answer. – mpiktas May 17 '11 at 5:49
@mp "Best" isn't necessarily a problem. I'm just trying to give you a hard time :-) (But "induce" did get my attention...) If you're truly after the "best" explanation, recall that a common point of confusion among the uninitiated is how to interpret interaction coefficients: after all, you can't independently vary (say) $X Y$; you do so by varying either $X$ or $Y$ or both. So an explanation that handles that situation would be most welcome. – whuber♦ May 17 '11 at 5:52
@whuber, yes induce was a poor choice. I'll leave the explanation of interaction terms for somebody else :) – mpiktas May 17 '11 at 6:06
1  
@mp re Note 2: Ah, Taylor's Theorem! But real data aren't even continuous, much less differentiable. The model might enjoy these mathematical properties. Especially in explanations for the uninitiated, it may be worthwhile to distinguish the model's behavior from the behavior we expect of the data. Also, Taylor's Theorem says little about the range of $X$ values over which near-linearity holds. The regression model says this range is infinite! – whuber♦ May 17 '11 at 14:13
up vote 5 down vote

  • Beta weights: As @whuber mentioned, "beta weights" are by convention standardised regression coefficients (see wikipedia on standardised coefficient). In this context, $b$ is often used for unstandardised coefficients and $\beta$ is often used for standardised coefficients.

  • Basic interpretation: A beta weight for a given predictor variable is the predicted difference in the outcome variable in standard units for a one standard deviation increase on the given predictor variable holding all other predictors constant.

  • General resource on multiple regression: The question is elementary and implies that you should read some general material on multiple regression (here is an elementary description by Andy Field).

  • Causality: Be careful of language like "the dependent variable has increased in response to greater use of the independent variable". Such language has causal connotations. Beta weights by themselves are not enough to justify a causal interpretation. You would require additional evidence to justify a causal interpretation.

share|improve this answer
+1 Note, however, that there are differing conventions w/ respect to the use of terms in statistics. For example, 'beta' / $\beta$ is often used to denote the true parameter that governs the data generating process, & 'beta hat' / $\hat\beta$ refers to the slope estimate calculated in your sample. In this case, they do not imply that the variables have been standardized 1st. This varying usage is unfortunate, but nonetheless real. It is important to be clear about how terms are being used when one encounters them, rather than assuming that everyone means the same thing. – gung Nov 4 '12 at 14:42

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.