8
$\begingroup$

I received this elementary question by email:

In a regression equation am I correct in thinking that if the beta value is positive the dependent variable has increased in response to greater use of the independent variable, and if negative the dependent variable has decreased in response to an increase in the independent variable - similar to the way you read correlations?

Improve this question
$\endgroup$
7
  • $\begingroup$ @Jeromy, by beta weights you mean the linear regression coefficients? $\endgroup$
    – mpiktas
    Commented May 17, 2011 at 5:29
  • 1
    $\begingroup$ @mp Conventionally, betas are the coefficients when all variables have been standardized. (That should instantly make them recognizable as partial correlations, answering the question... :-) $\endgroup$
    – whuber ♦
    Commented May 17, 2011 at 5:30
  • 3
    $\begingroup$ @ayush I realise it is an elementary question, so feel free to not answer it yourself. However, I think the site can benefit from having questions at various levels of difficulty; and I'd like to add my own answer after giving others a chance to answer that picks up on a few general issues. $\endgroup$
    – Jeromy Anglim
    Commented May 17, 2011 at 5:41
  • $\begingroup$ Good point, @Jeromy. I'm sure @ayush would not have provided such a comment (which could easily be misinterpreted as rude or worse) were the same question to be posed by a new user. Let's take this as testimony to your high reputation here and see whether any of the replies help enlighten your correspondent. $\endgroup$
    – whuber ♦
    Commented May 17, 2011 at 5:48
  • 4
    $\begingroup$ @whuber. good point. Being a stats consultant in psychology, I sometimes get questions by email that are fairly elementary. My ideal situation is to encourage such students to post directly here. In general, I prefer to answer these questions on this site rather than send an email response to the student. That way, my response can be an ongoing resource for the internet, and others may come up with an even better response. $\endgroup$
    – Jeromy Anglim
    Commented May 17, 2011 at 6:05

2 Answers 2

Reset to default
8
$\begingroup$

In explaining the meaning of regression coefficient I found that the following explanation very useful. Suppose we have the regression

$$Y=a+bX$$

Say $X$ changes by $\Delta X$ and $Y$ changes by $\Delta Y$. Since we have the linear relationship we have

$$Y+\Delta Y= a+ b(X+\Delta X)$$

Since $Y=a+bX$ we get that

$$\Delta Y = b \Delta X.$$

Having this is easy to see that if $b$ positive, then positive change in $X$ will result in positive change in $Y$. If $b$ is negative then positive change in $X$ will result in negative change in $Y$.

Note: I treated this question as a pedagogical one, i.e. provide simple explanation.

Note 2: As pointed out by @whuber this explanation has an important assumption that the relationship holds for all possible values of $X$ and $Y$. In reality this is a very restricting assumption, on the other hand the the explanation is valid for small values of $\Delta X$, since Taylor theorem says that relationships which can be expressed as differentiable functions (and this is a reasonable assumption to make) are linear locally.

Improve this answer
$\endgroup$
5
  • $\begingroup$ ...assuming the behavior is truly linear across the entire range of $X$ values! (A more cautious answer might couch the same idea in terms of average changes and also avoid any hint of suggesting the relationship is causal.) $\endgroup$
    – whuber ♦
    Commented May 17, 2011 at 5:44
  • $\begingroup$ @whuber, I knew that putting the word best was not a wise choice:) Thanks for your comment, I'll try to rephrase the answer. $\endgroup$
    – mpiktas
    Commented May 17, 2011 at 5:49
  • $\begingroup$ @mp "Best" isn't necessarily a problem. I'm just trying to give you a hard time :-) (But "induce" did get my attention...) If you're truly after the "best" explanation, recall that a common point of confusion among the uninitiated is how to interpret interaction coefficients: after all, you can't independently vary (say) $X Y$; you do so by varying either $X$ or $Y$ or both. So an explanation that handles that situation would be most welcome. $\endgroup$
    – whuber ♦
    Commented May 17, 2011 at 5:52
  • $\begingroup$ @whuber, yes induce was a poor choice. I'll leave the explanation of interaction terms for somebody else :) $\endgroup$
    – mpiktas
    Commented May 17, 2011 at 6:06
  • 1
    $\begingroup$ @mp re Note 2: Ah, Taylor's Theorem! But real data aren't even continuous, much less differentiable. The model might enjoy these mathematical properties. Especially in explanations for the uninitiated, it may be worthwhile to distinguish the model's behavior from the behavior we expect of the data. Also, Taylor's Theorem says little about the range of $X$ values over which near-linearity holds. The regression model says this range is infinite! $\endgroup$
    – whuber ♦
    Commented May 17, 2011 at 14:13
8
$\begingroup$

As @gung notes, there are varying conventions regarding the meaning of ($\beta$, i.e., "beta"). In the broader statistical literature, beta is often used to represent unstandardised coefficients. However, in psychology (and perhaps other areas), there is often a distinction between b for unstandardised and beta for standardised coefficients. This answer assumes that the context indicates that beta is representing standardised coefficients:

  • Beta weights: As @whuber mentioned, "beta weights" are by convention standardised regression coefficients (see wikipedia on standardised coefficient). In this context, $b$ is often used for unstandardised coefficients and $\beta$ is often used for standardised coefficients.

  • Basic interpretation: A beta weight for a given predictor variable is the predicted difference in the outcome variable in standard units for a one standard deviation increase on the given predictor variable holding all other predictors constant.

  • General resource on multiple regression: The question is elementary and implies that you should read some general material on multiple regression (here is an elementary description by Andy Field).

  • Causality: Be careful of language like "the dependent variable has increased in response to greater use of the independent variable". Such language has causal connotations. Beta weights by themselves are not enough to justify a causal interpretation. You would require additional evidence to justify a causal interpretation.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ +1 Note, however, that there are differing conventions w/ respect to the use of terms in statistics. For example, 'beta' / $\beta$ is often used to denote the true parameter that governs the data generating process, & 'beta hat' / $\hat\beta$ refers to the slope estimate calculated in your sample. In this case, they do not imply that the variables have been standardized 1st. This varying usage is unfortunate, but nonetheless real. It is important to be clear about how terms are being used when one encounters them, rather than assuming that everyone means the same thing. $\endgroup$
    – gung - Reinstate Monica
    Commented Nov 4, 2012 at 14:42
  • $\begingroup$ @gung good point; I've updated my answer to incorporate this. $\endgroup$
    – Jeromy Anglim
    Commented Jul 7, 2013 at 4:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.