13
$\begingroup$

I have a "basic statistics" concept question. As a student I would like to know if I'm thinking about this totally wrong and why, if so:

Let's say I am hypothetically trying to look at the relationship between "anger management issues" and say divorce (yes/no) in a logistic regression and I have the option of using two different anger management scores -- both out of 100.
Score 1 comes from questionnaire rating instrument 1 and my other choice; score 2 comes from a different questionnaire. Hypothetically, we have reason to believe from previous work that anger management issues give rise to divorce.
If, in my sample of 500 people, the variance of score 1 is much higher than that of score 2, is there any reason to believe that score 1 would be a better score to use as a predictor of divorce based on its variance?

To me, this instinctively seems right, but is it so?

$\endgroup$
  • $\begingroup$ Interesting question, I believe Whuber's answer explains it perfectly well. My first response to the question was: 'increased variance does not entail higher class-discriminatory information'. $\endgroup$ – Zhubarb Aug 14 '13 at 7:48
11
$\begingroup$

A few quick points:

  • Variance can be arbitrarily increased or decreased by adopting a different scale for your variable. Multiplying a scale by a constant greater than one would increase the variance, but not change the predictive power of the variable.
  • You may be confusing variance with reliability. All else being equal (and assuming that there is at least some true score prediction), increasing the reliability with which you measure a construct should increase its predictive power. Check out this discussion of correction for attenuation.
  • Assuming that both scales were made up of twenty 5-point items, and thus had total scores that ranged from 20 to 100, then the version with the greater variance would also be more reliable (at least in terms of internal consistency).
  • Internal consistency reliability is not the only standard by which to judge a psychological test, and it is not the only factor that distinguishes the predictive power of one scale versus another for a given construct.
$\endgroup$
9
$\begingroup$

A simple example helps us identify what is essential.

Let $$Y = C + \gamma X_1 + \varepsilon$$

where $C$ and $\gamma$ are parameters, $X_1$ is the score on the first instrument (or independent variable), and $\varepsilon$ represents unbiased iid error. Let the score on the second instrument be related to the first one via

$$X_1 = \alpha X_2 + \beta.$$

For example, scores on the second instrument might range from 25 to 75 and scores on the first from 0 to 100, with $X_1 = 2 X_2 - 50$. The variance of $X_1$ is $\alpha^2$ times the variance of $X_2$. Nevertheless, we can rewrite

$$Y = C + \gamma(\alpha X_2 + \beta) = (C + \beta \gamma) + (\gamma \alpha) X_2 + \varepsilon = C' + \gamma' X_2 + \varepsilon.$$

The parameters change, and the variance of the independent variable changes, yet the predictive capability of the model remains unchanged.

In general the relationship between $X_1$ and $X_2$ may be nonlinear. Which is a better predictor of $Y$ will depend on which has a closer linear relationship to $Y$. Thus the issue is not one of scale (as reflected by the variance of the $X_i$) but has to be decided by the relationships between the instruments and what they are being used to predict. This idea is closely related to one explored in a recent question about selecting independent variables in regression.

There can be mitigating factors. For instance, if $X_1$ and $X_2$ are discrete variables and both are equally well related to $Y$, then the one with larger variance might (if it is sufficiently uniformly spread out) allow for finer distinctions among its values and thereby afford more precision. E.g., if both instruments are questionnaires on a 1-5 Likert scale, both are equally well correlated with $Y$, and the answers to $X_1$ are all 2 and 3 and the answers to $X_2$ are spread among 1 through 5, $X_2$ might be favored on this basis.

$\endgroup$
1
$\begingroup$

Always check the assumptions for the statistical test you're using!

One of the assumptions of logistic regression is independence of errors which means that cases of data should not be related. Eg. you can't measure the same people at different points in time which I fear you may have done with your anger management surveys.

I would also be worried that with 2 anger management surveys you're basically measuring the same thing and your analysis could suffer from multicollinearity.

$\endgroup$
  • 1
    $\begingroup$ I think N26 is suggesting a thought experiment. I.e., if when designing a study you have a choice between two scales, should you prefer, prima facie, the one with the greater variance. Also, having two predictors that represent the same construct, but are measured differently does not violate the assumption of independence of observations. $\endgroup$ – Jeromy Anglim Apr 22 '11 at 14:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.