2
$\begingroup$ 
lm(y~x1 + x2 -1)

where x1 is a continuous numerical variable and x2 is a categorical factor variable with 4 levels.

Is there a way to measure the "correlation" between the x1 variable and each level of the factor variable x2? By putting correlation into double quotes, I admit that I don't really know what is a good definition for the associatedness between a continuous variable and a specific level of the factor variable. Hopefully readers get my intuition. I mean that some levels of x2 may associated with x1 more actively than other levels of x2.

Not knowing how to measure it, I am thinking of the following procedure:

  1. run lm(y~x2 +1)

  2. run lm(y~x2 + x1 -1)

i.e. replace the intercept in "Step 1" by the continous variable x1 in "Step 2" and then see which $\beta$ (of associated factor level) changed most.

My questions are:

  1. Does my approach make sense?
  2. How do I measure if a beta (of a specific associated factor level) changed and by how much? Is there a way to make fair comparison and draw some meaningful conclusions?

Could anybody please shed some lights on me?

Thanks a lot!

Improve this question
$\endgroup$
6
  • $\begingroup$ For the purposes of this question, are you only interested in the association between x1 and x2? $\endgroup$
    – Macro
    Commented Jun 25, 2012 at 23:50
  • 1
    $\begingroup$ Luna, I am struggling to understand what role $y$ plays in this question about association between $x_1$ and $x_2$. Could you enlighten us about this? $\endgroup$
    – whuber ♦
    Commented Jun 26, 2012 at 0:06
  • $\begingroup$ @whuber, it could be sensible to only be interested in the association between $x_1, x_2$ insofar as it affects the regression coefficient estimates when they are both entered into the same regression model. This seems like an ad hoc method of diagnosing confounding. Although, if $y$ doesn't play any role at all, then of course the problem becomes much simpler. $\endgroup$
    – Macro
    Commented Jun 26, 2012 at 0:10
  • $\begingroup$ Thanks, @Macro. I'm still confused, because I understand "association" quite generally to refer to a relationship between two things whereas "affects the regression coefficients" appears to refer to something else altogether. To circumvent guessing and possible misinterpretation of what is being asked, I am hoping that Luna can make this all clear with a suitable edit. $\endgroup$
    – whuber ♦
    Commented Jun 26, 2012 at 0:13
  • $\begingroup$ Hi Macro and whuber, thanks a lot for your help. The more complete picture is: initially we did the regression lm(y~x1 + x2 -1)... since our end goal was to fit the data y~(x1, x2) and it seems that whenver factor is involved, we should do y~x1+x2-1, so did we. But then we wanted the further study the relationship between the two variables x1 and x2. And what impact do they have on the regression, partially and jointly... and do they have a lot of collinearity between them and how does that impact the regression, etc. These are typical data exploring steps... $\endgroup$
    – Luna
    Commented Jun 26, 2012 at 2:26

2 Answers 2

Reset to default
1
$\begingroup$

The two variables' degree of association (put another way, the extent to which x1 means differ by level of x2) can be tested with an anova, among other ways. And ordering up an eta-squared statistic as part of an anova procedure will tell you the percent of variance in x1 that can be explained by level of x2.

Also, in places it seems that you are seeking to learn about this relationship between x1 and x2, but in other places you say things such as

[...] some levels of x2 may associated with x1 more actively than other levels of x2.

This seems to indicate a misunderstanding. If x2 is fixed at a single level, it cannot be meaningful to talk about its association or correlation with anything. It is only when x2 is free to vary that it can covary with something else.

Improve this answer
$\endgroup$
-1
$\begingroup$

If the covariate x1 is relevant to explaining variation in y, it should be included in your model. Adding it in and then taking it out does not do you much good...if should definitely always be in your model. Theory will help guide you in this decision.

Once you've decided to include or exclude the regressor x1, then you can move on to 'diagnostics' such as exploring multicollinearity. I highly doubt that multicollinearity will be a problem, since your categorical variable can only take on four values. For multicollinearity diagnostics, google "variance inflation factor" which for you, using R's 'HH' package, would be vif(lm(y ~ x1 + x2)).

1) Does my approach make sense?

No. The first regression lm(y~x2) merely will report the mean of y in each category of x2. The second regression will report the intercepts of each category, and an additional slope parameter on x1 for the entire sample. Do you understand how these are different?

So, all the parameters will likely change, because they represent different things.

2) How do I measure if a beta (of a specific associated factor level) changed and by how much? Is there a way to make fair comparison and draw some meaningful conclusions? Could anybody please shed some lights on me?

See my answer to your first question. You do not want to approach the problem in this way.

Also, it would help clarify things if you (1) stated what your variables represent (e.g., apples, temperature, ??) and (2) what research question you are attempting to answer.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ (-1) Model is selection involves too many considerations for your "definitely always" statement in paragraph 1 to hold true. Also, it doesn't seem right to conclude that just because x2 only takes 4 values it cannot be closely associated with x1. For all we know, the mean of x1 might depend in large part on the level of x2, which could be group membership, region, industry, type of apple, etc. $\endgroup$
    – rolando2
    Commented Sep 24, 2012 at 13:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.