2
$\begingroup$

I am a very beginner of statistic. Recently a project require me to analyse data using logistic regression & SPSS within a specific time frame. Although I have read few books, but still very blur on how to start off. Can someone guide me through? What is the 1st ste and what next?

Anyway, I have started some. Once entered the data into SPSS, I have done crosstab (categorical IV), descriptive (continuous IV) and spearman correlation.

Then, I proceed to test for nonlinearity by transforming into Ln which give me some problems. I have re-coded all zero cells to a small value (0.0001) to enable the Ln transformation. Then, I re-test the nonlinearity.

Question:

1) The only solution for violation is to transform the variable from continuous to categorical? I got one violation.

2) One Exp(B) is extremely large (15203.835). What does this means? Why?

3) There is one interaction has Exp(B) = 0.00. Why?

Many thanks.

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ why exactly are you testing for non-linearity? And what are you trying to model? What is IV? $\endgroup$
    – mpiktas
    Commented Jan 15, 2011 at 18:28
  • $\begingroup$ IV stands for independent variables. I thought the first step is testing assumption before we start the regression analysis? $\endgroup$
    – lcl23
    Commented Jan 16, 2011 at 1:47
  • $\begingroup$ could you be more precise, your questions are for the results of non-linearity test, or for logistical regression? Also it would help if we knew more about your model, what is dependent variable, etc. $\endgroup$
    – mpiktas
    Commented Jan 16, 2011 at 7:32
  • $\begingroup$ it is the result of nonlinearity test using Ln transformation. My research question is to know the existence(yes/no) of management committee in public company. Predictors involved comprised of categorical & continuous variable, such as board size, auditor, etc.... $\endgroup$
    – lcl23
    Commented Jan 16, 2011 at 10:28
  • $\begingroup$ I am perplexed about your recode of zero cells. Have you tried a sensitivity analysis? What happen if you use for example 0.01 or 0.00000001? Isn't to possible to make a conditional analysis after discarding observations with zeroes? $\endgroup$
    – glassy
    Commented Jan 16, 2011 at 13:37

2 Answers 2

Reset to default
3
$\begingroup$

The large value of "B" would be a coefficient of a variable usually called "X" in your model. Usually, "X" has a real world meaning (could be income, could be a measured volume of something, etc.). So the job is to interpret this "B" in terms of "X". The usual definition (in ordinary least squares) is that a ONE UNIT increase in "X" corresponds to a "B" increase in "Y" (where "Y" is the dependent variable which you are modeling). It is similar (but not exact) when interpreting for a logistic regression: a ONE UNIT increase in "X" corresponds to a B increase in the log-odds (which is "Y" in this case). Therefore EXP(B) tells you the proportional increase in the odds for a ONE UNIT increase in "X". So the question which may help is "What does a ONE UNIT increase in X mean in the real world?" This may make the apparent extreme value seem more sensible. Another thing to ask is what is the range of X values in your data? for if this is much smaller than 1 then EXP(B) is effectively extrapolating well beyond what you have observed in the "X space" (something which is generally not recommended because the relationship may be different).

Put more succinctly the numerical value of your betas is related to the scale at which you measure your X variables (or independent variables).

The easiest mathematical way to see this is the form of the simple ordinary least squares estimate which is

B=(standard deviation of Y) / (standard deviation of X) * (correlation between Y and X)

This result is not exact in logistic regression, but it does approximately happen.

I would also recommend that replacing the observed proportions with (r_i+1)/(n_i+2) is not a bad way to go before plugging them into the logistic function, because it can help guard against creating extreme logit values (which are only an artifact of your choice of "small number"). Extreme logit values can create outliers and influential points in the regression, and this can make your regression coefficients highly sensitive to these observations, or to be more precise, to your particular choice of "small number".

This has been called "Laplace's rule of succession" (add 1 success and 1 failure to what was observed) and it effectively "pulls" the proportion back towards 1/2, with less "pulling" the greater n_i is. In Bayesian language it corresponds to the prior information that it is possible for both of the binary outcomes to occur.

Improve this answer
$\endgroup$
1
  • $\begingroup$ "Another thing to ask is what is the range of X values in your data?" This variable ranges from 2 to 6. $\endgroup$
    – lcl23
    Commented Jan 16, 2011 at 13:40
2
$\begingroup$

Generally large beta coefficients signal multi-collinearity. You should look for marginals that are zero in your cross-tabulations. You should also pay attention to mpiktas's comment. Testing for linearity (and transforming to categorical) is not generally needed if you have been setting up your data correctly.

Improve this answer
$\endgroup$
6
  • $\begingroup$ crosstab result is ok, with no zero values. $\endgroup$
    – lcl23
    Commented Jan 16, 2011 at 1:52
  • $\begingroup$ crosstab result is ok, with no zero values. The problematic continuous variable gives me this result: B=9.62, SE=7.0, Wald=1.89, Sig=0.16, Exp(B)=15203.83. So, it is not sig., thus no issue of multicollinearity right? But the Exp(B) is so large which I think something wrong here. Another interaction (X*LnX) is sig. at 0.05, thus multicollinearity exists. From what I read from the book, the only solution is to transform this continuous variable to categorical? $\endgroup$
    – lcl23
    Commented Jan 16, 2011 at 2:00
  • $\begingroup$ @lcl23, what is $B$? If it is a coefficient from logistic regression, why are you interested in $\exp(B)$? $\endgroup$
    – mpiktas
    Commented Jan 16, 2011 at 7:33
  • $\begingroup$ yes, B is the coefficient. Exp(B) is the odd ratio. Does this extremely large odd ratio indicate something wrong? or I can just ignore it? $\endgroup$
    – lcl23
    Commented Jan 16, 2011 at 10:55
  • $\begingroup$ I suppose it's possible that it means something but more likely it means something went wrong. Multi-colinearity will not necessarily show up in the cross-tabulations. You need to run some diagnostics. There should be documentation in your software about how to get a condition number for the XX' matrix or perhaps it will tell you how to get a report on variance inflation factors. $\endgroup$
    – DWin
    Commented Jan 16, 2011 at 14:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.