14
$\begingroup$

I know that zero-inflated models (e.g. zero-inflated Poisson or negative binomial models) can be used for dependent variables. I also know that in general there are no assumptions for the independent variables (i.e. predictors) in regression analyses. However, I have a quantitative (continuous or count) predictor which has many (say, 40-60%) zeros. When I used it as a quantitative predictor in regression (linear or logistic) I got a small P value (i.e. P<0.01), but when I used it as a binary predictor (zero or not) I got a P value>0.05. Why did it happen? How do I interpret this result?

$\endgroup$
12
  • $\begingroup$ I suggest you plot the two sets of fitted vs actual on the same plot (with y=x line marked lightly). You should see why quite quickly. If it doesn't show on that, try actual-fitted vs fitted on the same plot (with y=0 line marked in) $\endgroup$ – Glen_b Jul 1 '13 at 4:54
  • 7
    $\begingroup$ There is no assumption for the distribution of the independent variables, but if you simply add that variable to your model you do assume linearity of its effect. If you have that many 0s, then it is reasonable to suspect that there is something special about that value, and you'll probably want to incorporate that in your model. Here is a discussion on how to do that: stats.stackexchange.com/questions/56306 $\endgroup$ – Maarten Buis Jul 1 '13 at 6:59
  • $\begingroup$ @Maarten Buis: Thanks for the illuminating answer. Regarding the breastfeeding example with beta1 (for weeks breastfeeding) and beta2 (for non-breastfeeding) coefficients, suppose that the outcome is baby health: If I get a significant beta1 (e.g. P=0.001) and a non-significant beta2 (e.g. P=0.1), should I interpret the result that more breastfeeding is good for the baby but non-breastfeeding does not affect the baby? $\endgroup$ – KuJ Jul 2 '13 at 4:20
  • 1
    $\begingroup$ In my example $\beta_1$ refers to the linear effect, while $\beta_2$ refers to the discrete jump at 0. So, the null hypothesis for the second test is that there is no discrete jump, i.e. you could just as well describe the relationship with just a linear effect. With the $p$-values you report you cannot reject that hypothesis. $\endgroup$ – Maarten Buis Jul 2 '13 at 6:39
  • 1
    $\begingroup$ @ Maarten Buis: Recently I find that your suggestion is implemented in an article “On the implication of structural zeros as independent variables in regression analysis: applications to alcohol research (ncbi.nlm.nih.gov/pmc/articles/PMC5628625)”. $\endgroup$ – KuJ Feb 22 '18 at 1:56
2
$\begingroup$

So it's important to think of the source of zero-inflation. Two sources that come to mind are:

  • floor effect: your measurement instrument cannot detect values below a certain threshold and so the instrument simply returns zero. Think of a scale with 40 questions supposedly measuring "high-end math knowledge", so the test is really difficult. For each question, you could be incorrect (0 score) or correct (1 score). A test-taker could be incorrect on all 40 questions resulting in a total score of 0. Assuming the scale is indeed unidimensional/valid for high end math knowledge, this 0 score does not mean the respondent has zero math knowledge. But it suggests their math knowledge is at a level below which the test can detect. If someone administered this test to the general population, you might get a high proportion of zero scores. This scenario can occur in many different contexts. Sometimes, a measurement instrument can only detect differences beyond a threshold, and several respondents are below that threshold. Instruments for physical quantities might simply return a < X score, where X is that threshold.
  • true zero: Sometimes, people really have a zero score. For example, this can happen with counts, e.g. how many homes do you own? And sometimes, there is something fundamentally different between someone who does not own their home and someone who owns one or more homes.

So it's important to think of the source of zeroes and what you're trying to measure.

In the floor effect situation, I might claim that the math scores would be approximately normally distributed but for the floor effect. So I'd assume we have a normally distributed variable that has been censored at 0. The effect of analyzing with the predictor as is is range restriction which - all other factors held constant - can reduce power. So you could build a model that accommodates a censored predictor - Bayesian modeling should make this easy.

In the true zero situation, I might decide to dichotomize the variable to create two predictors: one binary (homeowner or not) and one continuous (how many homes). Then use both predictors in my model, allowing you to measure the effect of home ownership separate from owning more homes.

These are just two scenarios that come to mind. Also, it's never clear cut. Number of homes could also be a proxy for another variable, where number of homes is a proxy that cannot detect levels of that variable beneath a threshold. If your interest is in measuring the relation between the outcome and the true variable number of homes is a proxy for, you have another example of a censored predictor.

In all, having a predictor with many zeroes welcomes you to think about why that might be happening. And what you might want to do about it. Also, the easiest viewpoint to take is regression places no assumption on predictor distributions and just proceed with the predictor as is.

$\endgroup$
1
$\begingroup$

Zero-inflation is a probabilistic concept, and it means something different than "the distribution has a lot of zeroes". For instance, a Poisson or negative binomial model can use parameters so that 90% or 99% of the values are 0, but that doesn't "make" it zero-inflated. Zero-inflation is a mixture of a known probability model and a 0 constant "variable".

In statistics, when you consider the whole host of multimodal distributions out there, and collecting a sample like what you describe, and declaring a. that you know what distribution it's supposed to be and b. that it did not fit that distribution because it's zero-inflated... well the plausibility goes almost surely to 0.

If we broaden the scope, there's a more illustrative question to consider: when do we care about the impact of the distribution of the covariates $X$ when conducting a regression analysis? The answer is: almost never. More precisely, most default regression methods make no use whatsoever of the distribution of the covariates. The exceptions would be in cases of correcting biased sampling with weights, measurement error methods, indirect standardization, performing expectation maximization to handle truncation, or parametric bootstrap: i.e. advanced methods. However OLS, for instance, geometrically is a projection so that the residuals are orthogonal to the predictors, this does not guarantee that the error term and the regressors are independent hence the need for detailed diagnostic plots and analyses.

Lastly, your finding is not surprising. Dichotomizing a variable is still a bad idea for irregular covariates for the same reason it's a bad idea for nicely behaved covariates. Reducing the variability of $X$ reduces the precision of the analysis. In fact, a nice formula to recall is the following:

$$ \text{SE}(\hat{\beta}) = \sigma^{2} / \text{var}(X)$$

(a bivariate result that extends more or less to higher dimensional analyses).

In other words, the confidence interval for $\hat{\beta}$ and $p$-value for the statistical test of the hypothesis $\mathcal{H}_0: \beta=0$ shrink when considering designs where $X$ is more variable. Dichotomizing a variable fundamentally reduces its variability.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.