6
$\begingroup$

This question is about handling zeros in an independent variable for a regression.

In particular, the zeros are not missing data or true zeros, but occur because of quantization. As a concrete example, lets say the observations are cities, and the variable is the number (or fraction) of people in some category, based on a sample. If the sample for a particular city is small, it might have zero people in a category, even if the true number in the city's population is nonzero.

In this case, what are possible ways to work with zeros if the variable is heavy tailed? Normally I would log-transform, but I can't do that when zeros are present, and because many of the observations are zero, excluding them would introduce a large bias.

Some things I'm considering: other transformations, replacing the variable with a bayesian estimate of the fraction, switching from regression to ANOVA with people as observations and city as a categorical variable. Are these valid approaches? Am I missing any? What are the pros and cons?

Improve this question
$\endgroup$
1
  • $\begingroup$ Square root should work ok $\endgroup$
    – probabilityislogic
    Commented Dec 24, 2017 at 1:32

2 Answers 2

Reset to default
2
$\begingroup$

There are no distributional assumptions made for variables you condition on such as predictors. Having zero frequency of a categorical variable cell for a city should not be a problem. If you believe there is a slope discontinuity at zero for all cities, you could model that variable (let's say it's coded as a fraction) using at least two variables: an indicator variable to denote non-zero and the actual value to allow for a post-zero linear effect. Nonlinear effects can also be added.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Are you asking whether the distribution of the independent variable is known? If so, let's assume that people are sampled at random, so it is a binomial distribution with a different mean for each city, and that I know the distribution of means over cities. $\endgroup$
    – elplatt
    Commented Dec 25, 2017 at 0:00
  • $\begingroup$ By 'conditioned on' I mean that the values are known and that we do not entertain any distributional variation in them. For example, conditioning on yearly rainfall of 30in would mean that we are evaluating properties of $Y$ when rainfall=30in. We are interested in the distribution of $Y$ given $X$, and the distribution of $X$ is not at issue. Where the distribution of $X$ does have a major effect is that for certain poorly represented values of $X$ we don't have a sample size large enough to study the distribution of $Y$. $\endgroup$
    – Frank Harrell
    Commented Dec 25, 2017 at 13:56
  • 1
    $\begingroup$ I might add that this phenomenon, which I've seen called "heaping" in surveys where respondents "round off" self-response recalls to convenient numbers, theoretically only influences the regression by reducing the efficiency slightly. $\endgroup$
    – AdamO
    Commented Dec 29, 2017 at 15:37
  • 1
    $\begingroup$ There's a nice paper on Frank's suggestion of using an additional variables if there truly is discontinuity: google.ca/url?sa=t&source=web&rct=j&url=https://… $\endgroup$
    – Thomas Speidel
    Commented Dec 31, 2017 at 5:00
0
$\begingroup$

You are on the right track. You could do a log constant transformation, where you add a constant to each observation and then log transform it. Determining the constant should be defensible, but one suggestion is given by Rob Hybdman on his blog (https://robjhyndman.com/hyndsight/transformations/) as half of the smallest non-zero value. Make sure to account for this constant when interpreting the coefficients.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.