# What to do with almost-continuous variable in regression?

I've been taught that binning a continuous variable into categories is almost never a good idea, because you lose information in the process. But now I'm facing a situation where I have an age variable that is "mostly continuous", which is to say that about 90% of the values represent age in years, and the remaining 10% are recorded as an ordered factor with haphazard levels, e.g., >18, 18-34, >50 ...

I want to use age as a predictor of a continuous outcome variable, but I'm not sure how to proceed. Should I make this into an ordered factor, even though this will mean throwing away information in 90% of the cases? If not, what do I do with the 10% of the cases where age is already an ordered factor? Any suggestions will be appreciated.

-
Can the reason that the 10% are binned be ascribed to a random process or is there some underlying factor influencing the binning? –  schenectady May 7 '11 at 15:03
Do the age bins have any chance of overlapping each other? –  whuber May 7 '11 at 15:30
@schenectady I don't think the binning can be ascribed to a random process. My understanding is that survey participants who refused to give their age were asked to give an age range. @whuber Yes, there is overlap. For example, the >18 category is not exclusive of the 34 -- 60 category. –  Ista May 7 '11 at 19:45

I'd say interact continuous age with a dummy "continuous age is available", and categorical age with a dummy "continuous age is not available". That way you'll be using as much of the information you have as possible. Of course if the effect of age is something you'd like to be able to summarize with just one point estimate, you'll have to think a bit more (though the coefficient on continuous age should be a pretty good approximation for that).

-
That's clever. But it raises interesting questions. How would you code the categorical ages? What do you do if some of those categorical age intervals overlap each other? Does it even make sense to develop a model in which intervals of age are separate categories yet actual ages are supposed to be linearly related to the outcome? –  whuber May 7 '11 at 15:30
@crayola Nice, I'll give it a go. –  Ista May 7 '11 at 19:47
Using the "continuous age is available" as a predictor will result in major problems of interpretation and inference. The problem as originally specified is an ideal candidate for multiple imputation. For those subjects for whom continuous age is known you also know the bins. That allows you to develop a multiple imputation model for continuous age given the bin. Then you can form a model with only continuous age (90% real, 10% imputed). –  Frank Harrell May 8 '11 at 14:26
I agree that your solution would ultimately result in a more interpretable output. However, if - the goal is just to control for age in a regression where the interest is on another variable (e.g. find the effect of some kind of treatment controlling for age), and - the individuals for whom continuous age is missing are an as-if-random sample of the population (conditional on the other covariates), I fail to see where the "major problems of interpretation and inference" come from? –  crayola May 8 '11 at 15:26
Creating indicator variables as outlined creates major difficulties even if one is only wanting to control for age. See @Article{kno10unp, author = {Knol, Mirjam J. and Janssen, Kristel J. M. and Donders, Rogier T. and Egberts, Antoine C. G. and Heerding, E. Rob and Grobbee, Diederick E. and Moons, Karel G. M. and Geerlings, Mirjam I.}, title = {Unpredictable bias when using the missing indicator method or complete case analysis for missing confounder values: an empirical example}, journal = J Clinical Epidemiology, year = 2010, volume = 63, pages = {728-736} } –  Frank Harrell May 8 '11 at 17:55
show 1 more comment

You can incorporate this in the Bayesian framework by specifying a prior distribution for the age variable. And for the posterior, you have:

$$p(\theta|DI)\propto p(\theta|I)p(D|\theta I)$$

Now you simply take $D\equiv (18+)$ for example. This is no more difficult "in-principle" compared to when you actually do know the ages. The difference is that your likelihood function must be a cumulative distribution function instead of a density. As an example, suppose age is the only regressor you have (denoted $x_i$), and you are fitting a OLS model. This is for my benefit - but the generalisation is just details, rather than conceptual. If you have observed the ages exactly the likelihood function is:

$$p(y_1\dots y_N|x_1\dots x_N\alpha\beta\sigma I)=(2\pi\sigma^2)^{-\frac{N}{2}}\exp\left(-\frac{1}{2\sigma^2}\sum_{i=1}^{N}(y_i-\alpha-\beta x_i)^2\right)$$

But now suppose that the $(N+1)$th observation, you only observe that $L<x_{N+1}<U$. Lets call this piece of information $Z$. Now we can use the brilliant trick of introducing a nuisance parameter and then integrating it out again (via the sum rule). The nuisance parameter we introduce is $x_{N+1}$ (the actual unobserved age), and we have:

$$p(y_1\dots y_N y_{N+1}|x_1\dots x_N Z\alpha\beta\sigma I)=\int_{L}^{U} p(y_1\dots y_N y_{N+1}x_{N+1}|x_1\dots x_N Z\alpha\beta\sigma I)dx_{N+1}$$

Now we can split the integrand by using the product rule $P(AB|C)=P(A|C)P(B|AC)$ and we get:

$$p(x_{N+1}|x_1\dots x_N Z\alpha\beta\sigma I)p(y_1\dots y_N y_{N+1}|x_1\dots x_N x_{N+1}Z\alpha\beta\sigma I)$$

Note that in the second density, the information $Z\equiv L<x_{N+1}<U$ is redundant because we are already conditioning on the true value $x_{N+1}$. So we can remove it. Note that this second term could be called the "clean" data. The first term is basically a statement of how likely the unobserved age is given $L<x_{N+1}<U$, in addition to the position of the "true line" $(\alpha,\beta)$, the noise level $\sigma$, and the values of all other ages $(x_1\dots x_N)$. And so you have an integrated likelihood (sometimes called quasi-likelihood):

$$p(Y|XZ\alpha\beta\sigma I)=(2\pi\sigma^2)^{-\frac{N+1}{2}}\exp\left(-\frac{1}{2\sigma^2}\sum_{i=1}^{N}(y_i-\alpha-\beta x_i)^2\right)$$ $$\times\int_{L}^{U}p(x_{N+1}|X\alpha\beta\sigma I)\exp\left(-\frac{(y_{N+1}-\alpha-\beta x_{N+1})^2}{2\sigma^2}\right)dx_{N+1}$$

Now for every "messy" data, you will have a similar integral. You can take the above integral as multi-dimensional (with appropriate matrix sum of squares in the exponential).

I have heard something like this called the "Missing information Principle". You basically create a "nice" dataset from your "messy" one (i.e. the data set you wish you had), and then average out the "nice" inferences. You give more weight to certain nice data sets according to what your "messy" information is.

-
If you're using software like WinBUGS, you can incorporate this quite easily. For example, you could specify a uniform prior on the range: age ~ dunif(age.lower,age.upper) (where the ages are known, just set age.lower = age.upper). –  Simon Byrne Jul 4 '11 at 13:19