8
$\begingroup$

Apologies from the go as this question comes from an absolute newbie and will definitely not satisfy a lot of the detail required. Hence, your guidance in providing you the right information to allow for the adequate answering of my question may be inevitable.

A brief summary of what's nagging at my mind currently. I have received a dataset from a colleague, which contains ~5000 patients and we're trying to determine how an initial treatment affects various outcome variables. Some of these are categorical (with 3 levels), some are continuous. The categorical variable is currently the least of my concerns, because as far as I can tell, I would have to perform multivariate regression, perhaps using MANOVA, to see how this dependent variable is affected by the other independent variables.

Now, what really bothers me is the distribution of the continuous data. This data consists of classic "count" data (number of times the doctor was visited following treatment) but then contains "time to heal". The "count" data would assume a Poisson distribution, but in this case var>mean ... so perhaps a quasi-Poisson would be more adequate. The distribution of that "count" data is depicted in figure 1. Figure 1

Now the time to heal is what's really bugging me. Some people would argue, that this would require linear regression modelling, but what I can't get my head around, is that time to heal, cannot be zero or take on negative numbers. And if one looks at the distribution of the data, it looks a lot more Poisson than anything else (Figure 2). So would this in fact require Poisson regression? Also biologically, time to heal is probably not completely linear, but more along what we see in cell culture growth, that it has a more exponential pattern, with a linear phase in it.

Figure 2

OR - and this brings me to the question in my title - because of the size of the dataset, which is fairly large, could one say that the central limit theorem holds true and I don't have to worry about the data-distribution? Clearly my plots tell me I do, no?!?

If I log-transform my data, things start to look quite ok (Figure 3)...but is a linear model adequate as the values cannot be negative or zero?

Figure 3

I've plotted the residuals of a linear regression model for the time to heal data in Figure 4, which shows that the tails don't really hold true to linearity. Then I plotted the residuals of the regression using a Poisson regression (Figure 5). Then the residuals of a linear regression model using log-transformation (Figure 6).

Figure 3 - Linear Model Figure 4 - Poisson model Figure 6 - Log transformed linear model

Just by looking at what you have here, what more information would you require? Am I going down a completely wrong path here? And if we assume an exponential curve for time to heal, what do I have to do to correct for this in my modelling process? And just because of the rule of numbers, could I spare myself all this thinking and simply assume that the CLT holds true?

I am very new to this, so any guidance would be more than welcome.

Hope not to have wasted anyone's time and apologize if I'm not posting crucial information here. Happy to learn, what more is required.

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ Counts are NOT continuous; they're explicitly discrete - indeed, they're the non-negative integers 0,1,2,3, .... As an aside to your main question: -- since your data are counts, one might instead attempt to construct a suitable explicit multivariate model for count data. $\endgroup$
    – Glen_b
    Jul 3, 2014 at 2:32
  • $\begingroup$ Thanks @Glen_b, but measuring the time for something to heal, is that also count data seeing that there non-negative integers (however the integers are only because we cannot measure this more exactly in real life). And what model would you view as suitable? $\endgroup$
    – OFish
    Jul 3, 2014 at 2:43
  • $\begingroup$ Time to heal may be measured in integer days (or whatever), so it does seem like a count of days, but it's actually a duration, a continuous variable that's been rounded up/down/off in some way; that should be treated as a duration. Beware the impact censoring on durations, however. $\endgroup$
    – Glen_b
    Jul 3, 2014 at 3:42
  • $\begingroup$ That should say "...beware the impact censoring has on durations..." $\endgroup$
    – Glen_b
    Jul 3, 2014 at 4:33

6 Answers 6

Reset to default
7
$\begingroup$

It is very unclear why you are dwelling on the CLT. What matters for your purpose is robustness and goodness of fit. For time-to-event data it is common to use a semiparametric survival model such as the Cox proportional odds model. You can also use the proportional odds ordinal logistic model, if there is no censoring. Such an ordinal model can handle the other outcome variables. Both proportional hazards and other ordinal models do not require you to select a transformation for $Y$, and they handle arbitrary clumping (e.g., at zero days).

Among other problems with "CLT thinking" is its false assumption that standard deviations are good measures of dispersion for asymmetrically distributed $Y$.

Improve this answer
$\endgroup$
3
  • $\begingroup$ "Dwelling" may be the wrong term here, but due to the sample size of the cohort, I thought the principle that I don't have to worry about data distribution may hold true in this instance, which is essentially what the CLT says, no? $\endgroup$
    – OFish
    Jul 4, 2014 at 1:58
  • 1
    $\begingroup$ The CLT is about distribution of the mean, not distribution of the data itself. What the CLT says is that for examples like you have above, if you take the mean Epithel of many subsamples of your data, the distribution of the mean Epithel of the subsamples will be approximately normally distributed, the approximation getting better as the size of the subsample improves. You're very fundamentally misunderstanding the CLT. $\endgroup$
    – Joe
    Jul 4, 2014 at 12:19
  • $\begingroup$ Thanks for clarifying that Joe. Helps getting these answers here. $\endgroup$
    – OFish
    Jul 10, 2014 at 4:08
5
$\begingroup$

Not only do you have to worry about normality, you have to worry about equality of variance; with count data (and right skew continuous distributions for that matter), variance tends to be related to mean in some way. This variance issue won't be fixed by the central limit theorem.

I think you're correct in your thought that perhaps you should use something like Poisson regression for your count variable. There are some alternatives, but Poisson regression would be the first thing I'd consider. However, from the look of some of your plots, it's possible you'll need something heavier tailed, like a negative binomial.

I'd initially be considering some form of GLM for all your response variables. This will also make it easier to deal with the curvature (via use of appropriate link functions).

You mention censoring in your comment below; you're right not to ignore the censoring - Cox proportional hazards (which you mentioned) is a standard thing to try but there are other options available.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks glen, that's some great input and also confirms what I've been thinking. The count data is somewhat easier for me to deal with than the time to heal as I'm dealing with discrete variable that a) might have an underlying exponential curve, b) is similar to count data but as you said is a duration and c) has censoring affecting it. I've run some cox ph - (see other q on SV regarding that though), which gives some pretty interesting results. $\endgroup$
    – OFish
    Jul 4, 2014 at 1:44
  • 1
    $\begingroup$ It sounds like you have it pretty much in hand. I made a few additional comments but nothing that adds anything of much value to what you seem to be doing already. $\endgroup$
    – Glen_b
    Jul 4, 2014 at 3:45
  • $\begingroup$ thanks glen. Would be interested to hear your opinion on the q i posted regarding the cox model. will also be discussing all of these points with my (stats) supervisor. $\endgroup$
    – OFish
    Jul 4, 2014 at 4:54
3
$\begingroup$

OP, I may be misreading you: if I am, please let me know and I'll delete this answer.

It seems to me that you are fundamentally confused about what the CLT means. Nobody else has pointed this out, so perhaps I am wrong -- again, if so, please chime in.

Your post includes the following:

OR - and this brings me to the question in my title - because of the size of the dataset, which is fairly large, could one say that the central limit theorem holds true and I don't have to worry about the data-distribution? Clearly my plots tell me I do, no?!?

If I log-transform my data, things start to look quite ok (Figure 3)

What exactly do you mean when you say "the central limit theorem holds true"? If I am reading you correctly, you believe that, given a large enough sample size, all distributions become normal -- and that is false.

To give a concrete example: if $X_i$ are i.i.d. Uniform on $[0, 1]$, then $\frac{1}{n}\sum_{i=1}^nX_i$ will be asymptotically normally distributed as $n\rightarrow\infty$. But that does not mean that, given a large sample size, you should expect a histogram of the $X_i$s to look like a normal distribution.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Adrian thanks for pointing that out and yes you did put your finger on it. I thought the CLT did in fact mean, that given a big enough sample size, normalization occurs and thus one does not need to "worry" about correction of the datasets for non-normality. But thank you for teaching me otherwise. Given your explanation however: If your plots tell you your data are non-normally distributed, you adapt the tests accordingly, right? Regardless if your sample sie were very large? Cheers. $\endgroup$
    – OFish
    Jul 7, 2014 at 5:01
  • 4
    $\begingroup$ The relevance of the CLT for regression is that the coefficient estimates tend towards having a normal distribution as sample size increases, even when the errors have a different distribution. $\endgroup$
    – Scortchi - Reinstate Monica ♦
    Jul 7, 2014 at 9:14
0
$\begingroup$

The central limit theorem is an asymptotic result, which means that it has to hold as $n \rightarrow \infty$ (under certain regularity conditions), large enough is too vague a term.

In your case the distribution has a very heavy right tail (judging from the variance computed in figure 2). Thus, the occasional large value can make up a big chunk of the sum.

To make ideas clearer suppose 100 numbers sampled from the true distribution yielded 98 zeros and 2 numbers with value 300. Your estimate of the mean will be 6 which is far more affected by the occasional large numbers (300).

Improve this answer
$\endgroup$
1
  • $\begingroup$ I think what you're saying applies here. Especially the time to heal. The fact of the matter is, that the mean is being massively afffected by those patients who had long healing times. One can see this in the histogram. Perhaps I should just look at it as continous variable and log-transform it. Seems to do the job. @Glen_b: Because of the censoring effect (when someone's healed they drop out) I've been thinking of using a cox-proportional hazards model for the regression analysis. $\endgroup$
    – OFish
    Jul 3, 2014 at 5:30
-1
$\begingroup$

The distributions of $X$ and $Y$ are not important

The distribution of $X$ and $Y$ don't matter to linear regression. Recall the usual assumptions of the ordinary least squares. Nowhere it says anything about the shape of the dependent and independent variable.

To believe it, run this regression in R:

x<-rexp(1000)
y<-5*x+rnorm(1000)
summary(lm(y~x))

Both $X$ and $Y$ are exponentially distributed, regression still finds the right parameter.

The normality of the errors is overrated

You showed us that the residuals are not normal. Okay, what does that actually tell you? Very little. It doesn't mean the estimates are wrong, try this in R:

x<-rexp(1000)
y<-5*x+rexp(1000)
summary(lm(y~x))

And see, even with exponential errors, the regression is still on the mark.

The fact is: normality of the errors only matters to the estimation of confidence intervals and p values. What if you really need confidence intervals or p-values and the residuals aren't normal? Use bootstrap.
A real problem is outliers (see those residuals at 15 in your plot?), those do bias results and you might have to remove them, or use some robust regression or both.

Your real problem is model selection

What I see is that you have many models you can feed this data to. You are asking us to pick one. We can't answer that for you. You have to do this on your own. Luckily it's pretty easy to do. Run all the models you can think of. All the regressions, with all the transformations of the data and all the permutations of regressors. Then choose the one that predicts better.

Learn cross-validation. Which is easy and does precisely that. Choose the model with the lowest cross-validated error

Improve this answer
$\endgroup$
29
  • 2
    $\begingroup$ Normality of errors is exceedingly important if you want to estimate the median, other quantiles, or the probability that $Y > y$. $\endgroup$
    – Frank Harrell
    Jul 3, 2014 at 12:01
  • 2
    $\begingroup$ That is not correct. If you assume a Gaussian model when it is false, your estimates of quantiles and probabilities will be incorrect; only the mean will be correct. The estimate of the mean may be very inefficient. The CLT has nothing to do with accuracy/model fit. $\endgroup$
    – Frank Harrell
    Jul 3, 2014 at 13:48
  • 1
    $\begingroup$ No, it's not. The bootstrap can get somewhat accurate confidence intervals for the wrong quantity, without assuming normality. Take the log-Gaussian one-sample problem. The MLE of the scale parameter uses the sum of squared differences on the log-transformed scale. The MLE of the mean is the anti-log of the MLE of the location parameter, multiplied by an anti-log of a constant times the MLE of the scale parameter. Analyzing the data on the original scale will be very inefficient. And in your previous comment you confused quantiles of statistics with quantiles of $Y | X$. $\endgroup$
    – Frank Harrell
    Jul 3, 2014 at 14:06
  • 1
    $\begingroup$ The fact that something may come from a log-Gaussian distribution implies that treating it as coming from a Gaussian distribution (or residuals are Gaussian) leads to many problems that the bootstrap doesn't know enough to fix. You can derive bootstrap estimators of quantiles (e.g., the Harrell-Davis estimator) but the vast majority of bootstrap users use the bootstrap to study the performance of an established estimator. $\endgroup$
    – Frank Harrell
    Jul 3, 2014 at 14:49
  • 3
    $\begingroup$ Cross-validation will tell you that you paid a price for model uncertainty. IMHO it is better to use a semiparametric model to completely avoid the $Y$-transform issue, and use a regression spline for $X$ and pay for $X$-transformation uncertainty according to the degrees of freedom in the spline function. $\endgroup$
    – Frank Harrell
    Jul 3, 2014 at 17:27
-1
$\begingroup$

Некоторые вдохновения: 1) Если мы не так много нулей, ZIP & ZINB может быть рациональный выбор; 2) мне кажется, приятно и проще inpretability будет для барьер модель: logit для reg шансов - "0 или >0" и какой-то граф-регрессии для >0 (Пуассона, Q-Poiss., negBin., гамма, logNorm и др.). Опять же, последняя похожа на " выбор модели' проблема (choosinng виде stohastic спецификации части). http://cran.r-project.org/web/packages/pscl/vignettes/countreg.pdf Удачи!

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.