1
$\begingroup$

I was wondering if people had recommendations for modeling severely skewed and/or kurtotic (coefficients more extreme than + or -3) outcome data with negative values. I would traditionally use a Poisson or negative binomial or gamma regression (i.e., generalized linear model), but those models require all scores to be non-negative. Any idea of what you would use to 1) test whether the mean of the non-normal scores is different than zero and 2) predict the non-normal scores with IVs?

Added detail

I have count data at two timepoints (number of mental health referrals in the past 3 months). The count data at each timepoint are Poisson distributed with moderate overdispersion (75% zeros). I want to look at changes in the counts from pre-intervention to post-intervention. So the outcome variable I am working with is the difference of the two counts (post - pre). The difference scores are very kurtose positive in that most people don't change (difference score = 0), but then it is also skewed positive with more people increasing than decreasing. But there are people who decreased, leading to negative values. I want to estimate a central tendency for the difference scores and predict the difference scores with demographic variables, such as gender and age. I can get the point estimates with the mean and OLS regression, but how do I get accurate measures of uncertainty? The standard error of the mean and the standard error of the regression coefficients will be biased by the positive kurtosis and skew.

I was hoping there was a generalized linear model that I wasn't familiar with whose distribution fit my outcome variable; however, maybe that was naively optimistic...

$\endgroup$
1
$\begingroup$

It depends in part on what it is you're looking at; frequently an understanding of the variable (rather than will help in either choosing a model or identifying a better way to look at the model (transformations, re-expression in combination to other variables and so on)

That said, if you want to deal with inference for the mean there's always permutation tests.

It can also be used for testing a regression model overall in the absence of a distributional model; prediction depends on what it is you're trying to predict (is it a mean prediction?). If you're not focused on the mean for regression you might look at something like robust regression methods.


Edit: in response to comments

A suitable umbrella term that includes both bootstrap tests and permutation tests would be resampling tests. However, since both can be used to construct intervals, sometimes the broader methodologies are called resampling methods.

Bootstrapping is large sample and approximate, but somewhat more flexible (and more aimed at finding standard errors and confidence intervals but can indeed be used for testing) and based on a different form of resampling than permutation tests.

If you can understand bootstrapping, and you understand hypothesis testing, permutation tests are (if anything) somewhat easier to understand.

Permutation tests work for both large and small samples and are exact in a particular sense.

Almost all the common nonparametric tests are permutation tests (many of those use ranks -- the advantage of basing tests on ranks is you don't need a computer to work out the distribution under the null for each new sample -- which was essential 60 or 70 years ago, in the era when nobody had computers).

However, you can use permutation tests with any convenient statistic, as long as you can assert exchangeability of observations (or at least of whatever is being permuted) under the null. With large samples bootstrapping should be fine. With small samples, I would prefer permutation tests - but for the most obvious permutation test of the mean being zero, that would require the assumption of symmetry about 0 when the null is true (under the alternative, symmetry is not required, so you don't necessarily have to have it in your sample -- only that you can argue it should be the case when the null is true; as an example, if your values are pair-differences, people are often happy to assert it).

The bootstrap doesn't require assuming exchangeability under the null but it has its own requirements / issues (not all of them obvious). Not least for your application, bootstrapping a mean when the distribution is heavy tailed can pose difficulties (and small sample properties may be quite far from the nominal properties when it is the case).

Another alternative might be to assume some parametric model; there are a variety of skewed heavy tailed distributions you might consider that could include negative values (including mixture distributions)

With regression models, at least for some tests you can do permutation tests, or approximate permutation tests. You can also do bootstrap tests. [Again there's also the alternative of a parametric model.]

$\endgroup$
  • $\begingroup$ Thanks for your reply. I haven't heard of permutation tests, but I have heard of bootstrapping. I read a little on permutation tests from googling. Is it an umbrella term for any resampling method, for which bootstrapping is a specific type? If yes, would bootstrapping be something you would consider? $\endgroup$ – David Disabato Mar 8 '17 at 4:13
  • $\begingroup$ @David see edit to my answer. Short version: 1. no, but they're both particular subcases of a more general term. If you've done a sign test or a Mann-Whitney test or a Kolmogorov-Smirnov two-sample test you've done a (particular kind of) permutation test. 2. Possibly, depending on a number of considerations $\endgroup$ – Glen_b Mar 8 '17 at 8:34
  • $\begingroup$ Thanks much @Gleb_n. I am curious what parametric models you are thinking of that allow for heavy tails and negative values? Are any of these regression models that don't involve mixtures? $\endgroup$ – David Disabato Mar 11 '17 at 1:21
  • $\begingroup$ Well, mixtures are a reasonably straightforward general approach, but without more of an idea what your data really look like and no discussion of subject area (so even if I had subject-area expertise I can't use it), suggesting models that are not mixtures would be like trying to hit a bullseye when your dartboard is at the bottom of a lake and you're trying to guess where it is by examining a few ripples on the surface... I could potentially name some possibilities (e.g. difference of two lognormals with different parameters) but they'll be sure to be in the wrong half of the lake. $\endgroup$ – Glen_b Mar 11 '17 at 4:33
  • $\begingroup$ btw it's @Glen_b not @Gleb_n. better chance of getting my attention if you tell the system the right username to notify $\endgroup$ – Glen_b Mar 11 '17 at 4:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.