17
$\begingroup$

I am trying to test the null $E[X] = 0$, against the local alternative $E[X] > 0$, for a random variable $X$, subject to mild to medium skew and kurtosis of the random variable. Following suggestions by Wilcox in 'Introduction to Robust Estimation and Hypothesis Testing', I have looked at tests based on the trimmed mean, the median, as well as the M-estimator of location (Wilcox' "one-step" procedure). These robust tests do outperform the standard t-test, in terms of power, when testing with a distribution that is non-skewed, but leptokurtotic.

However, when testing with a distribution that is skewed, these one-sided tests are either far too liberal or far too conservative under the null hypothesis, depending on whether the distribution is left- or right-skewed, respectively. For example, with 1000 observations, the test based on the median will actually reject ~40% of the time, at the nominal 5% level. The reason for this is obvious: for skewed distributions, the median and the mean are rather different. However, in my application, I really need to test the mean, not the median, not the trimmed mean.

Is there a more robust version of the t-test that actually tests for the mean, but is impervious to skew and kurtosis?

Ideally the procedure would work well in the no-skew, high-kurtosis case as well. The 'one-step' test is almost good enough, with the 'bend' parameter set relatively high, but it is less powerful than the trimmed mean tests when there is no skew, and has some troubles maintaining the nominal level of rejects under skew.

background: the reason I really care about the mean, and not the median, is that the test would be used in a financial application. For example, if you wanted to test whether a portfolio had positive expected log returns, the mean is actually appropriate because if you invest in the portfolio, you will experience all the returns (which is the mean times the number of samples), instead of $n$ duplicates of the median. That is, I really care about the sum of $n$ draws from the R.V. $X$.

$\endgroup$
  • $\begingroup$ Is there a reason that prohibit the use of the Welch t-test? Have a look at my answer to this question (stats.stackexchange.com/questions/305/…) where i refer to a paper advocating the use of Welch in case of non-normality and heteroscedasticity. $\endgroup$ – Henrik Aug 9 '10 at 13:09
  • 1
    $\begingroup$ well, the problem is that I want a 1-sample test, not a 2-sample test! I am testing the null $E[X] = \mu$, and not $E[X_1] = E[X_2]$. I will look up the Kubinger et. al., paper (Ich kann schlecht Deutsche). $\endgroup$ – shabbychef Aug 9 '10 at 17:02
  • $\begingroup$ Thanks for clarifying. In this case the Kubinger paper will not be very helpful to you. I am Sorry. $\endgroup$ – Henrik Aug 10 '10 at 11:18
5
$\begingroup$

Why are you looking at non-parametric tests? Are the assumptions of the t-test violated? Namely, ordinal or non-normal data and inconstant variances? Of course, if your sample is large enough you can justify the parametric t-test with its greater power despite the lack of normality in the sample. Likewise if your concern is unequal variances, there are corrections to the parametric test that yield accurate p-values (the Welch correction).

Otherwise, comparing your results to the t-test is not a good way to go about this, because the t-test results are biased when the assumptions are not met. The Mann-Whitney U is an appropriate non-parametric alternative, if that's what you really need. You only lose power if you are using the non-parametric test when you could justifiably use the t-test (because the assumptions are met).

And, just for some more background, go here...

http://www.jerrydallal.com/LHSP/STUDENT.HTM

$\endgroup$
  • $\begingroup$ the data is most definitely not normal. the excess kurtosis is on the order of 10-20, the skew is on the order of -0.2 to 0.2. I am doing a 1-sample t-test, so I'm not sure I follow you regarding 'unequal variances', or the U-test. $\endgroup$ – shabbychef Aug 7 '10 at 16:14
  • $\begingroup$ I am accepting the 'use a parametric test' advice. it doesn't exactly solve my question, but my question was probably too open-ended. $\endgroup$ – shabbychef Aug 10 '10 at 0:35
12
$\begingroup$

I agree that if you want to actually test whether the group means are different (as opposed to testing differences between group medians or trimmed means, etc.), then you don't want to use a nonparametric test that tests a different hypothesis.

  1. In general p-values from a t-test tend to be fairly accurate given moderate departures of the assumption of normality of residuals. Check out this applet to get an intuition on this robustness: http://onlinestatbook.com/stat_sim/robustness/index.html

  2. If you're still concerned about the violation of the normality assumption, you might want to bootstrap. e.g., http://biostat.mc.vanderbilt.edu/wiki/pub/Main/JenniferThompson/ms_mtg_18oct07.pdf

  3. You could also transform the skewed dependent variable to resolve issues with departures from normality.

$\endgroup$
  • 2
    $\begingroup$ +1 nice and clear answer. Jeromy, can I ask a question about point 3? I do understand the reasoning behind transforming the data, but something always bothered me about doing that. What is the validity of reporting the results of the t-test on the transformed data to the untransformed data (where you're not "allowed" to do a t-test)? In other words, if two groups are different when data is, for instance, log transformed, on what bases can you say the raw data is different too? Bare in mind, I'm not a statistician, so maybe I just said something absolutely stupid :) $\endgroup$ – nico Aug 7 '10 at 9:50
  • 2
    $\begingroup$ @nico I'm not sure about how to report or think about the results, but if all you want to show is that for some X and Y, mu_X != mu_Y, it should be true that for all X_i < X_j, log(X_i) < log(X_j) and for all all X_i > X_j, log(X_i) > log(X_j). That's why for non-parametric tests which operate with ranks, transformations of the data don't affect the result. I think from this, you can assume that if some test shows that mu_log(X) != mu_log(Y), then mu_X != mu_Y. $\endgroup$ – JoFrhwld Aug 7 '10 at 15:57
  • $\begingroup$ thanks for the answer(s). indeed, the t-test appears to maintain nominal type I rate under mildly skewed/kurtotic input. however, I was hoping for something with more power. re: 2, I have implemented Wilcox' trimpb and trimcibt, but they are a bit too slow to do my power tests, at least for my taste. re: 3, I had thought of this method, but I am interested in the mean of the un-transformed data (i.e., I am not comparing 2 R.V.s with a t-test, in which case, a monotonic transform would be fine for a rank-based comparison, as noted by @JoFrhwld.) $\endgroup$ – shabbychef Aug 7 '10 at 16:11
  • 2
    $\begingroup$ @nico If the population distribution of residuals is the same in two groups, then I imagine any time there is a difference in raw population group means there would also be differences in group means of an order-preserving transformation. That said, p-values and confidence intervals will tend to change slightly based on whether you are using raw data or transformed data. In general I prefer to use transformations when they seem like a meaningful metric for understanding the variable (e.g., Richter scale, decibels, logs of counts, etc.). $\endgroup$ – Jeromy Anglim Aug 8 '10 at 3:49
3
$\begingroup$

Johnson (1978) gives a modification for the $t$-statistic and confidence intervals which is a good starting point for my problem. The correction is based on a Cornish-Fisher expansion, and uses sample skew.

The 'latest and greatest' is due to Ogaswara, with references therein to Hall and others.

$\endgroup$
0
$\begingroup$

I don't have enough reputation for a comment, thus as an answer: Have a look at this calcuation. I think this provides an excellent answer. In brief:

The asymptotic performance is much more sensitive to deviations from normality in the form of skewness than in the form of kurtosis ... Thus Student's t-test is sensitive to skewness but relatively robust against heavy tails, and it is reasonable to use a test for normality that is directed towards skew alternatives before applying the t-test.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.