3 added 77 characters in body
source | link

It's true that the frequentist properties of the Welch corrected test are better than the ordinary Student's T, at least for errors. I agree that that alone is a pretty good argument for the Welch test. However, I neverI'm usually reluctant to recommend the Welch correction because itit's use is usuallyoften deceptive. Which is, admittedly not a critique of the test itself.

The reason I don't recommend the Welch correction is that it doesn't just change the degrees of freedom and subsequent theoretical distribution from which the p-value is drawn. It makes the test non-parametric. To perform a Welch corrected t-test one still pools variance as if equal variance can be assumed but then changes the final testing procedure implying either that equal variance cannot be assumed, or that you only care about the sample variances. This makes it a non-parametric test because the pooled variance is considered non-representative of the population and you conceded that you're just testing your observed values.

In and of itself there's nothing particularly wrong with that. However, I find it deceptive because a) typically it's not reported with enough specificity; and b) the people who use it tend to think about it interchangeably with a t-test. The only way I ever know that it has been done in published papers is when I see an odd DF for the t-distribution. That was also the only way Rexton (referenced in the Henrik answer) could tell in review. Unfortunately, the non-parametric nature of the Welch corrected test occurs whether the degrees of freedom have changed or not (i.e. even if the sample variances are equal). But this reporting issue is symptomatic of the fact that most people who use the Welch correction don't recognize this change to the test has occurred. I just interviewed a few colleagues and they admitted they had never even thought of it.

Therefore, because of this, I believe that if you're going to recommend a non-parametric test don't use one that often appears parametric or at least be very clear about what you're doing. The official name of the test should be Non-Parametric Welch Corrected T-test. If people reported it that way I'd be much happier with Henrik's recommendation.

It's true that the frequentist properties of the Welch corrected test are better than the ordinary Student's T, at least for errors. I agree that that alone is a pretty good argument for the Welch test. However, I never recommend the Welch correction because it is usually deceptive.

The reason I don't recommend the Welch correction is that it doesn't just change the degrees of freedom and subsequent theoretical distribution from which the p-value is drawn. It makes the test non-parametric. To perform a Welch corrected t-test one still pools variance as if equal variance can be assumed but then changes the final testing procedure implying either that equal variance cannot be assumed, or that you only care about the sample variances. This makes it a non-parametric test because the pooled variance is considered non-representative of the population and you conceded that you're just testing your observed values.

In and of itself there's nothing particularly wrong with that. However, I find it deceptive because a) typically it's not reported with enough specificity; and b) the people who use it tend to think about it interchangeably with a t-test. The only way I ever know that it has been done in published papers is when I see an odd DF for the t-distribution. That was also the only way Rexton (referenced in the Henrik answer) could tell in review. Unfortunately, the non-parametric nature of the Welch corrected test occurs whether the degrees of freedom have changed or not (i.e. even if the sample variances are equal). But this reporting issue is symptomatic of the fact that most people who use the Welch correction don't recognize this change to the test has occurred. I just interviewed a few colleagues and they admitted they had never even thought of it.

Therefore, because of this, I believe that if you're going to recommend a non-parametric test don't use one that often appears parametric or at least be very clear about what you're doing. The official name of the test should be Non-Parametric Welch Corrected T-test. If people reported it that way I'd be much happier with Henrik's recommendation.

It's true that the frequentist properties of the Welch corrected test are better than the ordinary Student's T, at least for errors. I agree that that alone is a pretty good argument for the Welch test. However, I'm usually reluctant to recommend the Welch correction because it's use is often deceptive. Which is, admittedly not a critique of the test itself.

The reason I don't recommend the Welch correction is that it doesn't just change the degrees of freedom and subsequent theoretical distribution from which the p-value is drawn. It makes the test non-parametric. To perform a Welch corrected t-test one still pools variance as if equal variance can be assumed but then changes the final testing procedure implying either that equal variance cannot be assumed, or that you only care about the sample variances. This makes it a non-parametric test because the pooled variance is considered non-representative of the population and you conceded that you're just testing your observed values.

In and of itself there's nothing particularly wrong with that. However, I find it deceptive because a) typically it's not reported with enough specificity; and b) the people who use it tend to think about it interchangeably with a t-test. The only way I ever know that it has been done in published papers is when I see an odd DF for the t-distribution. That was also the only way Rexton (referenced in the Henrik answer) could tell in review. Unfortunately, the non-parametric nature of the Welch corrected test occurs whether the degrees of freedom have changed or not (i.e. even if the sample variances are equal). But this reporting issue is symptomatic of the fact that most people who use the Welch correction don't recognize this change to the test has occurred. I just interviewed a few colleagues and they admitted they had never even thought of it.

Therefore, because of this, I believe that if you're going to recommend a non-parametric test don't use one that often appears parametric or at least be very clear about what you're doing. The official name of the test should be Non-Parametric Welch Corrected T-test. If people reported it that way I'd be much happier with Henrik's recommendation.

2 added 470 characters in body
source | link

It's true that the frequentist properties of the Welch corrected test are better than the ordinary Student's T, at least for errors. I agree that that alone is a pretty good argument for the Welch test. However, I never recommend the Welch correction because the it is usually deceptive.

The reason I don't recommend the Welch correction is that it doesn't just change the degrees of freedom and subsequent theoretical distribution from which the p-value is drawn. It makes the test non-parametric. To perform a Welch corrected t-test one still pools variance as if equal variance can be assumed but then changes the final testing procedure implying either that equal variance cannot be assumed, or that you only care about the sample variances. This makes it a non-parametric test because the pooled variance is considered non-representative of the population and you conceded that you're just testing your observed values.

In and of itself there's nothing particularly wrong with that. However, I find it deceptive because a) typically it's not reported with enough specificity; and b) the people who use it tend to think about it interchangeably with a t-test. The only way I ever know that it has been done in published papers is when I see an odd DF for the t-distribution. That was also the only way Rexton (referenced in the Henrik answer) could tell in review. Unfortunately, the non-parametric nature of the Welch corrected test occurs whether the degrees of freedom have changed or not (i.e. even if the sample variances are equal). But this reporting issue is symptomatic of the fact that most people who use the Welch correction don't recognize this change to the test has occurred. I just interviewed a few colleagues and they admitted they had never even thought of it.

Therefore, because of this, I believe that if you're going to recommend a non-parametric test don't use one that often appears parametric or at least be very clear about what you're doing. The official name of the test should be Non-Parametric Welch Corrected T-test. If people reported it that way I'd be much happier with Henrik's recommendation.

It's true that the frequentist properties of the Welch corrected test are better than the ordinary Student's T, at least for errors. I agree that that alone is a pretty good argument for the Welch test. However, I never recommend the Welch correction because the it is deceptive.

The reason I don't recommend the Welch correction is that it doesn't just change the degrees of freedom and subsequent theoretical distribution from which the p-value is drawn. It makes the test non-parametric. To perform a Welch corrected t-test one still pools variance as if equal variance can be assumed but then changes the final testing procedure implying either that equal variance cannot be assumed, or that you only care about the sample variances. This makes it a non-parametric test because the pooled variance is considered non-representative of the population and you conceded that you're just testing your observed values.

In and of itself there's nothing particularly wrong with that. However, I find it deceptive. The only way I ever know that it has been done in published papers is when I see an odd DF for the t-distribution. Unfortunately, the non-parametric nature of the Welch corrected test occurs whether the degrees of freedom have changed or not (i.e. even if the sample variances are equal).

Therefore, because of this, I believe that if you're going to recommend a non-parametric test don't use one that often appears parametric or at least be very clear about what you're doing. The official name of the test should be Non-Parametric Welch Corrected T-test. If people reported it that way I'd be much happier with Henrik's recommendation.

It's true that the frequentist properties of the Welch corrected test are better than the ordinary Student's T, at least for errors. I agree that that alone is a pretty good argument for the Welch test. However, I never recommend the Welch correction because it is usually deceptive.

The reason I don't recommend the Welch correction is that it doesn't just change the degrees of freedom and subsequent theoretical distribution from which the p-value is drawn. It makes the test non-parametric. To perform a Welch corrected t-test one still pools variance as if equal variance can be assumed but then changes the final testing procedure implying either that equal variance cannot be assumed, or that you only care about the sample variances. This makes it a non-parametric test because the pooled variance is considered non-representative of the population and you conceded that you're just testing your observed values.

In and of itself there's nothing particularly wrong with that. However, I find it deceptive because a) typically it's not reported with enough specificity; and b) the people who use it tend to think about it interchangeably with a t-test. The only way I ever know that it has been done in published papers is when I see an odd DF for the t-distribution. That was also the only way Rexton (referenced in the Henrik answer) could tell in review. Unfortunately, the non-parametric nature of the Welch corrected test occurs whether the degrees of freedom have changed or not (i.e. even if the sample variances are equal). But this reporting issue is symptomatic of the fact that most people who use the Welch correction don't recognize this change to the test has occurred. I just interviewed a few colleagues and they admitted they had never even thought of it.

Therefore, because of this, I believe that if you're going to recommend a non-parametric test don't use one that often appears parametric or at least be very clear about what you're doing. The official name of the test should be Non-Parametric Welch Corrected T-test. If people reported it that way I'd be much happier with Henrik's recommendation.

1
source | link

It's true that the frequentist properties of the Welch corrected test are better than the ordinary Student's T, at least for errors. I agree that that alone is a pretty good argument for the Welch test. However, I never recommend the Welch correction because the it is deceptive.

The reason I don't recommend the Welch correction is that it doesn't just change the degrees of freedom and subsequent theoretical distribution from which the p-value is drawn. It makes the test non-parametric. To perform a Welch corrected t-test one still pools variance as if equal variance can be assumed but then changes the final testing procedure implying either that equal variance cannot be assumed, or that you only care about the sample variances. This makes it a non-parametric test because the pooled variance is considered non-representative of the population and you conceded that you're just testing your observed values.

In and of itself there's nothing particularly wrong with that. However, I find it deceptive. The only way I ever know that it has been done in published papers is when I see an odd DF for the t-distribution. Unfortunately, the non-parametric nature of the Welch corrected test occurs whether the degrees of freedom have changed or not (i.e. even if the sample variances are equal).

Therefore, because of this, I believe that if you're going to recommend a non-parametric test don't use one that often appears parametric or at least be very clear about what you're doing. The official name of the test should be Non-Parametric Welch Corrected T-test. If people reported it that way I'd be much happier with Henrik's recommendation.