0
$\begingroup$

My is a series of individuals have attempted a test. They can fail this test for multiple reasons. Furthermore, on an test, a fail can be awarded multiple times for one reason, i.e.:

Steve: 1,0,2,1 Jane: 2,0,0,1 are both valid entries for a test with four reasons.

Having said that, many of the reasons are effectively a boolean, i.e., the column can hold either a 1 or a 0 only.

I'm interested in a subset of the rows of this table, e.g., Steve and his family. Specifically, I want to know if the distribution of the mean of each column (reason) within the subset differs significantly from the equivalent distribution for the whole population.

Is a $\chi^2$ test with the null hypothesis that the subset distribution matches the whole population distribution appropriate here?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Please edit to clarify: (1) By "subset", do you mean a set of rows, as opposed to a set of columns? (2) Wouldn't you want to compare the means rather than the sums, since the bounds on the sum depend on the number of subjects? $\endgroup$
    – Kodiologist
    Commented Jan 24, 2019 at 18:52
  • $\begingroup$ @Kodiologist Yes, thanks, edited to follow these suggestions. $\endgroup$
    – lovelyzoo
    Commented Jan 25, 2019 at 11:49

2 Answers 2

Reset to default
4
$\begingroup$

If the point is just to compare means, there's no need to compare entire distributional forms; you can use mean-specific tests. For each failure reason (column), you can frame the problem as comparing the means of two samples: one sample is Steve and his family, and the other sample is every other row of your data. You can test whether these sample means are significantly different with a two-sample $t$-test. It's probably best to use Welch's $t$-test, because the assumption of equal variances is unrealistic.

One could argue that Welch's $t$-test isn't appropriate, either, because the dependent variable clearly isn't normally distributed. To avoid this issue, you can use a permutation test instead. The usual nonparametric answer to the two-sample $t$-test, the Mann-Whitney $U$-test, doesn't compare means, so that's probably not a good choice.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Per this link the dependent variable should be continuous. These variables are counts of failures and therefore integers. Surely that means the continuity clause is violated? $\endgroup$
    – lovelyzoo
    Commented Jan 25, 2019 at 13:26
  • 2
    $\begingroup$ @lovelyzoo Specifically, the assumption is that the dependent variable is normally distributed. You're quite right that that's not true here; however, the usual nonparametric answer to the two-sample $t$-test, the Mann-Whitney $U$-test, doesn't compare means, so that's probably not a good choice. One thing you could try if you're worried about this is a permutation test. $\endgroup$
    – Kodiologist
    Commented Jan 25, 2019 at 13:46
0
$\begingroup$

Following the discussion with @Kodiologist, permutation testing seems to be the best way to go. More about which can be found here.

Improve this answer
$\endgroup$
3
  • $\begingroup$ The thing is that @Kodiologist's initial answer suggests Welch's t-test. It was in the subsequent discussion that the permutation test was suggested. For that reason, I thought it better to post this answer so that a future reader is directed to the most appropriate test. $\endgroup$
    – lovelyzoo
    Commented Jan 30, 2019 at 9:06
  • $\begingroup$ @Kodiologist, feel free to edit the original answer as per gung's suggestion. I'll then mark your answer as correct. $\endgroup$
    – lovelyzoo
    Commented Jan 30, 2019 at 17:05
  • 1
    $\begingroup$ @lovelyzoo Done. $\endgroup$
    – Kodiologist
    Commented Jan 30, 2019 at 17:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.