3
$\begingroup$

My dependent variable is the answer to an open numerical question. E.g. participants were asked to indicate how often within the last three months they were suffering from a headache. I want to know if a certain clinical group is suffering from a headache more often compared to the average population, whereas another clinical group is suffering from a headache less often compared to the average population.

Of course I can calculate two one-sample t-tests, and compare if the number of headaches for both groups deviates significantly from the number of headaches in the average population.

Let's say I want to compare a unipolar depression sample with a bipolar depression sample. The average number of headaches within the last 3 months in the average population is = 5 .

In R I would calculate this as follows:

t.test(data$headache[data$condition=="uni_depression"], mu=5, method = "bonferroni") t.test(data$headache[data$condition=="bipolar"], mu=5, method = "bonferroni")

However I was wondering if there is a more elegant solution/statistical approach than separate t-tests to calculate this?

$\endgroup$
2
  • $\begingroup$ Are there only 2 clinical groups in your analysis and are they complimentary sets of one another? $\endgroup$ Feb 22, 2019 at 17:07
  • $\begingroup$ I would not use t-tests. The number of headaches is likely to be extremely skew. Maybe Mann Whitney. $\endgroup$
    – Peter Flom
    Feb 22, 2019 at 18:50

4 Answers 4

6
$\begingroup$

NOTE: I am not sure this answer is right, but I think it is. It's an idea. I know others here have more math background than I do and will point out if I have messed up.

First, I'd not use t-tests. The data are likely to be extremely skew and are also counts.

Second, I like @StatsStudent 's idea, but I don't think a linear model is right (as he notes in his final paragraph). I think it should be some form of count model, such as Poisson or negative binomial, perhaps with zero inflation.

Third, the problem that arises is that you have a non-zero comparison. In the linear model, you might just subtract 5 from every value, but that is going to mess up the count nature of the data. So .... How about an offset of 5 and then a NB regression with "diagnosis" as the independent variable?

I know that's not the usual use of an offset, but, intuitively, it seems right to me.

$\endgroup$
2
  • 1
    $\begingroup$ +10, for this answer, but do note my final paragraph indicates the need to use a count model as you've suggested. The contrast method presented therein can be used there, but the model itself would change. $\endgroup$ Feb 22, 2019 at 19:06
  • $\begingroup$ Oops, I missed that. I will edit my answer $\endgroup$
    – Peter Flom
    Feb 22, 2019 at 19:34
6
$\begingroup$

It seems to me you might have a more "elegant" solution using a linear model with contrasts to test mean differences in headaches present in the different clinical groups. For example you could employ the following linear model (the "cell means model"):

\begin{eqnarray*} Y_{ij} & = & \mu+\epsilon_{ij} \end{eqnarray*}

where $Y_{ij}$ is the number of headaches reported for the $j$th individual in the $i$th clinical group, and $\epsilon_{ij}$ is the error term, where $\epsilon_{ij}\sim N\left(0,\sigma^{2}\right)$, and $i=1,\ldots r$; $j=1,\ldots n_{i}$. In terms of your data, the model can be specified in matrix terms as:

\begin{eqnarray*} \boldsymbol{Y} & = & \boldsymbol{X\beta}+\boldsymbol{\epsilon} \end{eqnarray*}

where

$\boldsymbol{Y}=\begin{bmatrix}Y_{11}\\ Y_{12}\\ \vdots\\ Y_{1n_{1}}\\ Y_{21}\\ Y_{22}\\ \vdots\\ Y_{2n_{2}} \end{bmatrix}$, $\boldsymbol{\beta}=\left[\begin{array}{c} \mu_{1}\\ \mu_{2} \end{array}\right]$, $\boldsymbol{X}=\left[\begin{array}{cc} 1 & 0\\ \vdots & \vdots\\ 1 & 0\\ 0 & 1\\ \vdots & \vdots\\ 0 & 1 \end{array}\right]$, and $\boldsymbol{\epsilon}=\begin{bmatrix}\epsilon_{11}\\ \epsilon_{12}\\ \vdots\\ \epsilon_{1n_{1}}\\ \epsilon_{21}\\ \epsilon_{22}\\ \vdots\\ \epsilon_{2n_{2}} \end{bmatrix}$.

In this case, you'd simply create $r$ columns for each of your $r$ treatments, where a 1 would be present if the patient was in the clinical group (zero otherwise). Then you could simply perform the hypothesis test:

\begin{align*} H_{0}\text{: } & \mu_{1}=\mu_{2} & \text{versus} & & H_{1}\text{: } & \text{not all } \mu_{i} \text{ are equal} \end{align*}

This would tell you if there were any differences in the mean number of headaches among the groups. This can be extended, without loss of generality, to the case where you specify multiple (i.e. $r > 2$) clinical groups. You could then create contrasts to determine, if, for example, one group's mean number of headaches was different from the mean of all groups, for example, if $r=4$ (i.e. you had four clinical groups), you could simply test:

\begin{align*} H_{0}: & L=0 & \text{versus} & & H_{1}: & L\ne0 \end{align*}

where $L=\mu_{1}-\frac{\mu_{1}+\mu_{2}+\mu_{3}+\mu_{4}}{4}$.

You could also test this with matrices using $H_0:\boldsymbol{C^{\prime}X}=0$ where $\boldsymbol{C^{\prime}}=\begin{bmatrix}\frac{3}{4} & -\frac{1}{4} & -\frac{1}{4} & -\frac{1}{4}\end{bmatrix}$.

Of course, you would need to make multiple comparison adjustments if you performed multiple tests of contrasts (e.g. Tukey's, Scheffe's, or Bonferroni's Multiple Comparison's Procedure). However, if you are really only interested in testing the equality of the mean number of headaches across clinical groups, you could do this with the one omnibus $F$ test. For additional details, see Chapters 16 and 17 in Applied Linear Statistical Models, 5th ed. by Kutner, et al.

One final note is that if you use a linear models approach, you'd have to ensure that the assumptions of the linear model are appropriate for your data (but to be clear, you'd need to do this in your t-test approach too). Since you are dealing with count data, the errors may not align with the error term assumptions of the model. Therefore, you may have to use a Poisson regression or consider using some other model appropriate for count data.

$\endgroup$
3
  • $\begingroup$ This is clever, but the underlying assumption of homoscedastic error gives me pause. $\endgroup$
    – AdamO
    Feb 22, 2019 at 19:13
  • $\begingroup$ That could be handled by weighted regression, or by sandwich-estimators or using a GEE model too, depending on the exact setup of the problem (I suspect additional variables are at play, perhaps multiple measures, etc.). $\endgroup$ Feb 22, 2019 at 19:29
  • 1
    $\begingroup$ I've been interested in this topic for a while: regression models with known but unequal variance in two or more subsets of observations. I can write these down as special cases of random effect/GEE models, but in simulations I cannot get consistent estimates of error from them. My hunch is that the contrived way of expressing the error as exactly two clusters causes unreliable estimates of ICC. I would not advocate the approach because it can be really inefficient in small sample sizes. $\endgroup$
    – AdamO
    Feb 24, 2019 at 14:04
5
$\begingroup$

Unfortunately, they truly seem to be independent hypotheses and so should be tested with separate tests. This speculation has to do with accepting data at its face value with just the two values headache and condition.

Note that specifying method="bonferroni" in each test will not intelligently look between either test. What you will need to do is extract the $p$-value from each analysis, create a vector, and input that vector to the p.adjust function in R to obtain Bonferroni adjusted $p$-values. Was that your question?

There are also other methods of handling multiple testing. Again, this consideration is not a matter of elegance as you put it. And questions about code are off-topic.

$\endgroup$
0
4
$\begingroup$

The approach to combine the two clinical groups in order to estimate their means simultaneously also makes strong assumptions about the error variance. Most obviously that the two clinical groups have the same error distribution. (And then the normal regression and the poisson regression make different assumptions about what that error distribution is). It doesn't seem necessary to make extra assumptions about the variance when you are only interested in comparing means.

You want to compare whether a person with bipolar depression tends to have more than 5 headaches in 3 months. As Peter Flom has noted the number of headaches might be skewed to the right (meaning that there might be more patients with more than 5 headaches than with fewer than 5 headaches).

Since you know the directionality you want to test for ("more than 5 headaches") you could convert the numerical responses to binary responses and instead test a simpler hypothesis that the probability of answering "more than 5" is 1/2. For example if you have responses {4,5,7,20} convert these to {0,0,1,1}. Obviously you lose information that one respondent had so many headaches in 3 months. But this deals with the skewness issue. And you can use a simple test for proportions.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.