5
$\begingroup$

I'm very rusty on statistics, so go easy on me haha.

I'm trying to compare two sample sets of values to see if there is any significance difference between the two means. From memory, an unpaired t-test should be able to do this? The issue is that one set has zero variance, so do I use an equal variance test or an unequal?

If there's a better test for this then feel free to let me know :)

Group 1 Group 2
3.97 4.00
3.63 4.00
3.63 4.00
4.00 4.00
4.29 4.00
3.95 4.00
3.63 4.00
3.63 4.00
3.63 4.00
3.96 4.00
3.63
4.00
3.63
3.63
3.93
3.97
4.00
3.63
4.00
3.92
4.26
4.08
3.43
4.26
4.17
3.63
$\endgroup$
14
  • 2
    $\begingroup$ What are you measuring? Why does the second group have no variation? $\endgroup$ Nov 14, 2022 at 16:37
  • $\begingroup$ @DemetriPananos They're yields of Beans for individual farm fields (tons per hectare). To be honest, I don't believe Group 2 should have no variance at all. It's likely we were given data rounded to the nearest ton but its what we have. $\endgroup$
    – S7ewie
    Nov 14, 2022 at 16:44
  • 2
    $\begingroup$ I think it matters if Group2 is rounded to the nearest ton / ha. Because it just so happens that if you round Group1 to the nearest whole number, almost all of the observations have a value of 4. But if you don't round Group1, the majority are less than 4. $\endgroup$ Nov 14, 2022 at 17:21
  • 2
    $\begingroup$ @SalMangiafico or worse, if we use typical midpoint rounding all of Group 1 is 4, except for one unlucky bean miser weighing in at 3.43. $\endgroup$
    – AdamO
    Nov 14, 2022 at 17:38
  • 1
    $\begingroup$ @SalMangiafico Don't get me started on precision agriculture! XD Combine harvesters are usually pretty good at tracking yield so my guess is that, for whatever reason, the farmer doesn't have direct access to that data so he took a rough guess on the (total tonnage / total area) and gave that same value to each field. A one-sample test suggests there is a significant difference between the two groups but its 3.87 vs 4.00 and I imagine if group 2 had any variance at all it would fail the test so I think I'm just going to go back and say the data isn't accurate enough. Thanks! $\endgroup$
    – S7ewie
    Nov 14, 2022 at 21:14

4 Answers 4

17
$\begingroup$

This answer doesn't address the issue of why the second group has no variation (and I really do suggest you get to the bottom of that).

If you're comfortable saying the yield of group 2 is definitely known to be 4.0 you can use a one-sample t-test to compare group 1 against that value.

Update

I should clarify some things, since this answer is getting more upvotes than it deserves. First, with the two-sample test you're testing the hypothesis that group 1 differs from group 2. With the one-sample test, you're testing the hypothesis that group 1 differs from a value of 4.0. These are subtly different questions.

Second, I should stress that treating the value of 4.0 as known is an assumption, and the one-sample test is only valid if this assumption is correct. All statistical tests involve some assumptions like this, for instance assuming that sampling is not biased. It's your job as the analyst to be clear about what assumptions you're making, to decide whether you think they're justified, and to bear in mind that if the assumptions turn out to be wrong then so do your conclusions.

$\endgroup$
10
  • $\begingroup$ How do you calibrate such a test? What if you have N=1 in group 2? $\endgroup$
    – AdamO
    Nov 14, 2022 at 17:13
  • 1
    $\begingroup$ If you have N = 1 in group 2 then you shouldn't be comfortable treating the the value of 4.0 as known. $\endgroup$
    – Eoin
    Nov 14, 2022 at 17:23
  • 1
    $\begingroup$ All statistical tests involve assumptions. If this assumption can be justified, then a one-sample t test is valid. It's up to the OP, with their domain knowledge, to decide if that's an assumption they're willing to stand by. $\endgroup$
    – Eoin
    Nov 14, 2022 at 17:29
  • 4
    $\begingroup$ Are you asking me to provide an epistemic justification for inductive inference? Because I can't do that in the comments of a question about bean yields. $\endgroup$
    – Eoin
    Nov 14, 2022 at 17:32
  • 1
    $\begingroup$ It is quite common that farmers report estimated or expected or even "standard" yields instead of measured yields. It looks like that is the case here. That would justify the one-sample t-test but it would compare with such a "standard" value and not with actual yields from the fields in group 2. It is important to state that very clearly when presenting results. (And OP should ask farmers or experts, what a standard yield for those beans would be.) $\endgroup$
    – Roland
    Nov 15, 2022 at 10:06
6
$\begingroup$

It requires a bit of explaining how or why this second sample is so "precise" so to speak. Is this rounding error, or are there accidental replications? There is a lot of science going on, not explained, that would inform which option(s) would or would not work. However, it's not a dead-end. Here are 3 suggestions (non-exhaustive) in order of difficulty depending on your chops and software at hand.

  1. Fit a T-test with an equal variance assumption: even if the assumption is clearly violated, you can argue that the test will be conservative. Similarly, you could jitter the response to within a rounding range - i.e. add random noise.

  2. Use a resampling based test: the permutation test is a powerful and flexible means to assess the test of mean difference under the assumption the two samples (under the null) come from the same distribution - note that that distribution would be approximately normal with a "point mass" at X=4.00.

  3. Use a Bayesian approach similar to option 2, it's just a convenient way to calculate the sampling distribution of the mean difference, and make inference correspondingly.

  4. Use expectation maximization: OK so you rounded all the variability out of one sample - generally not a good idea. When you report the result as 4.00 - appreciably we all expect that the result must not be 3.99 or 4.01, so for God's sake, use significant figures, you! Anyway, most of the tests above involve collecting the sample into a single group and assuming that follows some kind of normal distribution - the distribution under the null. Clearly that's not the case, if a value of 4.00 is really 4 and could in fact be 3.50 up to 4.49, then you have to integrate up whatever area of the normal curve of the assumed parametric framework at that iteration to find the true likelihood contribution at that time point. This is a somewhat standard, and most correct, approach to measurement error - which is the specific problem you have here - but is sadly the most technically difficult to implement.

$\endgroup$
7
  • $\begingroup$ I found the RAW data for group 2 and everything is 3.999. So it was rounded but they're still all the same value so no variance. I don't personally believe those figures are accurate, I think the farm has taken the average across all fields and given them the same value. But these are the values I've been given so there's not much else I can do about it other than to just assume they're correct? $\endgroup$
    – S7ewie
    Nov 14, 2022 at 17:56
  • $\begingroup$ @S7ewie are you sure those observations aren't subdivided based on yield, rather than based on area? It's no less perplexing that the yields are all 3.999 than as before when it was 4.00 $\endgroup$
    – AdamO
    Nov 14, 2022 at 18:07
  • $\begingroup$ I can't say for certain as I didn't collect the data myself. I've just working with what I've been given and I've been asked to find out if one group if fields yielded better than the other. In this situation, would it just be better to tell the client that the data isn't accurate enough for statistical testing? $\endgroup$
    – S7ewie
    Nov 14, 2022 at 18:16
  • 4
    $\begingroup$ @S7ewie As a professional, I would say this is a case to call a conference together to go over the data together and be sure everyone understands what's going on. $\endgroup$
    – AdamO
    Nov 14, 2022 at 18:30
  • 1
    $\begingroup$ @dariober relative to the suggestion to use a one sample test treating the group 2 variance as 0, it would be conservative. Following up Alexis' result, the test stat for this approach gives T=-1.73. $\endgroup$
    – AdamO
    Nov 14, 2022 at 18:47
5
$\begingroup$

One solution is to use a one-sample t-test of the null hypothesis:

$$\text{H}_{0}\text{: }\mu_{1} = 4\text{, with H}_{\text{A}}\text{: }\mu_{1} \ne 4$$

If $n_1=26$, $\bar{x}_1=3.865$ and $s_{1} = .24428$, our test statistic for this null would be:

$$t = \frac{\bar{x}_{1}-4}{\frac{s_{1}}{\sqrt{n_{1}}}} = \frac{3.865-4}{\frac{.24428}{\sqrt{26}}} = -2.82$$

At the $\alpha=0.05$ level of significance we would reject the two-sided null hypothesis above, and conclude that we found evidence that Group 1 was sampled from a population with a mean different (less than) than 4.

Alternately, you might try adding a tiny amount of artificial noise to estimate a nearly, but not exactly, zero variance to Group 2 and conduct a two-sample t test assuming different variances (this will produce almost identical results to the one-sample test).

$\endgroup$
1
  • $\begingroup$ "Alternately, you might try adding a tiny amount of artificial noise ... and conduct a two-sample t test" Or do the same without adding artificial noise. It is not necessary for the two-sample t-test. (The Welch's t-test will indeed be the same as a 1 sample t-test if one of the sample variance is equal to zero) $\endgroup$ Nov 15, 2022 at 20:51
2
$\begingroup$

As others have already said, it would be better to address the issue with rounding to get the original data. However, if you have to work with the data you have here's another fairly simple option that may have some advantages over current answers:

Do a t-test under the assumption of equal variance where the variance is calculated using only Group 1. Basically, using the procedure for unequal sample sizes, similar variances but replacing the variance of group 2 ($s^{2}_{X_{2}}$ on the wiki page) with the variance of group 1 ($s^{2}_{X_{1}}$).

You would have to do this by hand but it's should be doable even in e.g. Excel if you don't have much experience with software like R or python.

I think the problem with pre-packaged t-test either assuming equal or unequal variance is that Group 2 is going to bias downwards the estimate of variance (assuming you don't believe Group 2 has 0 variance). Similarly, testing Group 1 against a fixed mean of 4 (like t.test(Group_1, mu=4)) ignores the variance in Group 2.

More sophisticated solutions based on re-sampling, Bayesian statistics, etc (see @AdamO answer) may be better but more difficult to implement.


Sextus Empiricus in comment says:

If group2 has an unexpectedly low variance, then it is arbitrary to just decide to replace it with the variance of group 1

I'd say that eventually every analysis has some arbitrary elements. The question is how you justify them and how they impact the interpretation.

My suggestion assumes that the lack of variation in group 2 is unrealistic and all data points have been reset to 4.00. In such case, I think using the variance from group 1 is more sensible than taking the values in group 2 at face value or, equivalently, testing if group 1 differs from the point value of 4.00. I think it is more reasonable to assume that different farms have more or less the same variance than assuming no variance at all. I guess you could even present results for a range of variances for group 2 (from no variance to the same variance of group 1) and make an informed decision based on that.

In the end it's up to the OP to decide what assumptions are more sensible and make sure that collaborators are aware of those decisions. I would also consider whether it is more costly a false positive (claim farms are different when they are not) or a false negative (farms are indistinguishable when instead they are).

$\endgroup$
5
  • $\begingroup$ "Do a t-test under the assumption of equal variance where the variance is calculated using only Group 1" wouldn't this overestimate the variance? If group2 has an unexpectedly low variance, then it is arbitrary to just decide to replace it with the variance of group 1. $\endgroup$ Nov 15, 2022 at 20:44
  • $\begingroup$ @SextusEmpiricus I edited my answer with some explanation behind my reasoning. $\endgroup$
    – dariober
    Nov 16, 2022 at 8:29
  • 1
    $\begingroup$ "I'd say that eventually every analysis has some arbitrary elements. " This is handwaving. "My suggestion assumes that the lack of variation in group 2 is unrealistic and all data points have been reset to 4.00." if the 4.00 is unrealistic then you must not only distrust the variance being zero but also the mean being 4. Shall we suggest to do the analysis with the mean of group 2 replaced by the mean of group 1? $\endgroup$ Nov 16, 2022 at 8:32
  • $\begingroup$ @SextusEmpiricus Yes, it is handwaving and arbitrary and it is important to communicate that to collaborators. I'm not trying to give the answer to the question but rather an option that sits between the extremes of taking data and results at face value and discarding it altogether. $\endgroup$
    – dariober
    Nov 16, 2022 at 9:00
  • 1
    $\begingroup$ @darioner the problem is that this is not an option that sits in between two extremes. It is an option where we generate artificial data and continue an analysis pretending that it was real data, that is a new different extreme. $\endgroup$ Nov 16, 2022 at 11:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.