# Hypothesis testing with very few data

My biologist friend is having a bad time with her experiment. It's suspected that changing the temperature and organic content in water may affect the growth of crustaceans. In her experiment, two formulations of different organic content combined with four different temperatures (plus control) were combined. 10 tiny crustaceans were put to grow in each combination and weighed weeks later. However, the crustaceans are too small to be weighed individually, so they were all weighed together and the result divided by 10. The experiment generated the following table:

          Formulation 1    Formulation 2
Temp A        0.132            0.155
Temp B        0.136            0.143
Temp C        0.141            0.139
Temp D        0.128            0.132
Control       0.123            0.138


Now she'd like to find out whether any combination of Temperature x Formulation affects the crustaceans' weight considerably compared to the control groups. In my opinion, the best way to detect whether the Temperatures or Fractions affect the group's weight is via two-way ANOVA without replication, which gives the following:

Analysis of Variance Table

Response: weight
Df    Sum Sq   Mean Sq F value  Pr(>F)
temperature   4  0.0002966 7.415e-05  1.5561 0.33940
formulation   1  0.0002209 2.209e-04  4.6359 0.09764 .
Residuals     4  0.0001906 4.765e-05


But I have a few questions: how reliable such statistic would be? Is this really the correct way to detect the effect of each combination in each group (in this case, that there's no effect whatsoever)?

Or would it be necessary to re-run the experiment with replication for a proper result? In this case, it's gonna be difficult since these crustaceans are quite expensive.

• Because your ANOVA makes no distinction between treatment and controls, it does not appear to address the scientific question on the table. Because temperatures have a natural order, and that order can be expected on general principles to influence growth, what exactly are the temperatures involved here (including that of the control)? – whuber Jan 24 '14 at 16:29
• But since the test suggests that the null hypothesis can't be rejected, doesn't it mean these combinations are as uneffective as control? The temperatures are A=10°C, B=15°C, C=25°C, D=30°C, and Control=20°C. – soldeace Jan 24 '14 at 18:03
• Since the treatments are evenly spaced temperatures, you should probably treat it not as unordered categories but either as an interval scale or at worst, ordered categories. – Glen_b -Reinstate Monica Jan 25 '14 at 4:07
• To expand on my previous comment - with such small numbers, my first thought would be to fit a linear regression model against temperature (in C. not as categories), and to write the null in terms of the formulation and formulation x temperature effects. (With more data I'd include a quadratic term in temperature plus the interaction to accommodate nonlinear response, but here you don't really have the d.f. to support it; if it's an absolute necessity you might do it) – Glen_b -Reinstate Monica Jan 26 '14 at 23:04
• Thank you for the suggestion, Glen. In the end, it seems like she should re-run a better planed experiment. – soldeace Jan 27 '14 at 12:01