For an ecology project, my lab group added vinegar to 4 tanks containing equal volumes of pond water, 1 control with no elodea (an aquatic plant) and 3 treatments with the same amount of elodea in each. The purpose of adding the vinegar was to reduce the pH. The hypothesis was that the tanks with elodea would go back to their normal pH quicker. This was indeed the case. We measured the pH of each tank daily for about two weeks. All the tanks eventually returned to their natural pH, but the length of time that this took was much shorter for the tanks with elodea.
When we told our professor about our experimental design, he said that there exists no statistical test that can be performed on the data to compare the control to the treatment. That because there was no replicate for the control (we only used one control tank) we cannot calculate variance and so we can't compare the sample means of the control and the treatment. So my question is, is this true? I definitely understand what he means. For example, if you took the height of one man and one woman, you can't draw conclusions about their respective populations. But we did 3 treatments, and the variance was small. It seems reasonable to assume that the variance would be similar in the control?
Thank you for the excellent answer. We got more water and elodea from the wetland and decided we would run the experiment again with smaller tanks but this time with 5 controls and 5 treatments. We were going to combine this with our original data but the starting pH of the tanks was different enough that it doesn't seem valid to consider the new experiment to be sampled from the same population as the original experiment.
We considered adding different amounts of elodea and trying to correlate speed of pH remediation (measured as time elapsed until pH returned to its original value) with amount of elodea, but we decided that wasn't necessary. Our objective is only to show that the elodea makes a positive difference, not to construct some kind of predictive model for exactly how the pH responds to differing amounts of elodea. It would be interesting to determine the optimal amount of elodea, but that's probably just the maximum amount that can survive. Trying to fit a regression curve to the data wouldn't be especially illuminating because of the various complicated changes that occur to the community when adding a large amount. The elodea dies, decomposes, new organisms start to dominate, and so on.