1
$\begingroup$

I have data in the following format:

Concentration speed height distance
control 1.5 3.1 13.1
0.1 1.9 5.4 13.5
0.2 2.3 9 16.7
0.4 2.7 13.1 21.2

I am confused as to what test I should use to see if there is statistical significance in terms of speed, height, and distance as the concentration is increased. Initially I thought ANOVA was the answer as there is one categorical independent variable (the concentration, with at least three levels), and one or more quantitative dependent variables (three different ANOVAs, one for each column).

I have another set of values with the same parameters and want to compare them with this initial set too and was also wondering what test would be appropriate. The two-way ANOVA seemed to fit as I will be comparing concentration and time but I cannot get the one way ANOVA to work in the first problem.

$\endgroup$
2
  • $\begingroup$ Welcome to Cross Validated! What goes wrong when you fit the one-way ANOVA? // You could analyze each of your three variables (speed, height, and distance) separately, but you also could do a multivariate analysis of all three at once (which is what I would hope you would get if you hired a consulting statistician). Such an approach will be more complicated, but it’s also can be more powerful. // It turns out that two-sample t-testing is a special case of ANOVA, and that ANOVA is a special case of linear regression! (It’s fun when that idea (especially with generalizations) finally clicks.) $\endgroup$
    – Dave
    Apr 17 at 10:46
  • $\begingroup$ Thank you for your response Dave it is very helpful. Attempting to fit one-way ANOVA using only the speed column gives me a sum of squares of 4.136, 7 degrees of freedom, mean squares of 0.59, F ratio of 7.624, and finally a significant p-value less than 0.005. From what I understand so far now I should repeat this for other two columns. This brings to me the question of it now I should use two-way ANOVA (mixed) to compare speed column from set 1 to speed column of set 2. $\endgroup$ Apr 17 at 11:31

1 Answer 1

4
$\begingroup$

It is not a good goal to "see if there is statistical significance". There are many problems with the idea of "statistical significance", chief among them being dichotomous thinking that once you cross a threshold all of a sudden something is "significant". Better to make a goal of quantifying to what degree X is related to Y.

Your particular problem is that of a single X against a multivariate Y. Multivariate methods make a lot of assumptions, so it is good to use an approach advocated by Peter O'Brien in which you turn the problem around to try to predict the concentration simultaneously from the three response variables. Consider concentration as ordinal and use an ordinal regression model. From that you'll get a measure of rank correlation, a statistical test of the global null hypothesis of no association (likelihood ratio $\chi^2$ test with 3 degrees of freedom) and more.

$\endgroup$
4
  • $\begingroup$ Thank you for your reply Frank it is very helpful. I have used an ordinal regression model on the data and now I am struggling to interpret these results. I obtained a coefficient and p-value for each of the three features along with a coefficient and p-value for each concentration. $\endgroup$ Apr 18 at 7:28
  • $\begingroup$ Start with the overall model likelihood ratio $\chi^2$ statistic. It tests for whether any of the variables is associated with concentration. Then make partial effect plots to show how the probability of concentration $> y$ varies with each of the three variables holding the other two constant, for $y$ of your choosing. $\endgroup$ Apr 18 at 11:09
  • $\begingroup$ The rank correlation would be Spearman’s rank correlation applied to actual concentration vs predicted concentration? If so, you would classify a predicted concentration as the one with the highest modelled probability? $\endgroup$ Apr 18 at 11:23
  • 1
    $\begingroup$ Yes Spearman's $\rho$ is between predicted and observed. But "predicted" doesn't require predicting actual categories; it is a function of predicted exceedance probabilitiies that is much simpler: the linear predictor $X\hat{\beta}$. $\endgroup$ Apr 18 at 11:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.