Unusally high F value ANOVA

Question

I am running a one-way ANOVA to compare a numeric continuous variable between four groups, and getting a very high F-value. Below are the descriptives and code I used in R (I get similar values when trying a different package in Python):

> oneway.test(ideology_score ~ region, data = mydata)
   One-way analysis of means (not assuming equal variances)

data:  ideology_score and region
F = 1270.6, num df = 3, denom df = 101578, p-value < 2.2e-16

Total N of observations = 262,267

Group	Mean	SD	N
Group1	-0.43257859	1.285637	80,136
Group2	-0.13539030	1.355517	26,915
Group3	-0.08337834	1.362262	85,512
Group4	-0.39797591	1.263706	69,704

The means and standard deviations of the variable are almost the same for each group, and the variance and distribution of scores within each group are quite similar as well.

Is there anything else that may be going wrong here to explain the high F value?

Answer 1

With sample sizes as large as those you will almost always find that the statistical test gives very low p-values. Divide those SD values by the square root of the sample sizes to see the estimated SD of the means (i.e. the SEM) and you will see that they are tiny compared to the various differences between the means. That gives you the high F-value.

You need to decide on a way to deal with the fact that statistical significance does not correspond to real-world significance. What you need to decide is whether a difference between, for example, -0.43 and -0.13 is of any importance. It might be trivial, or not.