Let's say you want to compare the means of 25 groups and you also want to consider the confidence interval of the mean. There is an issue because some group will have higher observations than other groups . Below is some sample data that represents 5 of the 25 groups. You can see for group D the SEM is large so it has a high mean which in this case is good but it is highly uncertain.
I was going to choose a level of the standard error to make the comparison. I was going to choose ONLY groups with a SE below .03 or .04 to compare. And for groups with SE > .03 o r.04 I was going to say I need more observations.
I was wondering if there are any research articles you can point me to that talk about choosing the correct threshold for standard error level?
I am choosing .03 or .04 but should I look at only SEs withing the interquartile range of SEs or withing 2 SDs of the SEs?
Are there rules of thumb or papers that cover making comparisons like this?
group = c("A","B","C","D","E") mu= c(.5,.54,.53,.6,.52) StandardError = c(.01,.03,.04,.09,.025) upperCL = mu +1.96*StandardError lowerCL = mu -1.96*StandardError dat = data.frame(group = group, mean = mu , StandardError,upperCL,lowerCL) group mean StandardError upperCL lowerCL 1 A 0.50 0.010 0.5196 0.4804 2 B 0.54 0.030 0.5988 0.4812 3 C 0.53 0.040 0.6084 0.4516 4 D 0.60 0.090 0.7764 0.4236 5 E 0.52 0.025 0.5690 0.4710
library(ggplot2) ggplot(dat,aes(x = group, y = mean, group = 1))+geom_point()+ geom_errorbar(aes(ymax =as.numeric(upperCL),ymin = as.numeric(lowerCL), na.rm = TRUE))