# Results of 2-sample t-test does not seem intuitive; wrong procedure?

I have data shown below, with a response variable tabulated by hour (x-axis) as well as numerous other factors: larger chart available here

I want to see if the colored points (red, light and dark blue, orange) are statistically different from the grey points. Upon visual inspection it seems as though the colored points differ from the grey points at hours 6 and 7, so I tried to conduct a 2-sample t-test of colored points vs grey points for each of the 24 hours separately, and got the following results:

    Hour           p-value for 2-sample t test
[1] "00"             "0.226638013124"
[1] "01"             "0.691245588294"
[1] "02"             "0.840988947432"
[1] "03"             "0.147960817870"
[1] "04"             "0.673288043713"
[1] "05"             "0.0000640211129743"
[1] "06"             "0.000186641391119"
[1] "07"             "0.000417492339098"
[1] "08"             "0.000111826661499"
[1] "09"             "0.0000426931229112"
[1] "10"             "0.000274211619809"
[1] "11"             "0.0000342302853781"
[1] "12"             "0.00357160058606"
[1] "13"             "0.00108203195091"
[1] "14"             "0.00695746584398"
[1] "15"             "0.134903506946"
[1] "16"             "0.0792616667511"
[1] "17"             "0.258782638906"
[1] "18"             "0.265273732241"
[1] "19"             "0.0615368690249"
[1] "20"             "0.0738850389549"
[1] "21"             "0.215317462846"
[1] "22"             "0.0531097020924"
[1] "23"             "0.0344662311452"


The fact that many of these p-values are so low surprise me, given the points on the chart. Even with a simple Bonferroni correction for multiple comparisons, hours 5,6,7,8,9,10,11,13 and 14 are significant with a p-value < .05/24.

I can understand why this is so (large number of observations, small standard deviation), but intuitively it does not look right, especially for hours 9,10,11,13 and 14. I just wanted to check with people who are more knowledgeable to see if there is anything I am doing wrong or anything I have missed out. Is a 2-sample t-test on each hour the wrong procedure? Assuming the data is valid, would you be comfortable concluding that hours 9,10,11,13 and 14 are significant?

Any help would be greatly appreciated; I am more comfortable with data visualization than I am with statistics.

Thanks!

EDIT: added boxplot for the relevant hours. Using a 2-sample t-test with Bonferroni correction for multiple comparisons, all of the hours below show significant differences.

• Also, I have checked the hourly data for autocorrelation and found nothing significant. For the 2-sample t-test, I am also using the Welch approximation for degrees of freedom to avoid assuming equal variances. Apr 28, 2015 at 22:54
• Your plot makes it impossible to distinguish point colors reliably. Why don't you make some sets of side-by-side boxplots for each hour so you can see what's going on in your data?
– whuber
Apr 28, 2015 at 23:01
• You seem to have a pretty large data set, so almost any difference will be shown to be statistically significant. What are the means and standard deviations of the different groups? Also, why do you lump together the different colored dots? I look at the plot and it seems that in each of the hours after 12 there is an orange dot near the low end of the observations. This looks like outliers to me, and a t-test is susceptible to outliers (by contaminating both the mean and standard deviation). Apr 28, 2015 at 23:36
• @whuber, done. as you can see I have misgivings about hours 8-14. Apr 29, 2015 at 0:02
• @user3697176, I have about 250 points for each hour. I just added a boxplot, as you can see I have misgivings about hours 8-14. The data actually shows occupancy at each hour over different days, and the color represents adverse weather conditions (rain=blue, orange/red=snow). There aren't enough red/orange points for me to comfortably work with so I lumped it in with the blue (rain), but even if I omit the red/orange points from the sample I will still face the same issue (how to determine statistical significance with a large n since sd will be tiny). Apr 29, 2015 at 0:08