Has data increased, comparing months in a year

Question

I have data that represents the length of certain events.

I have 13 months of data and want to know if there has been a significant increase over the last year in these lengths.

The first method I can think of seems crude: withhold each month's mean, find the average of the remaining and see if that month's mean value is significantly different from the average of the others (within 2 standard deviations).

oct 9.54
nov 9.77
dec 9.38
jan 9.59
feb 9.59
mar 9.41
apr 10.4
may 10.31
jun 10.33
jul 10.38
aug 10.41
sep 10.44
oct2 10.51

 9.54-average(9.77,9.38,9.59,9.59,9.41,10.4,10.31,10.33,10.38,10.41,10.44,10.51) = -0.05 

2*standard deviation of nov-oct2 = 0.86

0.05 < 0.86 therefore not significant?

If I do this for every month it looks as if none are significantly different. So each month fits in with the year and therefore not an increase.

I then decided I wasn't confident with that method at all and so I did some tests. To start, I did a KS test, a T test and a Wilcoxon test on the two October months given:

Mean 1: 9.54
Mean 2: 10.51
N1: 147770
N2: 138371
Std Dev.1: 8.32
Std Dev.2: 8.07

Results look like:

Two-sample Kolmogorov-Smirnov test

data:  dat$A and dat$B
D = 0.060383, p-value < 2.2e-16
alternative hypothesis: two-sided    

Wilcoxon rank sum test with continuity correction

data:  dat$A and dat$B
W = 9556400000, p-value < 2.2e-16
alternative hypothesis: true location shift is not equal to 0

    Welch Two Sample t-test

data:  dat$A and dat$B
t = -31.585, df = 285780, p-value < 2.2e-16
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -1.0276513 -0.9075632
sample estimates:
mean of x mean of y 
 9.543439 10.511046 

So that's telling me there are (massively) significant differences between the two months.

Also looking graphically there are fluctuations but there seems to be an increase across the year.

EDIT

Each month has roughly 140,000 observations. I can split these by other metrics but thought first to try a method on everything and then to extend it to each group to improve accuracy. I thought I would compare distributions as if the newest month is skewed more towards longer events then that shows an overall average increase in event lengths. I used R for the tests and simply uploaded a csv with two columns for each October as the data.frame dat. I thought I would look at each October as that cancels out seasonality. Each SD is from the vector of ~140,000 obs for each month. I have experience in using multiple regression for financial forecasting so I am open to those techniques. I'm not sure why I would want to make the period longer if I am only interested in this year. I suppose two years is a better choice to compare months with the same seasonality rather than just two Octobers.

Regression attempt using data from 2015, using daily averages rather than monthly and d being the length and using months and years as dummy variables:

Model:  Model 01                            
Dependent Variable:     d                       

Regression Statistics:    Model 10 for d    (12 variables, n=695)                               
    R-Squared   Adj.R-Sqr.  Std.Err.Reg.    Std. Dev.   # Cases # Missing   t(2.50%,682)    Conf. level
    0.198   0.184   0.745   0.824   695 0   1.963   95.0%

Coefficient Estimates:    Model 10 for d    (12 variables, n=695)                               
Variable    Coefficient Std.Err.    t-Stat. P-value Lower95%    Upper95%    Std. Dev.   Std. Coeff.
Constant    9.926   0.143   69.403  0.000   9.645   10.207      
month_EQ_1  -0.212  0.164   -1.290  0.197   -0.535  0.111   0.285   -0.073
month_EQ_10 0.203   0.164   1.233   0.218   -0.120  0.525   0.285   0.070
month_EQ_11 0.263   0.168   1.567   0.118   -0.067  0.593   0.268   0.085
month_EQ_2  -0.283  0.167   -1.696  0.090   -0.610  0.045   0.275   -0.094
month_EQ_3  -0.446  0.164   -2.714  0.007   -0.769  -0.123  0.285   -0.154
month_EQ_4  0.034   0.165   0.203   0.839   -0.291  0.358   0.281   0.011
month_EQ_5  0.004784    0.164   0.029   0.977   -0.318  0.327   0.285   0.002
month_EQ_6  0.123   0.165   0.742   0.458   -0.202  0.447   0.281   0.042
month_EQ_7  0.126   0.164   0.768   0.443   -0.196  0.449   0.285   0.044
month_EQ_8  0.080   0.164   0.485   0.628   -0.243  0.402   0.285   0.028
month_EQ_9  0.111   0.165   0.671   0.502   -0.214  0.435   0.281   0.038
year_EQ_2015    -0.615  0.058   -10.653 0.000   -0.728  -0.501  0.500   -0.373

Now obviously the R-squared is horrific, but I don't care about the model per se. Looking at the negative coefficient for 2015 (-0.615) implies this year there is an increase on last. But how do I know if that increase is deemed significant?

Answer 1

Your methods don't seem directed at your question, but there are several puzzles over what your data are and what you have done.

First off, it seems that what you show are themselves means for each month of larger datasets. Whether that reduction discards useful detail is an open question.

Looking at each month relative to the others makes some sense but there are better ways to do it. I won't comment in detail on your procedure except that it is problematic in various ways.

I can't fully follow your Kolmogorov-Smirnov test, t test or Wilcoxon test.

The first (K-S) is a test of whether distributions differ and has nothing at all to do with testing trend in time.

Without explicit previous steps, your analysis doesn't seem reproducible even by those using the unnamed software you used (quite apart from the apparent reduction to monthly means).

I can't see the logic of comparing just the two Octobers while getting the standard deviations ... from where??? Looking for trend can't validly focus just on beginning and end periods. The fallacy lurking here is seen in its strongest form by imagining a test comparing the heights of very tall and very short people selected from a larger group.

A simple method that should be relevant is just to plot the monthly data (or means) and check for a trend. Conventional linear regression results lead to rejecting a null hypothesis at conventional levels and so supports the idea of an increase. But the $R^2$ (e.g.) is inflated by just looking at means, as within-month variation is suppressed, whereas conversely, almost any trend would be deemed significant if the raw data were analysed and the true sample size is much larger than 13.

It's not at all obvious that a straight-line fit is quite right, but without any ideas on the process here or a longer dataset, it seems prudent just to stop there, and let a graph tell us that there's an increase and that a straight line trend is a coarse summary. There has to be a higher-level reservation about dependence in time of the errors and what is being assumed.

Some might want to direct you to time series analysis here, and in principle they have a good case, but my prejudice with data like yours is that time series analysis would make a simple problem more complicated and that you have far too short a dataset to fit anything more than very simple summaries. If this is just a token problem and you have, or will have, much more data, then the balance tips in favour of that approach. However, time series analysis itself might need to give daily or monthly summaries if the raw data are unevenly spaced in time.

Somehow we seem to be assuming that season of year is irrelevant.