# What is the correct way to test if there are any statistically significant differences between values on some variable between any two years?

Let's say I have the murder rate for a large number of years. What would the correct way to test to see if there are any statistically significant differences between these rates for any two years?

My initial instinct was to run t-tests for all possible permutations of years, which seems computationally intensive and might also yield incorrect estimates. I considered a linear regression with each of the years as dummies, but I'm not sure if this is testing what I need it to be testing. Is an ANOVA appropriate? An F-test?

• it is not fully clear to me if you want to test if a specific couple of years is different, or if all the year are not equal Apr 22, 2016 at 8:42
• Have a look to stats.stackexchange.com/questions/199008/… Apr 22, 2016 at 9:07
• Let me be clear: Imagine I want to conduct the following sequence of tests. First, generate all possible two-way permutations of years from 1990 to 2005 (1990 & 1991, 1990 & 1992, and so on). Then, essentially run t-tests to see if there are mean differences in the crime rate for those two years. If there are ANY year combinations for which there is a significant difference, then we have failed the test. Apr 22, 2016 at 16:03
• t-test is for normal distributed variables, so it depends if you can assume that. If you repeat multiple times your test comparing different couples to say something about the full set of years you need to study the trial factor (look elsewhere effect). Apr 22, 2016 at 17:35

If the number of murders per year is large enough you can suppose that the distribution of your random variable $X = \text{number of murders}$ is Poissonian distributed $X\sim\text{Pois}(\lambda)$.

Having many observation, one for every year, you have a set of random variables $X_1\sim\text{Pois}(\lambda_1),\ldots, X_n\sim\text{Pois}(\lambda_n)$.

You want to test if there is a significant difference between $X_i$ and $X_j$ knowing the observation $x_i$ and $x_j$. So define the null hypothesis as $\lambda_i = \lambda_j$.

We have define the statistical model. Now you have to define your test-statistics. One natural choice is $x_i - x_j$. This is the difference between two Poissonian distribution, and it is know to be the Skellam distribution with pmf $f_{X_i-X_j}$. I guess you want to do a two-side test since you are interested in the difference in both directions. You just have to compute the p-value, since the Skellam distribution is symmetric under the null hypothesis:

$$\text{p-value} = 2\sum_{x\geq |x_i-x_j|} f_{X_i-X_j}(x, \lambda, \lambda)$$

One thing missing here is the value of $\lambda=\lambda_i=\lambda_j$, I guess you can just take the mean of $x_i$ and $x_j$.

Alternatively you can use a more general approach and use the profiled likelihood ratio.

$$\Lambda = -2 \log\frac{\sup_\lambda\text{Pois}(x_i|\lambda)\text{Pois}(x_j|\lambda)}{\sup_{\lambda,\Delta_1,\Delta_2}\text{Pois}(x_i|\lambda+\Delta_i)\text{Pois}(x_2|\lambda+\Delta_j)}$$

and then compute the p-value, using the fact that, asymptotically, $\Lambda$ is distributed as a $\chi^2$-distribution with 1 degree of freedom. After some math you can simplify the expression:

$$\Lambda = -2(x_i + x_j)\log\left(\frac{x_i + x_j}{2}\right) + 2x_i \log x_i + 2 x_j \log x_j$$

Here the histogram for $\Lambda$ generating 10k pseudo-experiment under the null-hypothesis

With these tests I am ignoring the information from the correlation between the years, but to take into account them you need a model describing the behaviour of the murders across the years.

• Hmm. It will take me a while to process this. Does it matter that X does not equal to the NUMBER of murders per year, but the murder rate? Also wondering why it isn't valid to run multiple t-tests or a regression? Apr 22, 2016 at 0:49
• what is the difference between "number of murder per year" and "murder rate"? $t$-test if for the difference of the means of two normal distribution, maybe you can use it. Apr 22, 2016 at 0:55