Let's suppose you have been given some data from a randomized block design with 4 repetitions and 23 treatments. After an initial inspection of the data, you notice that for 8 treatments all repetitions are identical, which is obviously wrong. After reporting the problem, you are told that it was due to a confusion from the person responsible for the data, who will later send you the "correct version" of the data. The corrected version of the data would be something like this:
Treatment Rep Value A 1 5727.000 A 2 5400.000 A 3 5800.000 A 4 5473.000 B 1 4618.000 B 2 4844.000 B 3 4966.000 B 4 4496.000 ... Z 1 4329.345 Z 2 4597.275 Z 3 4833.246 Z 4 4199.098
The first thing that would grab my attention in such data would be the fact that only the eight treatments for which the problem was reported did not have any decimal fraction (all remaining treatments being fine). So I would decide to give them a closer look and subtract each observation from its sample mean within treatments, finding something like
Treatment Rep Value Delta A 1 5727.000 +127 A 2 5400.000 -200 A 3 5800.000 +200 A 4 5473.000 -127 B 1 4618.000 -113 B 2 4844.000 +113 B 3 4966.000 +235 B 4 4496.000 -235 ... Z 1 4329.345 ... Z 2 4597.275 ... Z 3 4833.246 ... Z 4 4199.098 ...
After seeing that the differences are symmetrical around the mean, I would immediately call the person responsible for the project, and report the problem. Of course, I would also give up working on that project.
Although the evidence is quite compelling, it would be good to attach a probability to the report, just to give an idea of how bad that data looks. So I would have thought of something like the following and I would like to know if there would be any flaw in my reasoning:
Let's say that if the data was legitimate, it would be reasonable to assume normality for that kind of data, based on experience from previous analysis of that kind of data.
So, let's define 4 i.i.d. normal random variables for each of the four repetitions within each treatment: $$X_i\sim N(\mu_X,~\sigma_X); ~~~i = 1, 2, 3, 4$$ The symmetry (with a tolerance of .5) observed above could be expressed as the event: $$A: -0.5 < X_1 + X_2 - X_3 - X_4 < 0.5$$ There are still other equivalent ways in which the samples could be arranged to satisfy the symmetry (X1 + X3 - X4 - X2; X1 + X4 - X2 - X3) so the probability of the symmetry (S) would be: $$P(S) \leq 3 * P(A)$$
The inequality is simply because I don't want to remove the intersection.
If we define the random variable Y as follows: $$Y = X_1 + X_2 - X_3 - X_4$$
It follows from that: $$Y\sim N(0,~2*\sigma_X)$$
Let's say that from the residuals of the model with that data, I estimated the $\sigma_X$ as 350. From that, I would use the CDF of y to calculate the probability of Y falling between -0.5 and 0.5, which would be P(A) = 0.0005699175
pnorm(0.5, sd = 700, lower = TRUE) - pnorm(-0.5, sd = 700, lower = TRUE)
As such, the probability of S would be:
$$P(S) \leq 0.001709752$$
Because there would be no clear block effect and the data would come from a randomized experiment, it would be reasonable to assume statistical independence. Let's suppose that out of 8 suspect treatments, 3 had this symmetry. Then, assuming independence, we could calculate the probability of such event (let's call it D) from a binomial distribution:
D: 3 out of 8 treatments have symmetry of observations around the sample mean.
$$ P(D) \leq {8\choose3} \cdot p^3(1-p)^5$$
$$ P(D) \leq 2.7 \cdot 10^{-7}$$
I am not a statistician, so I would like to know if there is any flaw in that reasoning and whether you would also report the data as fraudulent.
