3
$\begingroup$

I've been tracking data and I am looking to see if it is truly uniformly random. The scenario is there can be a grid of 35 colour tiles with 5 different colours (Blue, Green, Purple, Red and Yellow). So in theory over time you should see an average of 7 tiles per each colour. I have collected a sample size of 112 grids and here is the average of each colour:

B - 7.053571 G - 7.098214 P - 6.633929 R - 7.223214 Y - 6.991071

Is the sample size large enough? I'm curious to why purple is coming up so low but need to know if statistically if it is random or not?

Any help would be greatly appreciated.

Regards, Paul

$\endgroup$
4
  • $\begingroup$ I assume "random" here means "uniform"? It could still be perfectly random but with different probabilities for different colors $\endgroup$
    – jld
    Jul 12 '18 at 19:35
  • $\begingroup$ Sorry yes uniform. The theory is that we should average 7 tiles per colour over time. Just don't know how big of a sample I would need to prove that this holds true? Sorry I don't have a statistics background so I'm stuck here on how to proceed. $\endgroup$
    – Paul
    Jul 12 '18 at 19:43
  • 2
    $\begingroup$ Do you have the total counts (i.e. how many times each color appeared)? You could do a $\chi^2$-test to test deviations from uniformity. $\endgroup$ Jul 12 '18 at 20:19
  • $\begingroup$ Would any of this change if I believed that Purple was always going to be the outlier? I have setup the experiment thinking that Purple is going to be less. Does everything remain the same? Or if you are trying to prove just that does that change anything? $\endgroup$
    – Paul
    Jul 13 '18 at 16:16
5
$\begingroup$

You have seen 112*35 = 3920 colors. The expected frequency of each color is then 784. You can use a Chi-square test to see if the colors are randomly distributed in your sample.

You saw 790 Blue, 795 Green, 743 Purple, 809 Red and 783 Yellow. To calculate your Chi-Square statistic you sum the squared difference between the observed frequencies and the expected frequencies and divide it by the expected frequency.

So, Chi-Square = (36 + 121 + 1681 + 625 + 1)/784 = 3.1429

We have 4 degrees of freedom (number of colors - 1).

Using a Chi-Square table we get a value of 0.5342

This means that if the colors were randomly distributed then then the probability of us receiving a Chi-Square statistic as large or larger than the one we did would be 53.42%

If the null hypothesis is that the colours are randomly distributed, then your p-value for this test would be 0.5342 which means you would nprobably not reject the null hypothesis.

Just to add, in terms of sample size, your sample is plenty big enough. A rule of thumb that I have read is that the expected frequency should not be less than 5, which clearly at 784 we are larger than.

$\endgroup$
3
  • $\begingroup$ Thank you for this. Sorry I'm not 100% sure I understand what the 53.42% means? $\endgroup$
    – Paul
    Jul 12 '18 at 21:53
  • 1
    $\begingroup$ Thank you for all your help on this. Just out of curiosity what would the p-value need to be before you could reject the null hypothesis? $\endgroup$
    – Paul
    Jul 12 '18 at 22:22
  • $\begingroup$ In most fields, 0.05 is considered the threshold at which you say "There is evidence to reject the null hypothesis" - it's the point at which results at least as unusual as yours would happen 1 time in 20. $\endgroup$
    – ConMan
    Jul 13 '18 at 0:19
4
$\begingroup$

In addition to the chi-squared test as described in @astel's answer, you could also do this by simulation.

For $j = 1,\dots, N$ where $N$ is a big number (say $10000$) I'll simulate $$ X_{ij} \stackrel{\text{iid}}\sim \text{Multinomial}(35, (1/5, \dots, 1/5)) $$ for $i = 1,\dots,112$. Each $X_{ij}$ represents a single grid with 35 cells made of 5 colors. I'm simulating these with each color having equal probability because I want to see what I'd expect under the null distribution, which in this case is exactly that, i.e. $p_0 = (1/5,\dots,1/5)$. From this I'll obtain $$ T_j = \frac 1{112}\sum_{i=1}^{112} X_{ij} $$ so for each simulated data set of 112 35-cell grids, I'll get a single quintuple $T_j$ giving the average number of each color that appeared in the 112 grids.

You are concerned that your observed average, I'll call it $t$, is suspiciously far from the expected value under the null of $(7,7,7,7,7)$. I can therefore use my simulations to exactly test this: let $\mu_0 = (7,7,7,7,7)$, and for each $T_j$, compute $$ D_j = \|T_j - \mu_0\| = \sqrt{\sum_{k=1}^5 (T_{jk} - 7)^2} $$ (I'm using "$D$" to stand for "distance"). Thus I'll get $N$ simulated values of the Euclidean distance of the average number of each color from the expected number $\mu_0$ under the null hypothesis. I can then compute $d = \|t - \mu_0\|$ and see how that compares.

I did this in R and here's the result:

D.sim

The red line is your observed value of this statistic. Just as a quick check, this value is $$ \sqrt{(7.053571 - 7)^2 + (7.098214 - 7)^2 +(6.633929 - 7)^2 +(7.223214 - 7)^2 +(6.991071 - 7)^2} \approx 0.443 $$ just as shown in the figure. And that is a completely typical result under the null distribution so I don't see any evidence that your observed $t$ is suspiciously far from the expected one in terms of the Euclidean norm.

Here's the code to do this:

g <- 35  # grid size
k <- 5 # number of colors
p.null <- rep(1/k, k)  # prob of each color under H_0; uniform
n <- 112  # number of grids you observe
mu.null <- rep(7, k)

nsim <- 1e4 # number of simulated data sets I'll make

# simulating the T_j
set.seed(132)
T.sim <- t(sapply(1:nsim, function(i) rowMeans(rmultinom(n, g, p.null))))

# computing the distances D_j
D.sim <- apply(T.sim, 1, function(t.curr) sqrt(sum((t.curr - mu.null)^2)))

t.obs <- c(7.053571, 7.098214, 6.633929, 7.223214, 6.991071)
d.obs <- sqrt(sum((t.obs - mu.null)^2))

# plotting all of the simulations and comparing to d.obs
hist(D.sim, 50)
abline(v=d.obs, col="red", lwd=2)

There are tons of other things you could do here, like consider different norms, look at each coordinate by itself, or try more analytical methods such as the chi-squared test suggested in @astel's answer. But I like simulation because it's nice and simple, and I find it easier to do simulation-based approaches when I'm trying to reason about a problem that I don't know how to solve in advance.


As for your concern about sample size, you need to be more clear about what you want to detect. If you say that you want to detect any difference, then there is a sample size sufficient to do that with high probability but it could be astronomical. Suppose in truth the probability of a purple square is not $0.2$ as it would be under $H_0$ but instead is $0.1999999$. As $n\to\infty$ you will eventually reject with high probability but the required sample size is going to be enormous.

The point of this is to say that when you ask "is the sample size big enough", the only real answer is "for what?".

To emphasize this point, I did another simulation. I'm allowing $n$ to vary so I'll be simulating $X_{ij}^{(n)}$ and I'll get $N$ $D_j^{(n)}$ for each $n$ under consideration. I'll take $\Delta^{(n)} = E(D_j^{(n)})$, and for each $n$ I'll obtain a 95% quantile-based confidence interval for $\Delta^{(n)}$ via my simulation.

I did this for $n = 50,100,150,\dots,900,950,1000$ so you can see how the confidence interval at 95% changes as $n$ increases.

n

Again, the red line is your observed value.

So for the particular $t$ you observed, even though with $n=112$ we saw no evidence of there being a meaningful deviation, we'd start rejecting this once $n \approx 400$. But the point of this is that that's not the right way to look at this. If this is a real situation you know the probabilities aren't EXACTLY $(1/5,\dots,1/5)$. Instead, you want to make sure you reject when the difference is big enough to matter, which requires using domain knowledge to decide what "big enough to matter" means here.

Here's the code for that plot:

nseq <- seq(50, 1000, by=50)
cis <- matrix(0,length(nseq), 2)

for(i in 1:length(nseq)) {
  T.sim.n <- t(sapply(1:nsim, function(s) rowMeans(rmultinom(nseq[i], g, p.null))))

  # computing the distances D_j
  D.sim.n <- apply(T.sim.n, 1, function(t.curr) sqrt(sum((t.curr - mu.null)^2)))
  cis[i,] <- quantile(D.sim.n, c(.025,.975))
}

plot(cis[1,] ~ c(nseq[1], nseq[1]), type="l", ylim=range(cis), xlim = range(nseq),
     main="Confidence intervals as n increases", xlab="n", ylab="Euclidean Distance")
for(i in 2:length(nseq)) {
  lines(cis[i,] ~ c(nseq[i], nseq[i]))
}
abline(h = d.obs, lwd=2,col=2)
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.