Check if a certain procedure is random or not

Question

I am not a statistician, but I have a personal project and I need a help.

I will simplify the project a lot to make it more clear and easier to understand. Then in base of your answers, I can fit it to my real project.

Let's say that I have a pot with many balls inside with a number on the balls.

For a week, every half an hour, I choose a few balls (each time a different number of them) and note down the number on them. Then I repeat the same procedure for many weeks. I will end up with a dataset like this:

Monday 00:00 - [15,45,24,57,437,23,89,2,42,.....]
Monday 00:30 - [12,345,643,64,23,4,64,754,......]
....
Tuesday 00:00 - [24,2,57,865,3,6,8,655,86,.....]
....
Sunday 23:30 - [14,543,64,32,57,43,768,.....]

The higher the number on the ball, the better one. So, I want to check if there is a certain datetime in the week that may have the higher numbers or it is a totally random procedure.

As I said, it is a simplified version. Picking a ball from a pot is certainly random. Maybe a Chi-square is right choice? But can I fit my data to this option?

My first thought was to create a graph (bar graph maybe) where the x-axis would be the datetime of the week and the y-axis the sum or the average of the numbers. But this will end up to another problem. If one day has only mid numbers and another days has a lot of high and low numbers, will end up the same bar. Obviously, it isn't.

May I have your ideas here?

Answer 1

Instead of a bar chart, you could just plot all of your data points on an X,Y (scatter) plot to visually see if you have the kind of distribution you describe.

You could also look at the mean and standard deviation of each time to see if they're similar. Once you've determined whether the spread is the same (or similar enough) for each time, you can use the average or max for each time stamp to decide if the procedure is random.

Answer 2

If you have no reason to believe that there's any correlation between sampling times (i.e. that Monday 00:00 and Monday 00:30 are no more similar than Monday 00:00 and Friday 12:30), I think a simple one-way ANOVA should do the job. This tests the null hypothesis that all groups have the same mean, and if $p \lt .05$, tells you that it's unlikely that your samples are totally random.

However, in my own field I've never come across an ANOVA with so many groups (336 from your example), so I can't honestly say if that's a problem or not.

Answer 3

Ill try to be more clear this time :)

For illustration, lets assume you have a total of 50 balls in your pit.

Lets start with the simple case, where each time you randomize several balls (and bring them back!), and your trying to find out if each ball from the pit has the same probability of getting out of there, no matter what time or day it accrued. In that case there is a classical solution, you have to aggregate all of the results into an array of length 50, and test it as if it came from the multinational distribution (sorry, not negative binomial) that has each cell a P of 0.02, then you could use the chi test for detecting if all balls really have the same probability.

Though you asked for some thing a bit more complicated,you want to know if a certain day / time isn't random, and that makes things a bit more complicated:

1. If you are suspecting that a specific day/ time is not creating proper randomness, Just use aggregated data of all the data from that specific time frame, and use the chi test.

2. If you are suspecting that a specific number is not being pulled by random, keep in mind that each cell (the total aggregation of all outcome per number) is Poisson distributed, there fore you can use the normal approximation and use a z test to see if that number was really random or not.

3. If you want to test all of the day's / time's (i.e you don't suspect that a specific time is non random in advance) then that makes things quit complicated, in that case you're running into the well known multiple testing problem: [http://en.wikipedia.org/wiki/Multiple_comparisons_problem][1]

This problem accrues when you are testing many times the same hypothesis, think of it as if your trying to see if a specific time isn't random for ~5% and you repeat that test many times, you'll eventually find just by random a time frame which will appear non random just by chance. (illustration : if you will test 20 different time frames with 95%, there is a 70% chance of concluding a non random time frame just by chance) In that case, there are a few solutions for it, but the simple one seems to be the Bonferroni correction, lets assume that you have 10 different dates/times, and you want to be sure for 90%, then in that case for each time frame that you'll run a chi test, and set the significance level for each test to 99%. (A better way to do it is through Sidak correction since the samples are independent, but that a bit more complicated)

Hope this answer is clear, ill be happy to respond to further questions.