I am asked to check if a categorical distribution with $3$ variables is uniform, which means each variable has $\frac{1}{3}$ probability in the real population. (Required significance level: $0.01$)
Lets say I have a dataset sample of the real population with $1000$ people, and I have a column in my dataframe that represents the monetary status of a person with three categorical variables (poor, moderate, rich).
My Work:
Null Hypothesis: The sample distribution is uniform.
Alternate hypothesis: The sample distribution is not uniform.
Test statistic: The Total Variation distance between the distribution of the sample and the uniform distribution.
In other words, using this formula for TVD: $\frac{|\sum_{i=1}^3p_i-q_i|}{2}$ (where $p_i$, and $q_i$ are probabilities of each categorical variable in each of the two samples).
Here I have $\frac{|\sum_{i=1}^3p_i-\frac{1}{3}|}{2}$. (Because the other sample has uniform distribution, or the Model assuming the null hypothesis is true).
Now, I started sampling from my Model dataset assuming the null hypothesis is true (uniform), decided to take sample size of $500$ (didn't really think of it too much).
For each of those samples, I calculated the TVD from the uniform distribution as described above.
Then I plotted the empirical distribution of the TVDs (the test statistic), which was between $0.00$ to $0.09$ (not exactly but close enough).
Drew a red dot on my graph of the TVD from my dataset and it was at $0.38$.
Calculated my p-value, and of course, none of the samples TVD's were even close to $0.38$, so I got p-value=0 exactly. And based on that I rejected the null hypothesis, and said that the distribution of the monetary status of a person in the population is most likely Not uniform.
Questions:
- Is getting a p-value=$0$ exactly weird? should that make me worry that what I did was wrong? (because that's exactly why I'm here).
- In the end, I wrote most likely not uniform in my conclusion, is that a formal way to write my results? if not, how would I formalize it more?
- Does the idea of what I did and the steps seem logical and alright? (Because alot of my friends didn't reject the null hypothesis, so I'm hesitating about my answer).
- When I checked the probabilities in my dataset I got $(0.713,0.179, 0.108)$, that makes me calm down a little, because it seems far away from $(\frac{1}{3},\frac{1}{3},\frac{1}{3})$, am I supposed to feel like that? or it doesn't matter since we're requiring significance level of $0.01$ so it might still be true?
I would appreciate any help or feedback, sorry for making this too long, just wanted to clear everything I did.
Thanks in advance to everyone!