I recently did flow cytometry to quantify cell numbers on 4 similar samples from one group and 4 from another different group (n=4). The samples each contained millions of cells and I wanted to compare the cell populations for a difference. For simplicity let's say n=1 million (mixed cells). For specific cell types in that mix, I created a contingency table and tested for a difference between mean cell proportions in the sample. I used the Chi squared test for a difference between two different non-parametric samples.
So for example, if I had a mean of 10,000 type A cells in group 1 and 1,000 type A cells in group 2, I also had 990,000 other cells in group 1 and 999,000 other cell types in group 2.
So the p value with these large numbers of cells comes out tiny.
My question is, if I only sampled 4 individuals, but then ran my stats tests against very large numbers of something contained within those 4 individuals, is this valid? To take it to its extreme, I could have compared 1 individual against 1 other (n=1) and still have done stats tests on cell numbers which would seem very high powered, but it's only powered for those two individuals. How do I describe this? Can/should I do the test at all? If not, then what n number of individuals do I need before I can say this is valid (given that when I test the cells I will have very high n numbers and no issue with the tests being underpowered)?
Also, is the chi squared test the correct test (I assumed the data are not normally distributed, but should I assume normality, or test for it, and then potentially run a t test)?