0
$\begingroup$

Suppose one wants to test whether the average sum of tickets in a box is X.

Since the z-score is computed by sampling N tickets and then seeing how many SE's away the average is from the expected value. One would then use this z-score to compute a p-value, and based on that they will decide if they can reject the null hypothesis.

Why isn't this process repeated many times? It would seem more sensible to obtain more z-scores and then get a more "holistic" p-value out of that.

$\endgroup$

2 Answers 2

2
$\begingroup$

I lack context for your question, but I don't see where a repeated drawing would be beneficial.

One, if the process can be repeated many times then maybe it's even possible to get all the tickets and obtain the real average of all tickets in that box.

Two, if you can only repeat the process M times where M would not exhaust all N tickets from the box - you could simply increase your sample size and draw N*M number of tickets one time instead of getting M averages.

In other words, in order to justly compare the two approaches we have to compare them on equal grounds. In the question you are comparing one approach that is allowed to draw N tickets, with another that is allowed to draw N*M tickets. To have a fair comparison we have to make the conditions equal.

$\endgroup$
0
$\begingroup$

The whole reason that z values and p values exist is because they allow us to "simulate" the process of drawing repeated samples without actually doing it. Clearly the underlying logic of frequentism, which drives the interpretation of p values is based on the idea of repeatedly sampling from a population. So you are right that it might be more "sensible" to actually draw multiple samples instead of just one. But in the real world of research, this is rarely possible, which is precisely why we developed the idea of p values in the first place.

When public opinion researchers do a poll of 1,000 people, or virologists run an RCT on a new vaccine among 10,000 people, or (to adapt the example that led to the creation of the t test) beer brewers sample a small bit of a new batch of beer to check the quality, it is not feasible to keep repeating the same study over and over again on different random samples, even though that would obviously lead to more precise estimates. The reason that t, z and p values exist is to give real-life researchers (who usually only have one sample to work with) a way of estimating what would have happened if they had drawn a large number of samples, and thereby quantify the uncertainty associated with estimates from the one sample they actually have.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.