Suppose I have a proportion estimate defined as the proportion of times some event occurs out of $N$ trials. I call this proportion the sample proportion $\hat{p}$, while I call a second as the true population proportion $p$. Suppose I have something akin to a bootstrap over different inputs, where I can generate many such pairs, and I would like to compare these two values as my evaluation criteria. Generally, I am thinking of just taking the difference as a metric in comparing how different they are:

$$ \hat{p}-p $$

However, are there better metrics for comparing these? Would the ratio be better in certain cases?

  • $\begingroup$ +1 for the interesting q, but how do you know the population value $p_2?$ $\text{//}$ It would be more common to refer to your $p_2$ parameter as $p$ and your $p_1$ estimate of $p_2$ as $\hat p$. $\endgroup$
    – Dave
    Commented Aug 27, 2022 at 11:10
  • $\begingroup$ @Dave Thanks, I have changed it. I am assuming that the population value is known. For example, I have the actual value and I want to evaluate how well a model gets $\hat{p}$ to the true value $p$. $\endgroup$
    – user321627
    Commented Aug 27, 2022 at 11:52
  • $\begingroup$ If both $p$ and $\hat{p}$ would tend to be small, the difference would also be small, so the ratio would be better, I think. $\endgroup$ Commented Jul 20, 2023 at 21:27
  • $\begingroup$ Possibly related $\endgroup$
    – Dave
    Commented Jul 20, 2023 at 21:35


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.