2
$\begingroup$

Suppose you take a sample of size 3 with values 0, 0 and 1. (3 is the minimum size to make the sample asymmetrical and make the question non-trivial.) Ever since my first stats class, I've been vaguely confused because I didn't understand: why is it useful to calculate the mean of this sample, 1/3?

At first I naively thought that if this meant that if you assumed the background population had some standard deviation (e.g. 1), that 1/3 would be the value of the population mean that would maximize the probability of getting the observed outcome. (Of course I mean the probability of getting the observed outcome within some arbitrarily small delta, since the probability of any specific outcome is zero.) i.e. for my sample and assuming standard deviation 1, the value k that maximizes the value of f(k)*f(k)*f(1-k) where f is the standard normal distribution. But presumably that's not exactly 1/3, is it? (By the way, is there a technical term for this "observed outcome probability maximizer" value?)

[Edited to add: I was wrong. I explicitly f(k)*f(k)*f(1-k) where f is the standard normal distribution, and it has a maximum at 1/3. This is convenient, although it was far from obvious to me at the outset that this would be the case. This renders the rest of the question moot.]

But if 1/3 is not the value that maximizes the probability of the observed outcome, then what good is it?

Is it simply the case that before everyone had a computer in their pocket, calculating the "observed outcome probability maximizer" value was too hard, and calculating the mean was good enough? And now that we all have pocket computers, we should just go with the "observed outcome probability maximizer" instead, and calculating the mean is obsolete? If not, then what does "1/3" tell you about the background population?

Improve this question
$\endgroup$
11
  • $\begingroup$ 1/3 is not an MLE of the mean unless we make some assumption about the distribution of the sample. Transforming a likelihood to a probability - a posterior specifically - additionally requires a prior. In other words, the "probability" that a mean = X is a fundamentally Bayesian concept. On the other hand, the sample mean is always a consistent estimator of the population mean because of the law of large numbers: you are on your way to getting the true mean as you continue sampling. $\endgroup$
    – AdamO
    Mar 17, 2022 at 20:42
  • $\begingroup$ @AdamO OK, yes you're right I should have specified a prior, like "a priori the mean follows a uniform probability distribution from -5 to 5". However, even with a specification like that, I still don't see how "1/3" is the answer to a useful question. When you say "the sample mean is always a consistent estimator of the population mean", this is what I meant in the title that I don't see what this means in concrete terms. A "consistent estimator"? For my three-element sample (0, 0, 1), what is the concrete question whose correct answer is "1/3"? $\endgroup$
    – Bennett
    Mar 17, 2022 at 23:11
  • $\begingroup$ If you assume the population is Poisson, then with observations $\endgroup$
    – BruceET
    Mar 17, 2022 at 23:15
  • $\begingroup$ The concrete question is, “What is the sum of $0$, $0$, and $1$ divided by $3?$” This could be related to a proportion, for instance, by asking about the probability of a future value being $1$ or $0$. Given the data, the “most likely” value is $1/3$. $\endgroup$
    – Dave
    Mar 17, 2022 at 23:15
  • $\begingroup$ @Dave I mean what is the concrete question about the population, where the correct answer is 1/3? I am assuming the population is normally distributed. (I am not assuming that the variable can only be 0 or 1. It turns out that if the value can only be 0 or 1, and you observe 0,0,1, then 1/3 actually IS the value that maximizes the likelihood of that observation, although I didn't think this was obvious and I had to differentiate p(1-p)(1-p) to figure it out. But I did not think 1/3 would be the value that would maximize the likelihood of this outcome for a normal distribution.) $\endgroup$
    – Bennett
    Mar 17, 2022 at 23:24

1 Answer 1

Reset to default
2
$\begingroup$

1/3 would the value of the population mean that would maximize the probability of getting the observed outcome

This is a more profound statement than you might realize, as it alludes to maximum likelihood estimation, a common estimation method in statistics. Being a little loose, there is a technical sense in which the calculated $\bar x$ is the most likely value of the population parameter in order for you to obtain the data you obtained (and it isn't always true, but it's true under nice-enough conditions, such as normality). That sounds good, right? And that's why statisticians like maximum likelihood estimation.

To address your final paragraph, you're onto something by saying that finding the "probability maximizer" (likelihood estimator) is a good approach, perhaps better than the sample mean, but the sample mean often is exactly that likelihood maximizer by being the maximum likelihood estimate!

Getting away from maximum likelihood estimation in particular, "estimating a population parameter" means "guessing the correct value of the population parameter". There are ways of guessing that we know have desirable properties ("good" estimates of the unknown value). Since, the usual sample mean $\bar X$ has a number of desirable properties, we often like $\bar x$ as our estimate of $\mu$.

Improve this answer
$\endgroup$
4
  • $\begingroup$ I don't quite understand what you are saying in the first paragraph - "the calculated x¯ is the most likely value of the population parameter in order for you to obtain the data you obtained". Are you saying that (assuming normal distribution with standard deviation 1 and unknown mean), that 1/3 actually is the value for the population mean that would maximize the probability of getting the observed data (0, 0, 1)? I assumed that was not the case; it's far from obvious. $\endgroup$
    – Bennett
    Mar 17, 2022 at 23:17
  • $\begingroup$ In statistical terminology, “probability” and “likelihood” do not have the same meaning. $\endgroup$
    – Dave
    Mar 17, 2022 at 23:21
  • $\begingroup$ thanks right, so modify my comment to insert at the beginning "Assume that the pdf of the population mean is uniform from -5 to 5." Now would it be correct to say "probability"? And, am I correct in assuming that 1/3 is not the value that would maximize the probability of seeing the observed data (assuming population distribution is normal)? $\endgroup$
    – Bennett
    Mar 17, 2022 at 23:26
  • $\begingroup$ Now you’re putting a prior distribution on the population value and should use some Bayesian method. I haven’t run through the math, but maybe you have and have gotten $1/3$ to be the MAP (maximum a posteriori) estimate. $\endgroup$
    – Dave
    Mar 17, 2022 at 23:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.