We seem to distinguish empirical estimates of parameters from 'true' values, and make comparisons between the two. I can understand what an empirical estimate is. What is a 'true' value?

For instance, my course notes have:

Definition 4.7 (Consistent estimator) An estimator $T(X_1,...,X_n)$ for a parameter $\theta$ is consistent if, for any $\epsilon>0$, we have $$ \lim_{n\rightarrow\infty}P(|T(X_1,...,X_n)-\theta|<\epsilon)=1 $$

How do we determine $\theta$ apart from by estimating it, in order to compare it to an estimator?

  • $\begingroup$ Could you share some context? Where did you saw this term? It may be important for getting adequate answer. $\endgroup$ – Tim May 20 at 12:30
  • 1
    $\begingroup$ A parameter is any definite quantitative property of the "box" that models a population or process. See stats.stackexchange.com/a/54894/919. $\endgroup$ – whuber May 20 at 12:42
  • 2
    $\begingroup$ @mjc the modeled value is always an estimator. What you describe, first assuming that the probability is $0.01$ and then later observing that $0.5$ is better, is the way the estimator tends probabilistically towards the true value. So anything that you use in practice is always (except simulation studies) an estimator. In your example you assumed that the coin ACTUALLY has a probability of $0.5$. That is the true value, you gave it by DEFINITION. In practice the true value can never be determined or observed, it can only be estimated. $\endgroup$ – LiKao May 20 at 13:22
  • 1
    $\begingroup$ @mjc Not quite. 3 is correct. But whenever you "observe" something, you have to make inferences. I can't "observe" the mean of a sample, you have to compute it from the data. Likewise, you can't "observe" the probability of a coin, you have to calculate it. So 1 and 2 should be the same thing. If you calculate the probability of a coin from observed tosses, you are using a model. So what you call "observation" (dividing heads by total tosses) is fitting a model. Any model should provide a consistent estimator, so 1 and 2 are just two different (hopefully) consistent estimators. $\endgroup$ – LiKao May 20 at 13:34
  • 1
    $\begingroup$ @mjc If $f$ is consistent with $\theta_1$ does not depend on $d$. Consistency is a theoretical property of an estimator, not one that depends on the sample $d\subset D$. That's why it must converge probabilistically, and not deterministically. If you get a bad sample, your estimate can be as much of as you like, it's just very improbable to get such a bad sample, if you have a lot of observations. As for the model: You mean a predictive model? The best predictive model should match the generating process. In that case $\theta_1=\theta_2$. For wrong models (i.e. all models): more complicated. $\endgroup$ – LiKao May 20 at 20:55

In general the "true value" is a fiction, defined within a model that in reality won't fit perfectly, in which case consequently there is also no such thing as a "true value". Assuming that there is a true parameter value is a device for doing theory and developing methods. It allows us to theoretically show that this-or-that method to estimate it has this-or-that property and works better or worse, which is a motivation for these methods even though it doesn't correspond exactly to the real situation (but then no model does).

If we simulate artificial data, however, we can fix and control the true parameter values, in which case we can compare the estimate to the true value (ignoring here potential issues with random number generation).

There are also some real situations in which true values can be "controlled" or known to some extent, for example if we have a measurement instrument that is meant to measure a certain quantity with stochastic measurement error, and in some situations we may be able to control the quantity that is measured. This still cannot guarantee the truth of the measurement error model within which the model parameter is defined, but at least we can control the real quantity that is interpreted to correspond to the true parameter value. An example for this are indirect estimates from age determination methods applied to individuals of which we know the precise age.

Sometimes we estimate parameters that correspond to existing population quantities of a usually big but finite population (such as population means of something that is well defined for all population members such as age) from a sample, in which case the population quantity would correspond to the "true" parameter value.

| cite | improve this answer | |

It all boils down to theory vs. practice.

The true value of a parameter is always a theoretical quantity. Thus, you can never determine the true $\theta$.

The idea behind this is, that there is some kind of process, which generates the data. This process has some parameter. If you knew that parameter and the process you could generate data, which would be indistinguishable from the real process. Other parameters always generate data, which can eventually be distinguished from the real process.

The only way to "know" the true $\theta$ is if you define the process mathematically. For example, if you define a fair coin, then the true probability of showing heads is $0.5$, but not because you determine it by an experiment, but because you defined it to be.

A consistent estimator will tend (probabilistically!) towards the true value, which is captured by the definition you provided. As you collect more data, the probability that the consistent estimator differs by a certain amount $\epsilon$ from the true value goes towards zero (NOTE: it is a common misconception that probability of zero means something never happens, but that is not necessarily true. It only means, it practically never happens).

So if you want to "determine" a true value just based on the data alone, a consistent estimator is your best method. You can never be sure, that you really got the right value (which is why any estimator should come with confidence intervals, etc).

Now, how do you know an estimator is a consistent estimator if you can't know the true value in practice? If some estimator is consistent is not a practical observation, but a theoretical property. Thus, you prove (theory!) that something is a consistent estimator.

Take for example the expected value and variance (true parameters) of a normal distributed variable (the process). The mean of the samples is a consistent estimator of the expected value. So you define $X \sim Normal(\mu,\sigma^2)$ (i.e. $\theta = (\mu,\sigma)$), and then you prove (!) that the mean will tend probabilistically towards $\mu$ for any $\mu$ and $\sigma$. Then you know, that the method you have just proven can be applied in practice to estimate the true $\mu$.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.