45
$\begingroup$

For example, I have historical loss data and I am calculating extreme quantiles (Value-at-Risk or Probable Maximum Loss). The results obtained is for estimating the loss or predicting them? Where can one draw the line? I am confused.

$\endgroup$
64
$\begingroup$

"Prediction" and "estimation" indeed are sometimes used interchangeably in non-technical writing and they seem to function similarly, but there is a sharp distinction between them in the standard model of a statistical problem. An estimator uses data to guess at a parameter while a predictor uses the data to guess at some random value that is not part of the dataset. For those who are unfamiliar with what "parameter" and "random value" mean in statistics, the following provides a detailed explanation.

In this standard model, data are assumed to constitute a (possibly multivariate) observation $\mathbf{x}$ of a random variable $X$ whose distribution is known only to lie within a definite set of possible distributions, the "states of nature". An estimator $t$ is a mathematical procedure that assigns to each possible value of $\mathbf{x}$ some property $t(\mathbf{x})$ of a state of nature $\theta$, such as its mean $\mu(\theta)$. Thus an estimate is a guess about the true state of nature. We can tell how good an estimate is by comparing $t(\mathbf{x})$ to $\mu(\theta)$.

A predictor $p(\mathbf{x})$ concerns the independent observation of another random variable $Z$ whose distribution is related to the true state of nature. A prediction is a guess about another random value. We can tell how good a particular prediction is only by comparing $p(\mathbf{x})$ to the value realized by $Z$. We hope that on average the agreement will be good (in the sense of averaging over all possible outcomes $\mathbf{x}$ and simultaneously over all possible values of $Z$).

Ordinary least squares affords the standard example. The data consist of pairs $(x_i,y_i)$ associating values $y_i$ of the dependent variable to values $x_i$ of the independent variable. The state of nature is specified by three parameters $\alpha$, $\beta$, and $\sigma$: it says that each $y_i$ is like an independent draw from a normal distribution with mean $\alpha + \beta x_i$ and standard deviation $\sigma$. $\alpha$, $\beta$, and $\sigma$ are parameters (numbers) believed to be fixed and unvarying. Interest focuses on $\alpha$ (the intercept) and $\beta$ (the slope). The OLS estimate, written $(\hat{\alpha}, \hat{\beta})$, is good in the sense that $\hat{\alpha}$ tends to be close to $\alpha$ and $\hat{\beta}$ tends to be close to $\beta$, no matter what the true (but unknown) values of $\alpha$ and $\beta$ might be.

OLS prediction consists of observing a new value $Z = Y(x)$ of the dependent variable associated with some value $x$ of the independent variable. $x$ might or might not be among the $x_i$ in the dataset; that is immaterial. One intuitively good prediction is that this new value is likely to be close to $\hat{\alpha} + \hat{\beta}x$. Better predictions say just how close the new value might be (they are called prediction intervals). They account for the fact that $\hat{\alpha}$ and $\hat{\beta}$ are uncertain (because they depend mathematically on the random values $(y_i)$), that $\sigma$ is not known for certain (and therefore has to be estimated), as well as the assumption that $Y(x)$ has a normal distribution with standard deviation $\sigma$ and mean $\alpha + \beta x$ (note the absence of any hats!).

Note especially that this prediction has two separate sources of uncertainty: uncertainty in the data $(x_i,y_i)$ leads to uncertainty in the estimated slope, intercept, and residual standard deviation ($\sigma$); in addition, there is uncertainty in just what value of $Y(x)$ will occur. This additional uncertainty--because $Y(x)$ is random--characterizes predictions. A prediction may look like an estimate (after all, $\hat{\alpha} + \hat{\beta}x$ estimates $\alpha+\beta x$ :-) and may even have the very same mathematical formula ($p(\mathbf{x})$ can sometimes be the same as $t(\mathbf{x})$), but it will come with a greater amount of uncertainty than the estimate.

Here, then, in the example of OLS, we see the distinction clearly: an estimate guesses at the parameters (which are fixed but unknown numbers), while a prediction guesses at the value of a random quantity. The source of potential confusion is that the prediction usually builds on the estimated parameters and might even have the same formula as an estimator.

In practice, you can distinguish estimators from predictors in two ways:

  1. purpose: an estimator seeks to know a property of the true state of nature, while a prediction seeks to guess the outcome of a random variable; and

  2. uncertainty: a predictor usually has larger uncertainty than a related estimator, due to the added uncertainty in the outcome of that random variable. Well-documented and -described predictors therefore usually come with uncertainty bands--prediction intervals--that are wider than the uncertainty bands of estimators, known as confidence intervals. A characteristic feature of prediction intervals is that they can (hypothetically) shrink as the dataset grows, but they will not shrink to zero width--the uncertainty in the random outcome is "irreducible"--whereas the widths of confidence intervals will tend to shrink to zero, corresponding to our intuition that the precision of an estimate can become arbitrarily good with sufficient amounts of data.

In applying this to assessing potential investment loss, first consider the purpose: do you want to know how much you might actually lose on this investment (or this particular basket of investments) during a given period, or are you really just guessing what is the expected loss (over a large universe of investments, perhaps)? The former is a prediction, the latter an estimate. Then consider the uncertainty. How would your answer change if you had nearly infinite resources to gather data and perform analyses? If it would become very precise, you are probably estimating the expected return on the investment, whereas if you remain highly uncertain about the answer, you are making a prediction.

Thus, if you're still not sure which animal you're dealing with, ask this of your estimator/predictor: how wrong is it likely to be and why? By means of both criteria (1) and (2) you will know what you have.

$\endgroup$
  • $\begingroup$ Very interesting answer! Can you provide us some references about it? $\endgroup$ – user1420303 Aug 21 '15 at 16:58
  • 2
    $\begingroup$ @user1420303 Here are two. (1) Kiefer, Introduction to Statistical Inference (1987), p. 30. ("A prediction problem is one in which the decision is a guess not of some property of $F$, but rather of some property of a random variable... .") (2) Hahn & Meeker, Statistical Intervals (1991). See section 2.3 for examples and interpretations. $\endgroup$ – whuber Sep 13 '15 at 17:03
  • $\begingroup$ +1. I came across your answer because I'm trying to understand the terminological difference between BLUE and BLUP in mixed models, and I am still not sure I get it. In case of a mixed model $y=\alpha+\beta x + u_i + \epsilon$, where random intercepts $u_i \sim \mathcal N(0, \sigma^2_u)$, we estimate $\alpha, \beta, \sigma,$ and $\sigma_u$. Then we can predict $y$. This difference I understand. But what about $u_i$? They are computed with a BLUP, i.e. with a "predictor"; but it seems that with $n \to \infty$ any uncertainty disappears, so shouldn't we say that $u_i$ are estimated? $\endgroup$ – amoeba Oct 27 '15 at 15:37
  • 2
    $\begingroup$ @amoeba It might be helpful to understand this situation as a hierarchical model: at one level of the hierarchy $u_i$ is random (so statements about it would be predictors) while at a later level it has been realized and subsequent estimation is conditional upon the realization (making statements about it estimators). $\endgroup$ – whuber Oct 27 '15 at 16:03
  • 2
    $\begingroup$ @whuber The most important point you pointed out is that estimators are always aimed at approximating a quantity with non-stochastic/nonrandom nature like parameters in a SLR model; predictors are always aimed at approximating a quantity with stochastic/random nature like response variable(including the error term) in a SLR model. This point is highlited in Rao's early works. $\endgroup$ – Henry.L Jan 18 '16 at 0:46
8
$\begingroup$

Estimation is always for unknown parameter whereas prediction is for random variable.

$\endgroup$
  • 5
    $\begingroup$ You predict a realization of a random variable while you estimate a parameter of a random variable (e.g. its expected value). $\endgroup$ – Richard Hardy Apr 13 '17 at 8:44
  • $\begingroup$ @CowboyTrader, I do not know enough about kernel density estimation to comment on your claim. $\endgroup$ – Richard Hardy May 12 at 10:37
2
$\begingroup$

There is no difference in the models. There is indeed a (slight) difference in the action conducted. Estimation is the calibration of your probabilistic model using data ("learning" in the AI terminology). Prediction is the "guessing" of a future observation. Assuming this "guessing" is based on past data- this might be a case of estimation; such as the prediction of the height of the next person you are about to meet using an estimate of the mean height in the population. Note though, that prediction is not always an instance of estimation. The gender of the next person you are about to meet, is not a parameter of the population in the classical sense; Predicting the gender, might require some estimation, but it will require some more...

In the value-at-risk case, the prediction and estimation coincide since your predicted loss, is the estimated expectancy of the loss.

$\endgroup$
  • 2
    $\begingroup$ You start out well with a correct distinction between estimation and prediction, but then the last two-thirds of the reply appears to confound prediction with estimation once again. Introducing the example of gender gets more confusing still, because it's not related to the initial distinction (in fact, it's nonsensical, because underlying it is a shift of statistical model between the estimation and prediction step). $\endgroup$ – whuber Nov 1 '11 at 3:46
0
$\begingroup$

Prediction is the use of sample regression function to estimate a value for the dependent variable conditioned on some an unobserved values of the independent variable.

Estimation is the process or technique of calculating an unknown parameter or quantity of the population.

$\endgroup$
  • 3
    $\begingroup$ Brevity is laudable, but here it might lead to confusion. Prediction is not limited to regression applications: it is as fully general as estimation. Regardless, what exactly do you mean by "conditioned on some an unobserved values of the independent variable"? Is that just a way of saying that prediction requires data? If so, what about estimation, for which you do not supply such a requirement? Your description makes it sound like a textbook exercise, such as "what is the mean of a Normal distribution whose SD is $1$ and upper quartile is $2$?" Does estimation need data or not? $\endgroup$ – whuber Jul 15 '15 at 20:37
0
$\begingroup$

Usually "estimation" is reserved for parameters and the "predicition" is for values. However, sometimes the distinction gets blurred, e.g. you may have seen something like "estimate the value tomorrow" instead of "predict the value tomorrow."

The value-at-risk (VaR) is an interesting case. VaR is not a parameter, but we don't say "predict VaR." We say "estimate VaR." Why?

The reason in that VaR is not a random quantity IF you know the distribution, AND you need to know the distribution to calculate VaR. So, you if you're using parametric VaR approach, then you first estimate the parameters of the distribution then calculate VaR. If you're using the nonparametric VaR, then you directly estimate VaR similar to how you would estimate parameters. In this regard it's similar to quantile.

On the other hand, the loss amount is a random value. Hence, if you're asked to forecast losses, you'd be predicting them not estimating. Again, sometimes we say "estimate" loss. So, the line is blurred, as I wrote earlier.

$\endgroup$
  • $\begingroup$ You say VaR is not a parameter, but I wonder if that is really the case. VaR is the (conditional or unconditional) quantile of the distribution of the dependent variable. As such it looks like a parameter of the distribution to me, or at least a function of some other, more fundamental parameters, which does not seem to change the essence. It does not look like a realization of a random variable. $\endgroup$ – Richard Hardy May 28 at 13:27
  • $\begingroup$ Also, when you say that prediction is for values, it applies to values of parameters just as much as to realizations of random variables (which are also values). Hence, I suggest replacing values with realization of random variables; then you would have the dichotomy you are aiming at. $\endgroup$ – Richard Hardy May 28 at 13:29
-1
$\begingroup$

I would like to go against all other answers even though some turned out to be very popular. Why? Because they build their case based on a moot distinction between a parameter and a random value. They are focused on old school set membership approach to statistics. Moreover they don't resonate much with non-parametric statistics, such as kernel density estimation.

Suppose that we are locked in a room and somebody is feeding us data points from the feeding hole in the door. We receive $y_1,...,y_n$ according to two different scenarios below.

Scenario 1:

$y_t = \theta_t + \epsilon_t$

$\theta_t = \theta_*$

Scenario 2:

$y_t = \theta_t + \epsilon_t$

$\theta_t = \theta_* + \eta_t$

where $\epsilon_t \sim N(0,\sigma_\epsilon)$ and $\eta_t \sim N(0,\sigma_\eta)$ are uncorrelated. Variances are known to us.

$t$ is time index, but if you are not comfortable with that because you are not making a time series analysis you may think of it as order of data arrival.

From a control theory perspective, we can guess several things given what we have.

a: $\hat{y_{n+1}}(y_1,...,y_n)$ - prediction

b: $\hat{\theta_{n}}(y_1,...,y_n)$ - filtering/estimation

c: $\hat{\theta_{n+1}}(y_1,...,y_n)$ - prediction

d: $\hat{\theta_{n}}(y_1,...,y_n,...,y_m)$ - smoothing

Discussion:

Let's start by observing that scenario 1 is simply scenario 2 where $\sigma_\eta = 0$. According to other posts (c) is estimation under scenario 1 and but prediction under scenario 2. Whether underlying state of nature $\theta$ changes or whether there is uncertainty while we are making our guesses doesn't affect the definitions of estimation and prediction. Control theory focuses on which data are used to make inference when making the distinction. When we are using current data to make inference about "current" state of nature it is estimation, when we are using current data to make inference about "future" state of nature then it is prediction, when we are using data to make inference about "past" state of nature it is smoothing. This interpretation is generic and works under all approaches to statistics, not only under set membership approach, or frequentist approach. Note that both scenarios above can be successfully implemented with a Kalman Filter, Predictor, and Smoother without changing anything; and don't you worry all your intervals will be correctly calculated as well (rant on Difference between confidence intervals and prediction intervals)

$\endgroup$
-3
$\begingroup$

I find below definitions more explanatory:

Estimation is the calculated approximation of a result. This result might be a forecast but not necessarily. For example, I can estimate that the number of cars on the Golden Gate Bridge at 5 PM yesterday was 900 by assuming the three lanes going toward Marin were at capacity, each car takes 30 feet of space, and the bridge is 9000 feet long (9000 / 30 x 3 = 900).

Extrapolation is estimating the value of a variable outside a known range of values by assuming that the estimated value follows some pattern from the known ones. The simplest and most popular form of extrapolation is estimating a linear trend based on the known data. Alternatives to linear extrapolation include polynomial and conical extrapolation. Like estimation, extrapolation can be used for forecasting but it isn't limited to forecasting.

Prediction is simply saying something about the future. Predictions are usually focused on outcomes and not the pathway to those outcomes. For example, I could predict that by 2050 all vehicles will be powered with electric motors without explaining how we get from low adoption in 2011 to full adoption by 2050. As you can see from the previous example, predictions are not necessarily based on data.

Forecasting is the process of making a forecast or prediction. The terms forecast and prediction are often used interchangeably but sometimes forecasts are distinguished from predictions in that forecasts often provide explanations of the pathways to an outcome. For example, an electric vehicle adoption forecast might include the pathway to full electric vehicle adoption following an S-shaped adoption pattern where few cars are electric before 2025, an inflection point occurs at 2030 with rapid adoption, and the majority of cars are electric after 2040.

Estimation, extrapolation, prediction, and forecasting are not mutually exhaustive and collectively exhaustive terms. Good long-term forecasts for complex problems often need to use techniques other than extrapolation in order to produce plausible results. Forecasts and predictions can also occur without any kind of calculated estimations.

see links definitions1 definitions2

$\endgroup$
  • 2
    $\begingroup$ Prediction does not necessarily have to be about the future. $\endgroup$ – miura Oct 23 '12 at 7:55
  • $\begingroup$ read it complete: Prediction is simply saying something about the future. Predictions are usually focused on outcomes and not the pathway to those outcomes. $\endgroup$ – s.s.o Oct 23 '12 at 8:04
  • $\begingroup$ Yes, but the outcomes need not be in the future. You can, for example, also predict past unknown outcomes. $\endgroup$ – miura Oct 23 '12 at 8:13
  • 1
    $\begingroup$ This is a reasonable account of how the words "estimation" and "prediction" are used in a non-technical, non-statistical sense. As @miura is suggesting, it is apparent from the other answers here that those colloquial senses differ from conventional statistical ones. I see a valid argument in favor of interpreting the original question in a non-statistical way. However, that interpretation introduces an uncomfortable and unnecessary limitation by not allowing "prediction" to apply to past (completed) events with unknown outcomes. $\endgroup$ – whuber Oct 23 '12 at 13:46
  • $\begingroup$ İf you apply it to past e.g.a minimum-variance Kalman filter and a minimum-variance smoother may be used to recover data of interest from noisy measurements. The afore-mentioned techniques rely on one-step-ahead predictors... so, still prediction one step ahead (the future) :) $\endgroup$ – s.s.o Oct 23 '12 at 14:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.