What is the difference between estimation and prediction?

Question

For example, I have historical loss data and I am calculating extreme quantiles (Value-at-Risk or Probable Maximum Loss). The results obtained is for estimating the loss or predicting them? Where can one draw the line? I am confused.

Answer 1

"Prediction" and "estimation" indeed are sometimes used interchangeably in non-technical writing and they seem to function similarly, but there is a sharp distinction between them in the standard model of a statistical problem. An estimator uses data to guess at a parameter while a predictor uses the data to guess at some random value that is not part of the dataset. For those who are unfamiliar with what "parameter" and "random value" mean in statistics, the following provides a detailed explanation.

In this standard model, data are assumed to constitute a (possibly multivariate) observation $\mathbf{x}$ of a random variable $X$ whose distribution is known only to lie within a definite set of possible distributions, the "states of nature". An estimator $t$ is a mathematical procedure that assigns to each possible value of $\mathbf{x}$ some property $t(\mathbf{x})$ of a state of nature $\theta$, such as its mean $\mu(\theta)$. Thus an estimate is a guess about the true state of nature. We can tell how good an estimate is by comparing $t(\mathbf{x})$ to $\mu(\theta)$.

A predictor $p(\mathbf{x})$ concerns the independent observation of another random variable $Z$ whose distribution is related to the true state of nature. A prediction is a guess about another random value. We can tell how good a particular prediction is only by comparing $p(\mathbf{x})$ to the value realized by $Z$. We hope that on average the agreement will be good (in the sense of averaging over all possible outcomes $\mathbf{x}$ and simultaneously over all possible values of $Z$).

Ordinary least squares affords the standard example. The data consist of pairs $(x_i,y_i)$ associating values $y_i$ of the dependent variable to values $x_i$ of the independent variable. The state of nature is specified by three parameters $\alpha$, $\beta$, and $\sigma$: it says that each $y_i$ is like an independent draw from a normal distribution with mean $\alpha + \beta x_i$ and standard deviation $\sigma$. $\alpha$, $\beta$, and $\sigma$ are parameters (numbers) believed to be fixed and unvarying. Interest focuses on $\alpha$ (the intercept) and $\beta$ (the slope). The OLS estimate, written $(\hat{\alpha}, \hat{\beta})$, is good in the sense that $\hat{\alpha}$ tends to be close to $\alpha$ and $\hat{\beta}$ tends to be close to $\beta$, no matter what the true (but unknown) values of $\alpha$ and $\beta$ might be.

OLS prediction consists of observing a new value $Z = Y(x)$ of the dependent variable associated with some value $x$ of the independent variable. $x$ might or might not be among the $x_i$ in the dataset; that is immaterial. One intuitively good prediction is that this new value is likely to be close to $\hat{\alpha} + \hat{\beta}x$. Better predictions say just how close the new value might be (they are called prediction intervals). They account for the fact that $\hat{\alpha}$ and $\hat{\beta}$ are uncertain (because they depend mathematically on the random values $(y_i)$), that $\sigma$ is not known for certain (and therefore has to be estimated), as well as the assumption that $Y(x)$ has a normal distribution with standard deviation $\sigma$ and mean $\alpha + \beta x$ (note the absence of any hats!).

Note especially that this prediction has two separate sources of uncertainty: uncertainty in the data $(x_i,y_i)$ leads to uncertainty in the estimated slope, intercept, and residual standard deviation ($\sigma$); in addition, there is uncertainty in just what value of $Y(x)$ will occur. This additional uncertainty--because $Y(x)$ is random--characterizes predictions. A prediction may look like an estimate (after all, $\hat{\alpha} + \hat{\beta}x$ estimates $\alpha+\beta x$ :-) and may even have the very same mathematical formula ($p(\mathbf{x})$ can sometimes be the same as $t(\mathbf{x})$), but it will come with a greater amount of uncertainty than the estimate.

Here, then, in the example of OLS, we see the distinction clearly: an estimate guesses at the parameters (which are fixed but unknown numbers), while a prediction guesses at the value of a random quantity. The source of potential confusion is that the prediction usually builds on the estimated parameters and might even have the same formula as an estimator.

In practice, you can distinguish estimators from predictors in two ways:

1. purpose: an estimator seeks to know a property of the true state of nature, while a prediction seeks to guess the outcome of a random variable; and

2. uncertainty: a predictor usually has larger uncertainty than a related estimator, due to the added uncertainty in the outcome of that random variable. Well-documented and -described predictors therefore usually come with uncertainty bands--prediction intervals--that are wider than the uncertainty bands of estimators, known as confidence intervals. A characteristic feature of prediction intervals is that they can (hypothetically) shrink as the dataset grows, but they will not shrink to zero width--the uncertainty in the random outcome is "irreducible"--whereas the widths of confidence intervals will tend to shrink to zero, corresponding to our intuition that the precision of an estimate can become arbitrarily good with sufficient amounts of data.

In applying this to assessing potential investment loss, first consider the purpose: do you want to know how much you might actually lose on this investment (or this particular basket of investments) during a given period, or are you really just guessing what is the expected loss (over a large universe of investments, perhaps)? The former is a prediction, the latter an estimate. Then consider the uncertainty. How would your answer change if you had nearly infinite resources to gather data and perform analyses? If it would become very precise, you are probably estimating the expected return on the investment, whereas if you remain highly uncertain about the answer, you are making a prediction.

Thus, if you're still not sure which animal you're dealing with, ask this of your estimator/predictor: how wrong is it likely to be and why? By means of both criteria (1) and (2) you will know what you have.

Answer 2

Estimation is always for unknown parameter whereas prediction is for random variable.

Answer 3

There is no difference in the models. There is indeed a (slight) difference in the action conducted. Estimation is the calibration of your probabilistic model using data ("learning" in the AI terminology). Prediction is the "guessing" of a future observation. Assuming this "guessing" is based on past data- this might be a case of estimation; such as the prediction of the height of the next person you are about to meet using an estimate of the mean height in the population. Note though, that prediction is not always an instance of estimation. The gender of the next person you are about to meet, is not a parameter of the population in the classical sense; Predicting the gender, might require some estimation, but it will require some more...

In the value-at-risk case, the prediction and estimation coincide since your predicted loss, is the estimated expectancy of the loss.

Answer 4

Usually "estimation" is reserved for parameters and the "predicition" is for values. However, sometimes the distinction gets blurred, e.g. you may have seen something like "estimate the value tomorrow" instead of "predict the value tomorrow."

The value-at-risk (VaR) is an interesting case. VaR is not a parameter, but we don't say "predict VaR." We say "estimate VaR." Why?

The reason in that VaR is not a random quantity IF you know the distribution, AND you need to know the distribution to calculate VaR. So, you if you're using parametric VaR approach, then you first estimate the parameters of the distribution then calculate VaR. If you're using the nonparametric VaR, then you directly estimate VaR similar to how you would estimate parameters. In this regard it's similar to quantile.

On the other hand, the loss amount is a random value. Hence, if you're asked to forecast losses, you'd be predicting them not estimating. Again, sometimes we say "estimate" loss. So, the line is blurred, as I wrote earlier.

Answer 5

Prediction is the use of sample regression function to estimate a value for the dependent variable conditioned on some an unobserved values of the independent variable.

Estimation is the process or technique of calculating an unknown parameter or quantity of the population.

Answer 6

I find below definitions more explanatory:

Estimation is the calculated approximation of a result. This result might be a forecast but not necessarily. For example, I can estimate that the number of cars on the Golden Gate Bridge at 5 PM yesterday was 900 by assuming the three lanes going toward Marin were at capacity, each car takes 30 feet of space, and the bridge is 9000 feet long (9000 / 30 x 3 = 900).

Extrapolation is estimating the value of a variable outside a known range of values by assuming that the estimated value follows some pattern from the known ones. The simplest and most popular form of extrapolation is estimating a linear trend based on the known data. Alternatives to linear extrapolation include polynomial and conical extrapolation. Like estimation, extrapolation can be used for forecasting but it isn't limited to forecasting.

Prediction is simply saying something about the future. Predictions are usually focused on outcomes and not the pathway to those outcomes. For example, I could predict that by 2050 all vehicles will be powered with electric motors without explaining how we get from low adoption in 2011 to full adoption by 2050. As you can see from the previous example, predictions are not necessarily based on data.

Forecasting is the process of making a forecast or prediction. The terms forecast and prediction are often used interchangeably but sometimes forecasts are distinguished from predictions in that forecasts often provide explanations of the pathways to an outcome. For example, an electric vehicle adoption forecast might include the pathway to full electric vehicle adoption following an S-shaped adoption pattern where few cars are electric before 2025, an inflection point occurs at 2030 with rapid adoption, and the majority of cars are electric after 2040.

Estimation, extrapolation, prediction, and forecasting are not mutually exhaustive and collectively exhaustive terms. Good long-term forecasts for complex problems often need to use techniques other than extrapolation in order to produce plausible results. Forecasts and predictions can also occur without any kind of calculated estimations.

see links definitions1 definitions2