Say I have a sequence of given variables $x_i$, $i=1,\ldots,n-1$ and the response $y_i$ and we explore the model $y_i\sim \text{Normal}(\alpha x_i,\sigma^2)$ with $y_i$ independent of $y_j$ for $i\neq j$. Parameters $\alpha$ and $\sigma$ we fit using maximum likelihood.
Say we wish to forecast $y_n$ given $x_n$. It seems clear that the best forecast on $y_n$ would be $\alpha_\text{MLE}\,x_n$. There are at least two types of uncertainties as far as I can see:
- An uncertainty intrinsic to the model, that is, $\sigma$. We cannot hope to predict $y_n$ with a better precision than $\sigma$, which is akin to an "observational error".
- An uncertainty derived from the fact that the MLE estimators have a variance given by the expected Fisher information, related with the second derivative or curvature of the likelihood function at maximum. This is what colloquially can be called the "fitting error/uncertainty" of $\alpha_\text{MLE}$. Certainly, this "error" is propagated to the best estimator for $y_n$, creating another source of uncertainty.
What standard terminology is used for these two types of uncertainty?
