# Interpretation of Mean Square Error formula

This is a very basic question. I'm looking at a physical problem where one wants to estimate a parameter $$\lambda$$ of a system. Suppose I perform a measurement on the system. I call the (stochastic) outcomes of each measurement $$x_i$$, where $$i=1,2,\ldots, M$$, $$M$$ being the total number of measurements. Assume I can construct an estimator $$\hat{\lambda}(x_i)$$ at each round. Let me call these individual estimators $$\hat{\lambda}_i$$. My final estimator should take into account all rounds. Let's call it $$\hat{\lambda}$$, where I should have something like $$\hat{\lambda} = \hat{\lambda}(x_1, x_2, \ldots, x_M)$$.

My intuition tells me that the Mean Square Error should be defined as: $$\text{MSE}(\hat{\lambda}) := E_\lambda [(\lambda -\hat{\lambda}_i)^2].\tag{1}$$ However, the formula I find in Wikipedia is $$\text{MSE}(\hat{\lambda}) := E_\lambda [(\lambda -\hat{\lambda})^2].\tag{2}$$ The problem I have with this formula is that my final estimator is a constant: it is computed after the $$M$$ measurements. The expected value over the $$M$$-sized process should be itself.

How should I interpret this formula in the context of my problem?

The motivation is a better understanding of the MSE formula as the sum of the bias and the variance.

Following my definition of the MSE (the first one), I get:

$$\text{MSE}(\hat{\lambda}) = E_\lambda[(\lambda - E_\lambda[\hat{\lambda}_i])^2] + E_\lambda[(E_\lambda[\hat{\lambda}_i]-\hat{\lambda}_i)^2].$$

I would appreciate if someone could clarify. Thanks!

The mean (or average) squared error is indeed defined as (1). Or (2). Indeed, the setting (and the meaning of the notations differ:

1. In (1), the expectation is against the distribution of the random variable $$X_i$$ and $$\hat\lambda_i=\hat\lambda_i(x_i)$$
2. In (2), the expectation is against the distribution of the random vector $$(X_1,\ldots,X_i,\ldots,X_M)$$ and $$\hat\lambda=\hat\lambda_i(x_1,\ldots,x_M)$$

Maybe the confusion stems from perceiving $$\hat\lambda$$ as a vector made of the $$\hat\lambda_i$$'s, which is not the case since they all estimate the same quantity and thus evolve in the same set $$\Lambda$$.

...my final estimator is a constant: it is computed after the 𝑀 measurements...

Both estimators $$\hat\lambda_i$$ and $$\hat\lambda$$ are "constant" once the sample is observed. But as non-constant functions of the random variable $$X_i$$ and of the sample $$(X_1,\ldots,X_i,\ldots,X_M)$$ they are random variables.

• Thanks for your answer. This is a subtle issue, I think. It seems more natural to me to define one's expectations against a set of values that are actually measured, rather than against the distribution of the $M$-vector of the $M$ values of each round of the measurement/experiment. In order to be able to talk about the distribution of this vector, I should repeat my $M$- measurements again, and again, a number of times of say $N$. Then I would be able to do statistics wrt to that set of $N$ $M$-vectors, right? It seems to me that definition 2 presuposes a "second layer of statistics". Commented Feb 20, 2019 at 19:45
• I am afraid we are not discussing the same problem. For me observing a real rv or observing a vector rv are conceptually identical. Commented Feb 20, 2019 at 20:25