0
$\begingroup$

This is a very basic question. I'm looking at a physical problem where one wants to estimate a parameter $\lambda$ of a system. Suppose I perform a measurement on the system. I call the (stochastic) outcomes of each measurement $x_i$, where $i=1,2,\ldots, M$, $M$ being the total number of measurements. Assume I can construct an estimator $\hat{\lambda}(x_i)$ at each round. Let me call these individual estimators $\hat{\lambda}_i$. My final estimator should take into account all rounds. Let's call it $\hat{\lambda}$, where I should have something like $\hat{\lambda} = \hat{\lambda}(x_1, x_2, \ldots, x_M)$.

My intuition tells me that the Mean Square Error should be defined as: $$\text{MSE}(\hat{\lambda}) := E_\lambda [(\lambda -\hat{\lambda}_i)^2].\tag{1}$$ However, the formula I find in Wikipedia is $$\text{MSE}(\hat{\lambda}) := E_\lambda [(\lambda -\hat{\lambda})^2].\tag{2}$$ The problem I have with this formula is that my final estimator is a constant: it is computed after the $M$ measurements. The expected value over the $M$-sized process should be itself.

How should I interpret this formula in the context of my problem?

The motivation is a better understanding of the MSE formula as the sum of the bias and the variance.

Following my definition of the MSE (the first one), I get:

$\text{MSE}(\hat{\lambda}) = E_\lambda[(\lambda - E_\lambda[\hat{\lambda}_i])^2] + E_\lambda[(E_\lambda[\hat{\lambda}_i]-\hat{\lambda}_i)^2].$

I would appreciate if someone could clarify. Thanks!

$\endgroup$

1 Answer 1

1
$\begingroup$

The mean (or average) squared error is indeed defined as (1). Or (2). Indeed, the setting (and the meaning of the notations differ:

  1. In (1), the expectation is against the distribution of the random variable $X_i$ and $\hat\lambda_i=\hat\lambda_i(x_i)$
  2. In (2), the expectation is against the distribution of the random vector $(X_1,\ldots,X_i,\ldots,X_M)$ and $\hat\lambda=\hat\lambda_i(x_1,\ldots,x_M)$

Maybe the confusion stems from perceiving $\hat\lambda$ as a vector made of the $\hat\lambda_i$'s, which is not the case since they all estimate the same quantity and thus evolve in the same set $\Lambda$.

...my final estimator is a constant: it is computed after the 𝑀 measurements...

Both estimators $\hat\lambda_i$ and $\hat\lambda$ are "constant" once the sample is observed. But as non-constant functions of the random variable $X_i$ and of the sample $(X_1,\ldots,X_i,\ldots,X_M)$ they are random variables.

$\endgroup$
2
  • $\begingroup$ Thanks for your answer. This is a subtle issue, I think. It seems more natural to me to define one's expectations against a set of values that are actually measured, rather than against the distribution of the $M$-vector of the $M$ values of each round of the measurement/experiment. In order to be able to talk about the distribution of this vector, I should repeat my $M$- measurements again, and again, a number of times of say $N$. Then I would be able to do statistics wrt to that set of $N$ $M$-vectors, right? It seems to me that definition 2 presuposes a "second layer of statistics". $\endgroup$
    – mcasariego
    Commented Feb 20, 2019 at 19:45
  • $\begingroup$ I am afraid we are not discussing the same problem. For me observing a real rv or observing a vector rv are conceptually identical. $\endgroup$
    – Xi'an
    Commented Feb 20, 2019 at 20:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.