Consider any random variable $X$ and any random sample $(X_1,\dots, X_n)$ such that $X_i \sim X$.
As is well-known, $E(X)$ is the constant that minimizes the MSE of $X$, i.e., $E(X) = \arg\min_a E[(a-X)^2]$.
It seems that the minimizer does not change if we consider minimization over statistics from the random sample $m(X_1,\dots, X_n)$ instead, i.e., $\arg\min_{m(X_1,\dots, X_n)} E[(m(X_1,\dots, X_n)-X)^2]$ is the constant function $m(X_1,\dots, X_n) = E(X)$ for all $(X_1,\dots, X_n)$.
Informally, we don't really care about the random sample. If I am asking "what is the best function of random sample $(X_1,\dots, X_n)$ to predict $X$" (and "better prediction" is in terms of MSE), the answer remains: A constant function that's always equal to $E(X)$ that does not depend on the sample, but "implicitly assumes" you know $E(X)$.
What I am trying to do, if that's possible at all, is to formalize the idea/intuition that, if you did not know anything about $X$ --- and in particular, if you knew nothing about $E(X)$ --- then the best prediction would be $m(X_1,\dots, X_n) = \frac{1}{n} \sum_{i=1}^n X_i$.
My question is: Is there a standard statistical sense in which this is true?
I've tried to formalize this in a Bayesian sense, but I am struggling to
- Model the idea that "you don't know anything about the distribution" (to the point where I doubt whether that's feasible at all), and
- Make a general argument that's not dependent on a particular class of distribution (again, to the point where I doubt whether that's feasible at all).
Essentially, any assumption I make about the distributions of $X$ and $E(X) = \theta$ is tantamount to assuming "something is known" about the distribution $X$ and ends up creeping into the solution for the minimizer.
I am guessing I should use an uninformative prior for $\theta$ (https://www.statlect.com/fundamentals-of-statistics/uninformative-prior). Maybe I should look at the limit of a "flatter and flatter" and "wider and wider" discrete uniform for $\theta$. But I still don't know how to handle trying to make "as little assumption as possible" on the distribution of $X$ and argue that $m(X_1,\dots, X_n) = \frac{1}{n} \sum_{i=1}^n X_i$ is the best predictor (if that's doable at all).
Clarification following Dave's answer: I think I understand that the sample average $\bar{X}$ is the best MSE estimator of $E(X)$. I also understand that $E(X)$ is the best MSE predictor of $X$ itself. Part of what I am missing seems to be a connection between these two facts that would allow me to conclude that the $\bar{X}$ is the best MSE predictor of $X$. What I am after is a sense in which $\bar{X} = \arg\min_{m(X_1,\dots, X_n)} E[(m(X_1,\dots, X_n)-X)^2]$. I understand that, as is, the latter is wrong since the minimizer is a constant function always equals $E(X)$. But I am wondering whether it is true in some sense provided one does not have any information about the distribution of $X$ (and in particular about $E(X)$).