Given a regression model on interval $[0,1]$ $$ Y_{i}=f(x_{i})+\epsilon_{i},\ i=1,\ldots,N $$ with fixed design and standard error assumptions $E(\epsilon_{i})=0;\ E(\epsilon_{i}\epsilon_{j})=\delta_{i,j}\sigma^{2}$. The regression function $f$ is from Sobolev space $$ f\in W^{q}\left[0,1\right]=\left[f,\ldots,f^{(q-1)} \text{ are absolutely continuous, } \int_{0}^{1}\left| f^{(q)}(x)\right| ^{2}<\infty\right] $$ Is the optimal convergence rate with respect to $L_{2}$-norm $N^{-q/(2q+1)}$ ? If yes, could you give a reference? If not, what additional assumptions are required ?


1 Answer 1


Yes, this is the classical Pinsker's theorem for Sobolev ellipsoids. Your rate is slightly off; also you need to restrict to $L^2$-balls. Part of the theorem says that, for $R > 0$,

$$ \inf_{T_n} \sup_{f \in W^q[0,1], \|f\|_{L^2}<R} \| T_n f - f \|_{L^2} = O(n^{-\frac{2q}{2q+1}}) $$

where $\inf_{T_n}$ denotes infimum over all estimators measurable with respect to data. Pinsker actually gave an linear shrinkage estimator that achieve the minimax risk exactly. Pinksker's estimator is, however, nonadaptive---it requires the smoothness parameter $q$ to be known. Later Stein gave a SURE block-threshold estimator that is adaptive over all $q>0$.

Wasserman's All of Nonparametric Statistics should outline these results.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.