# Orthogonal series density estimation

I am going through this paper Orthogonal series density estimation. I have a doubt in following

Assume that the random variable X is supported on [0, 1], that is, P(X ∈ [0, 1]) = 1, and that the probability density f of X is square integrable. Then the density may be approximated with any wished accuracy by a partial sum

$f_J(x) :=\sum_{j=0}^J \theta_j \varphi_j(x), 0\le x\le1$

Here $\{\varphi_j\}$ is the cosine basis

$\varphi_0(x)=1, \varphi_j(x)=\sqrt 2 cos(\pi j x), j=\{1,2,\cdots\}$

$\theta_j$ is the $j_{th}$ coefficient and $J$ is the cut-off

Note that $\theta_0=\int_0^1f(x) dx=1$

Also $\theta_j=\int_0^1 \varphi_j(x) f(x) dx= E\{\varphi_j(x)\}$

This implies that $\theta_j$ can be estimated by sample mean estimator.

$\hat{\theta}=n^{-1}\sum_{l=1}^n \varphi(X_l)$

$Var(\hat{\theta_j})=E(\hat{\theta}-\theta)^2$

$=n^{-1}[1+2^{-1/2}\theta_{2j}-\theta_j^2]=n^{-1}d$ Using $cos^2(\alpha)=(1+cos(2\alpha))/2$

Here d is defined as coefficient of difficulty.

My doubt is in last step $E(\hat{\theta}-\theta)^2=n^{-1}[1+2^{-1/2}\theta_{2j}-\theta_j^2]$

I think $E({\hat{\theta_j}^2}-2\hat{\theta_j}{\theta}+{\theta}^2)=E({\hat{\theta_j}^2}-{\theta}^2)$

If we assume the points are drawn iid $E(\varphi(x_l),\varphi(x_m))=E(\varphi(x_l))E(\varphi(x_m))$

So $E((\sum_{l=1}^n\varphi(x_l))^2)=E(\sum_{l=1}^n\varphi(x_l)^2+2*\sum_{l,m=0}^n\varphi(x_l)\varphi(x_m))$

$=E(\sum_{l=1}^n\varphi(x_l)^2)+2*E(\sum_{l,m=0}^n\varphi(x_l)\varphi(x_m))$

$E({\hat{\theta_j}^2})=n^{-2}[n(1+2^{-1/2}\theta_{2j})+\big(n(n-1)/2\big)2*{\theta_j}^2]$

$E({\hat{\theta_j}^2})=n^{-1}[1+2^{-1/2}\theta_{2j}+(n-1){\theta_j}^2]$

Explanation for equation (8):

$\hat{f_j}= 1+\sum_{j=1}^J\hat\theta_j \varphi_j(x)$

$f_j= 1+\sum_{j=1}^\infty\theta_j \varphi_j(x)$

Substituting above values:

$E\int_0^1[\hat{f_j}-f_j]^2dx=E\int_0^1[\sum_{j=1}^J(\hat\theta_j-\theta_j)\varphi_j(x)-\sum_{J+1}^\infty \theta_j \varphi_j(x)]^2$

Note that $\int_0^1 \varphi_j(x) \varphi_i(x)= 1$ if $i=j$ and $0$ otherwise.

Therefore above expression becomes $E\sum_1^J(\hat\theta_j-\theta_j)^2+E\sum_{J+1}^\infty \theta_j^2= \sum_1^JVar(\hat\theta_j)+\sum_{J+1}^\infty \theta_j^2$

Rest is obtained by substituting (5) and (6)

Please let me know if my understanding is correct