# Minimize $\mathrm{E}(U-bV-a)^{2}$, $U=\sum_{i=1}^{3}X_{i}.\, V=\sum_{i=1}^{3}i X_{i}\quad X_{1},X_{2},X_{3}\in N(1,1)$ i.i.d. r.v

Let $$X_{1}, X_{2}$$ and $$X_{3}$$ be independent , $$N(1,1)$$-distributed random variables. Set $$U=X_{1}+X_{2}+X_{3}$$ and $$V=X_{1}+2X_{2}+3X_{3}$$. Determine the constants $$a$$ and $$b$$ so that $$\mathrm{E}(U-a-bV)^{2}$$ is minimized.

Here is what I did

$$\begin{equation*} \begin{split} U-bV\in N(B\mu,B\Lambda B^{T})\\ \mathrm{E}((U-bV)-a)^{2}\\ \frac{d}{d a} \mathrm{E}((U-bV)-a)^{2}=-2\mathrm{E}(U-bV)+2a=2(a-\mathrm{E}(U-bV))\\ \mu= \left[\begin{array}{c} U \\ bV \end{array}\right] \left[\begin{array}{ccc} 1&1& 1 \\ b&2b&3b \end{array}\right]\left[\begin{array}{c} 1 \\ 1\\ 1 \end{array}\right]=\left[\begin{array}{c} 3 \\ 6b \end{array}\right]\\ B\mu=\left[\begin{array}{cc} 1 & -1 \end{array}\right]\left[\begin{array}{c} 3 \\ 6b \end{array}\right]=\left[3-6b\right]\\ 2(a-\mathrm{E}(U-bV))=2(3-6b-a) \end{split} \end{equation*}$$

$$\begin{equation*} b=a=\frac{3}{7}\Leftrightarrow 2(3-6b-a)=0 \end{equation*}$$ which of course works. But is there any reason that this is the only solution? Is there any reason for example

$$\begin{equation*} b=\frac{1}{3},\, a=1\quad \text{or}\quad b=-1,\, a=9 \end{equation*}$$

wouldn't work as well?

You can multiply out $$\mathrm{E}(U-a-bV)^{2}$$ keeping $$U$$ and $$V$$. Differentiate wrt $$a$$ to get an equation involving $$a$$ and $$b$$ in terms of expectations of $$U$$ and $$V$$. Do the same for $$b$$. You then have two simultaneous equations in $$a$$ and $$b$$.

Solve for $$a$$ and $$b$$. You should get..

$$\hat b = Cov (U,V)/Var(V)$$

$$\hat a = E(U)-\hat bE(V)$$

This agrees with the usual least squares result.

Next, plug in the values of $$U$$ and $$V$$ in terms of the $$X_i$$.

$$Cov (X_i, X_j) = 0$$ if $$i \ne j$$. $$Cov (X_i, X_i) = Var(X_i)$$

$$\begin{equation*} \begin{split} U,V\in N(\mu,\Lambda)\\ \mu=\left[\begin{array}{ccc} 1&1& 1 \\ 1&2&3 \end{array}\right]\left[\begin{array}{c} 1 \\ 1\\ 1 \end{array}\right]=\left[\begin{array}{c} 3 \\ 6 \end{array}\right]\\ \Lambda= \left[\begin{array}{ccc} 1 & 1& 1 \\ 1 & 2& 3 \end{array}\right]\left[\begin{array}{cc} 1 &1 \\ 1 & 2\\ 1& 3 \end{array}\right]=\left[\begin{array}{cc} 3 & 6\\ 6 & 14 \end{array}\right]\\ \mathrm{E}((U-a)-bV)^{2}=\mathrm{E}\Big[(U-a)^{2}-2b(U-a)V+b^{2}V^{2}\Big]\\ =\mathrm{E}\Big[U^{2}-2aU+a^{2}-2bUV+2baV+b^{2}V^{2}\Big]\\ =\mathrm{E}(U^{2})-2a\mathrm{E}(U)+a^{2}-2b\mathrm{E}(UV)+2ba\mathrm{E}(V)+b^{2}\mathrm{E}(V^{2})\\ \mathrm{E}(U)=3,\quad \mathrm{E}(V)=6,\qquad \mathrm{Var}(U)=\mathrm{E}(U^{2})-(\mathrm{E}(U))^{2}\\ \mathrm{Var}(V)=\mathrm{E}(V^{2})-(\mathrm{E}(V))^{2}\\ \mathrm{E}(U^{2})=3+(3)^{2}=12\\ \mathrm{Var}(V^{2})=14+(6)^{2}=50\\ \mathrm{Cov}(U,V)=\mathrm{E}(UV)-\mathrm{E}(U)\mathrm{E}(V)\\ \mathrm{E}(UV)=6+3\cdot 6=24\\ \mathrm{E}((U-a)-bV)^{2}=12-6a+a^{2}-48b+12ba+50b^{2}\\ \frac{d}{da}\mathrm{E}((U-a)-bV)^{2}=0=-6+2a+12b\\ a=\frac{-6+12b}{-2}=3-6b\\ \frac{d}{db}\mathrm{E}((U-a)-bV)^{2}=0=-48+12a+100b\\ =-48+12(3-6b)+100b\\ =-12+28b\\ \frac{3}{7}=b\\ a=3-6(\frac{3}{7})=\frac{3\cdot 7-3\cdot 6}{7}=\frac{3}{7} \end{split} \end{equation*}$$