0
$\begingroup$

Let $X_{1}, X_{2}$ and $X_{3}$ be independent , $N(1,1)$-distributed random variables. Set $U=X_{1}+X_{2}+X_{3}$ and $V=X_{1}+2X_{2}+3X_{3}$. Determine the constants $a$ and $b$ so that $\mathrm{E}(U-a-bV)^{2}$ is minimized.

Here is what I did

\begin{equation*} \begin{split} U-bV\in N(B\mu,B\Lambda B^{T})\\ \mathrm{E}((U-bV)-a)^{2}\\ \frac{d}{d a} \mathrm{E}((U-bV)-a)^{2}=-2\mathrm{E}(U-bV)+2a=2(a-\mathrm{E}(U-bV))\\ \mu= \left[\begin{array}{c} U \\ bV \end{array}\right] \left[\begin{array}{ccc} 1&1& 1 \\ b&2b&3b \end{array}\right]\left[\begin{array}{c} 1 \\ 1\\ 1 \end{array}\right]=\left[\begin{array}{c} 3 \\ 6b \end{array}\right]\\ B\mu=\left[\begin{array}{cc} 1 & -1 \end{array}\right]\left[\begin{array}{c} 3 \\ 6b \end{array}\right]=\left[3-6b\right]\\ 2(a-\mathrm{E}(U-bV))=2(3-6b-a) \end{split} \end{equation*}

My book gives the answer

\begin{equation*} b=a=\frac{3}{7}\Leftrightarrow 2(3-6b-a)=0 \end{equation*} which of course works. But is there any reason that this is the only solution? Is there any reason for example

\begin{equation*} b=\frac{1}{3},\, a=1\quad \text{or}\quad b=-1,\, a=9 \end{equation*}

wouldn't work as well?

$\endgroup$
1
$\begingroup$

You can multiply out $\mathrm{E}(U-a-bV)^{2} $ keeping $ U $ and $ V $. Differentiate wrt $ a $ to get an equation involving $ a $ and $ b $ in terms of expectations of $ U $ and $ V $. Do the same for $ b $. You then have two simultaneous equations in $ a $ and $ b $.

Solve for $ a $ and $ b$. You should get..

$ \hat b = Cov (U,V)/Var(V) $

$ \hat a = E(U)-\hat bE(V) $

This agrees with the usual least squares result.

Next, plug in the values of $ U $ and $ V $ in terms of the $ X_i $.

$ Cov (X_i, X_j) = 0 $ if $ i \ne j $. $ Cov (X_i, X_i) = Var(X_i) $

$\endgroup$
0
$\begingroup$

\begin{equation*} \begin{split} U,V\in N(\mu,\Lambda)\\ \mu=\left[\begin{array}{ccc} 1&1& 1 \\ 1&2&3 \end{array}\right]\left[\begin{array}{c} 1 \\ 1\\ 1 \end{array}\right]=\left[\begin{array}{c} 3 \\ 6 \end{array}\right]\\ \Lambda= \left[\begin{array}{ccc} 1 & 1& 1 \\ 1 & 2& 3 \end{array}\right]\left[\begin{array}{cc} 1 &1 \\ 1 & 2\\ 1& 3 \end{array}\right]=\left[\begin{array}{cc} 3 & 6\\ 6 & 14 \end{array}\right]\\ \mathrm{E}((U-a)-bV)^{2}=\mathrm{E}\Big[(U-a)^{2}-2b(U-a)V+b^{2}V^{2}\Big]\\ =\mathrm{E}\Big[U^{2}-2aU+a^{2}-2bUV+2baV+b^{2}V^{2}\Big]\\ =\mathrm{E}(U^{2})-2a\mathrm{E}(U)+a^{2}-2b\mathrm{E}(UV)+2ba\mathrm{E}(V)+b^{2}\mathrm{E}(V^{2})\\ \mathrm{E}(U)=3,\quad \mathrm{E}(V)=6,\qquad \mathrm{Var}(U)=\mathrm{E}(U^{2})-(\mathrm{E}(U))^{2}\\ \mathrm{Var}(V)=\mathrm{E}(V^{2})-(\mathrm{E}(V))^{2}\\ \mathrm{E}(U^{2})=3+(3)^{2}=12\\ \mathrm{Var}(V^{2})=14+(6)^{2}=50\\ \mathrm{Cov}(U,V)=\mathrm{E}(UV)-\mathrm{E}(U)\mathrm{E}(V)\\ \mathrm{E}(UV)=6+3\cdot 6=24\\ \mathrm{E}((U-a)-bV)^{2}=12-6a+a^{2}-48b+12ba+50b^{2}\\ \frac{d}{da}\mathrm{E}((U-a)-bV)^{2}=0=-6+2a+12b\\ a=\frac{-6+12b}{-2}=3-6b\\ \frac{d}{db}\mathrm{E}((U-a)-bV)^{2}=0=-48+12a+100b\\ =-48+12(3-6b)+100b\\ =-12+28b\\ \frac{3}{7}=b\\ a=3-6(\frac{3}{7})=\frac{3\cdot 7-3\cdot 6}{7}=\frac{3}{7} \end{split} \end{equation*}

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.