# Exercise in Bayesian Statistics

What I want to do is the first part of the exercise :

It is for chapter 14 of the book, an introduction to regression. I have minimal experience with regression and is the first time I see a vector being used as the mean.

I am stuck and would like your help. My goal is to solve the next exercise that uses the model from this one. So if you could help me with this one I can try on my own the next one.

So far I have that the likelihood can be written as a product $p(x) p(y|a,b)$. I know I need to find a formula for $p(y_i|a,b)$ but I am not sure how.

The data here is both $(x_i, y_i)$ and not just the $y_i$. You have been given that independently $$x_i, y_i|a, b, u_i \sim N \left((u_i \, \, a + bu_i)^T, \Sigma \right).$$

and you know that independently

$$u_i \sim N(\mu, \tau^2).$$

To find the data likelihood you need to find $p(x, y | a, b)$ \begin{align*} L(a,b) & = \prod_{i=1}^{n}p(x_i, y_i | a, b)\\ & = \prod_{i=1}^{n}\int_{u_i} p(x_i, y_i, u_i | a, b)\\ & = \prod_{i=1}^{n}\int_{u_i} p( x_i, y_i,| a, b, u_i )p(u_i)\\ \end{align*}

You can now evaluate each of the inside integrals separately. This should be not too difficult since you are dealing with Gaussian densities. This way you will have to deal with multivariate densities. If you want to break it up further it would be

$$\prod_{i=1}^{n}\int_{u_i} p( y_i,| x_i, a, b, u_i ) \,p(x_i|u_i) \,p(u_i)\\$$

• In a discussion with my TA, I was given the hint that $x_i$ is not related to $(a,b)$ so the likelihood $p(x,y|a,b)$ can be written as $p(y|a,b)p(x)$. That is why I think we need to find only $p(y|a,b)$. Is not this correct? After that by conditioning we need to get $p(y_i|a,b)$. How do we find the distribution of $y_i|x_i,v_i,u_i$ ? – Iniciador May 4 '16 at 23:37
• Also what do you mean by ''since we are dealing with Gaussian densities?'' So far I've only used Normal, so gaussian in one dimension. Is there a special property to do it? Also thank you very much for the help. – Iniciador May 4 '16 at 23:42
• @Iniciador When you expand the pdfs on there and try to integrate out $u_i$ you will find that by "completing the squares" it will be straightforward. There is no special property, I just meant that its not a difficult integration. – Greenparker May 4 '16 at 23:46
• Got it thanks ! I will try to understand it now and work the next exercise. – Iniciador May 5 '16 at 0:02
• Silly question: I integrate with respect to $u_i$ correct? – Iniciador May 5 '16 at 1:25