Probabilistic model without predictors (inspired by Bayesian regression)

I have an exercise where I have to use a probabilistic model to fit the data. Here is the original problem (Problem 1):

(Problem 2 with solution) At the lecture we learnt to derive Bayesian linear regression solution. We have dataset $\textbf{x} = (x^{(1)},...,x^{(N)})$ and $\textbf{y} = (y^{(1)},...,y^{(N)})$, parameters $\textbf{w} = (w_0,w_1)^T$.

We also have $\phi(x) = (1,x)^T$ and $y = h^w(x) + \epsilon$ with $\epsilon \sim N(0,\sigma^2)$

Then we have the Bayesian solution :

The question is how to use insight from Problem 2 to solve Problem 1. As I realize problem 1 is just problem 2 version without any predictors $x$ (in which $\mu = w_0$ and $e^{(i)} = \epsilon$ ). My best guess would be $\mu = y^{(N+1)} = \text{Mean}(y^{(i)})$ but I'm not sure how to prove it

• Is it necessary to use Problem 2 as a 'substep' in the solution? For example, since the shift by a constant of a normal distribution is again a normal distribution we have $p(y_i) = N(\mu, 1)$ and so we can just write down the likelihood and minimize $- \log L$ directly which will lead you to $\mu = \text{mean of the$y_i$}$ i believe... Nov 2, 2017 at 10:59
• Not necessarily. Actually I think what u just said is exactly what I'm looking for. Could you elaborate/derive what is the log-likelihood form and how to minimize it to get the answer ? That would be great ! Nov 2, 2017 at 11:02
• I have to go to lunch now :-) If you can't solve it on your own within the next 45 minutes then I will write a complete answer... deal? Nov 2, 2017 at 11:10
• Yes sir. I'll try to :) Nov 2, 2017 at 11:20

1 Answer

Well I think I got the answer for this. Thanks to Fabian Werner and based on this wikipedia article https://en.wikipedia.org/wiki/Maximum_likelihood_estimation:

So we have $p(y_i) = N(\mu,1)$ and log-likelihood $$\log(L(\mu,\sigma)) = (-n/2)log(2\pi)-(1/2)\sum(y^{i}-\mu)^2$$ Taking derivatives: $$0 = \frac{\partial}{\partial\mu}\log(L(\mu,\sigma)) = 0 - \frac{-2n(\bar{y} - \mu)}{2}$$ And we get the answer $\mu = \bar{y}$ (Mean of y)

• So, what about the other two questions? ;-) Nov 2, 2017 at 12:20