# Derive the ML-estimation for the parameter $\beta$

I think I know in general how to derive the maximum likelihood estimation for a parameter given a distribution. But I can't wrap my head around this one!

We have the observations: $$(y_1, x_1),...,(y_n, x_n)$$ from a random sample where: $$Y_i \sim N(\beta x_i, 1)$$ for $$i = 1,2,...,n$$. We are interested in the expected value of $$y_i$$ given $$x_i$$.

How do i derive the ML-estimate for the parameter $$\beta?$$

Don't know how to begin when the parameter $$\beta$$ shows up in that way.

EDIT 1

I set:

$$\tilde{\mu} = x_i\beta$$

The Likelihood is:

$$\prod_{i=1}^{n} \frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{1}{2}(\frac{y_i - \tilde{\mu}}{\sigma})^2} \iff (\frac{1}{\sqrt{2\pi\sigma^2}})^n e^{-\sum_{i=1}^{n}\frac{1}{2\sigma^2}(y_i - \tilde{\mu})^2}$$

Then the log-likelihood is:

$$-\frac{n}{2}\ln{2\pi} -\frac{n}{2}\ln{\sigma^2} -\frac{1}{2\sigma^2}\sum_{i=1}^{n}(y_i - \tilde{\mu})^2$$

If we take the derivative of the log-likelihood we get:

$$-\frac{1}{2\sigma^2}\frac{d}{d\tilde{\mu}}(\sum_{i=1}^{n}(y_i - \tilde{\mu})^2)$$

Which is:

$$-\frac{1}{2\sigma^2}\frac{d}{d\tilde{\mu}}(\sum_{i=1}^{n}(y_i^2 - 2y_i\tilde{\mu} + \tilde{\mu}^2))$$

Should i now insert $$\tilde{\mu} = x_i\beta$$?

If i do i get:

$$-\frac{1}{2\sigma^2}\frac{d}{d\beta}(\sum_{i=1}^{n}(y_i^2 - 2y_i x_i\beta + x_i^2\beta^2))$$

What should i do now, this approach below seems bad?

$$-\frac{1}{2\sigma^2}\sum_{i=1}^{n}\frac{d}{d\beta}(y_i^2 - 2y_i x_i\beta + x_i^2\beta^2)$$

$$-\frac{1}{2\sigma^2}\sum_{i=1}^{n} - 2y_i x_i + 2 x_i^2\beta$$

$$\frac{1}{2\sigma^2} 2n\bar{x} (n\bar{y} - n\bar{x}\beta)$$

And set to zero for finding the estimate: $$\frac{1}{2\sigma^2} 2n\bar{x} (n\bar{y} - n\bar{x}\beta) = 0$$

$$\bar{y} = \bar{x}\beta$$

$$\frac{\bar{y}}{\bar{x}} = \beta$$

Which is wrong.

EDIT 2

I do try my best to understand so please explain if you downvote.

• Start by writing out the likelihood. If you don't know how to do it for $\beta x_i$, write it as a product of $n$ factors each using $\mu_i$, then replace $\mu_i$ with $\beta x_i$ for all $i$.
– Sycorax
Commented Dec 1, 2022 at 19:01
• 1. treat X as known, not a random variable. 2. Recall the chain rule. 3. It's always easiest to calculate MLEs from a single observation. Commented Dec 1, 2022 at 19:07
• In the expression for the normal distribution, you have $g(\beta)=(y_i - x_i \beta)^2$. What's the derivative of $g$ wrt $\beta$?
– Sycorax
Commented Dec 1, 2022 at 19:15
• Nope. Intuitively, that doesn't make sense because it implies $\beta$ is not a real number when $\bar x$ is 0, and that the estimation would become very volatile for small changes in $\bar x$ around 0. Write out the steps from the start and don't skip any any of the algebra.
– Sycorax
Commented Dec 1, 2022 at 19:43
• You go astray at the outset: $\tilde\mu$ depends on a single observation $x_i.$ That's not going to be useful and, as a notation, it's misleading you, because $\tilde\mu$ needs a subscript $i.$ You also introduce an undefined and superfluous quantity "$\sigma,$" further confusing the issue. Since the log likelihood of $y_i$ is $-(y_i-\beta x_i)^2/2$ (up to an additive constant), use that to form the log likelihood for an independent set of $(x_i,y_i)$ data and go on from there. The rest can be done with elementary algebra -- you don't even need Calculus.
– whuber
Commented Dec 1, 2022 at 20:39

The derivation is basically fine until here. After this step, you introduce $$\bar x, \bar y$$ and the result is not correct. $$0=-\frac{1}{2\sigma^2}\sum_{i=1}^{n} - 2y_i x_i + 2 x_i^2\beta.$$ It's given that $$\sigma=1$$. We can multiply through by the leading fraction. The rest is rearranging. \begin{align} 0 &= \sum_{i=1}^{n} y_i x_i - x_i^2\beta\\ &= \sum_{i=1}^{n} y_i x_i - \sum_{j=1}^n x_j^2\beta \\ \sum_{i=1}^{n} y_i x_i &= \beta \sum_{j=1}^n x_j^2 \\ \frac{\sum_{i=1}^{n} y_i x_i}{ \sum_{j=1}^n x_j^2} &= \beta \end{align}