Better understanding Maximum Likelihood parameter estimation 
Suppose I am trying to model the dependence of a variable $B$ on another variable $A$ by a function $B=f(A;k)$, where $k$ is a parameter, whose value I would like to estimate. Given $n$ observations $\{A_i,B_i\}$, how can use MLE to estimate the value of $k$?

For example (and just as an example), I assume the value of glucose ($=G$) depends on hemoglobin ($=H$) by the following model:
$$
G=\frac{1}{1+H\cdot k},
$$
where $k$ is a parameter unique to each individual, that is known to be normally distributed in the population. I am given a set of $n$ measurements of glucose and hemoglobin of one person, and would like to estimate the value of $k$. How can I do it?
I will be happy to receive recommendations (or, even better, explicit explanations) on resources covering the solution to this problem. I keep finding explanations on estimating a parameter from a distribution, and can't figure out how to relate such explanations to this kind of a problem.
I thought of first isolating $k$ to obtain that (EDIT: I failed to correctly isolate $k$ in my first thread, and fixed it in an edit)
$$
k=\frac{1-G_i}{H_i G_i},
$$
but wasn't sure how to continue from here...I know that $k$ is normally distributed in the population, but how can this help? What if the distribution of $k$ in the population wasn't given?
Thank you!
 A: In ML estimation, one maximizes the likelihood of the data given the parameters. In your case:
$$
\hat{k}_{\mathrm{ML}} = \arg\max_k p(G_1, G_2, \ldots, G_N, H_1, H_2, \ldots, H_N \mid k).
$$
If we assume that
$$
G_i = \frac{1}{1+H_i k} + \varepsilon , \quad \varepsilon \sim \mathcal{N}(0, \sigma_\varepsilon ),
$$
i.e., we assume Gaussian measurement noise on the $G_i$ and independent measurements, we obtain
$$
\begin{align}
\hat{k}_{\mathrm{ML}} &= \arg\max_k p(G_1, G_2, \ldots, G_N, H_1, H_2, \ldots, H_N \mid k) \\
&= \arg\max_k p(G_1, G_2, \ldots, G_N \mid k) \\
&= \arg\max_k \prod_{i=1}^N p(G_i \mid k) \\
&= \arg\max_k \log \prod_{i=1}^N p(G_i \mid k) \\
&= \arg\max_k \sum_{i=1}^N\log p(G_i \mid k)
\end{align}
$$
Furthermore, we have that
$$ p(G_i \mid k) = \mathcal{N}(G_i; \mu_i(k)=\frac{1}{1+H_i k}, \sigma_{\varepsilon })=\frac{1}{\sigma_\varepsilon\sqrt{2\pi}} \mathrm{e}^{-\frac{1}{2}(\frac{G_i-\mu_i}{\sigma_\varepsilon})^2}.$$
Plugging that into the above optimization problem, we get
$$
\begin{align}
\hat{k}_{\mathrm{ML}} &= \arg\max_k \sum_{i=1}^N -\frac{1}{2}\left(\frac{G_i-\mu_i(k)}{\sigma_\varepsilon}\right)^2 -N\log (\sigma_\varepsilon \sqrt{2\pi}) \\
&= \arg\max_k \sum_{i=1}^N -\frac{1}{2}\left(\frac{G_i-\mu_i(k)}{\sigma_\varepsilon}\right)^2 \\
&= \arg\min_k \sum_{i=1}^N (G_i-\mu_i(k))^2
\end{align}
$$
A few remarks:

*

*This particular example is an ordinary least squares problem a nonlinear least-squares estimation problem (as can be seen in the last equation).

*If the measurements $H_i$ are also assumed to be noisy, things get more complex. In that case, we're dealing with a nonlinear errors-in-variables regression problem. The optimization problem can still be solved numerically, of course.

*What happened to the information that $k$ is normally distributed in the general population? That was not used at all in the above derivation, because ML estimation does not consider such priors. If you want to take that into account, you can do maximum a posteriori (MAP) estimation, which is essentially ML estimation + a prior on the parameters. MAP estimation maximizes $p(k \mid G_1, \ldots, G_N, H_1, \ldots, H_N).$ Using Bayes theorem, we have that
$$
\begin{align}
\hat{k}_{\mathrm{MAP}} &= \arg\max_k p(k \mid G_1, \ldots, G_N, H_1, \ldots, H_N) \\
&= \arg\max_k \frac{p(G_{1:N}, H_{1:N} \mid k) \, p(k)}{p(G_{1:N}, H_{1:N})} \\
&= \arg\max_k p(G_{1:N}, H_{1:N} \mid k) \, p(k) \\
&= \arg\max_k \log p(G_{1:N}, H_{1:N} \mid k) + \log p(k) \\
&= \arg\max_k \log p(k) + \sum_{i=1}^N\log p(G_i \mid k),
\end{align}
$$
where $p(k)$ denotes prior knowledge about the distribution of $k$. In your example, that could be the population average. We see that this is exactly what we had above for the ML estimate, except for the additional term $\log p(k)$.

