comparison of distributions based on the maximum likelihood I'm new to statistics, and I'm trying to understand the concept of evaluating distribution estimates.
I have one observed data point of a normal distribution, say of a continuous random variable $x$, and the observed value is $5$. There are 4 hypotheses about the distribution of $x$:

*

*$x \sim N(0,1)$

*$x \sim N(0,3)$

*$x \sim N(4,1)$

*$x \sim N(4,3)$
Question 1: Which of these 4 hypotheses is the most likely? Conceptually I understand that this is a maximum likelihood question, but I'm not sure how to approach the question. That is, I can tell 1 and 4 are probably the worst hypotheses, and one or both of 2 and 3 are the best, but how can I quantify it?
Question 2: I also understand that since I have only one observed value, I can't correctly find the distribution with two parameters that maximizes the likelihood, but since intuitively I can say 2 and 3 are better options than 1 and 4, I believe should be able to find a 1-dimensional family of distributions that achieves this. Is this correct? and if so, how can it be done?
Question 3: If this is not possible with one observed data point, how can it be done with 2, or a collection of data points observed? can it be done if all of the observations are equal to 5?
I'm not sure if I'm asking the right questions here, feel free to direct me towards the right questions.
--
P.S. If this is something that I can learn from reading a textbook, I'd appreciate any references.
 A: This is a good question. Here is how you can determine the maximum likelihood estimate (MLE):
$$
\widehat \iota := \widehat \iota(x) := \arg \max_i f_i(x)  \quad (*)
$$
where $f_i$ is the density of hypothesis $i$. Think of $i \in \{1,2,3,4\}$ as the parameter. Then, the likelihood is the function $i \mapsto f_i(x)$ where you fix $x$ at the observed value of 5. We have
$$
f_1(x) = C e^{-x^2/2}, \;\; f_2(x) = \frac{C}{\sqrt 3}e^{-x^2/6}, \;\; f_3(x) = C e^{-(x-4)^2/2}, \;\; f_4(x) = \frac{C}{\sqrt 3} e^{-(x-4)^2/6}
$$
where $C = 1/\sqrt{2\pi}$ whose exact value does not matter in determining $\widehat \iota$.
At this point you plug in your observed value of $x = 5$ and compare the likelihoods. We have $f_3(5) > f_1(5)$ so we can rule out hypothesis 1. We also have $f_4(5) > f_2(5)$ so we rule out hypothesis 2. So we have to compare $f_3(5)$ and $f_4(5)$. It is easier to look at the log-likelihood ratio:
$$
\log \frac{f_3(5)}{f_4(5)} = \frac{\log 3}2  - \frac{(5-4)^2}{2} + \frac{(5-4)^2}{6} =\frac{\log 3}2 -\frac13 > 0
$$
showing that $f_3(5) > f_4(5)$, hence the MLE is $\widehat \iota = 3$.
It really doesn't matter if you have a single sample or multiple: You form the likelihood and maximize it. If you have a sample $x_1,\dots,x_n$, independently drawn from one of $f_i$, then the likelihood is $i \mapsto \prod_{t=1}^n f_i(x_t)$.

If you have a prior on the hypotheses, you can also compute the maximum a posterior estimate (MAP). The MAP estimate is the one that minimizes the probability of error (i.e., the optimal Bayes estimator for the 0-1 loss). Assuming that the prior is uniform, this coincides with the MLE. This means that if you repeat your experiment many times, every time randomly selecting one of the hypotheses and generating $x$ from it and if you always decide using the MLE rule (*), then your probability of error is (roughly) the smallest possible (among all decision rules). The thing that having more samples achieve is that this smallest best possible error will be smaller with a larger sample size.
These more or less can be found in any standard text on theoretical statistics, e.g., Keener's book or Lehmann and Casella's point estimation book.

About optimality: Equivalently, if you don't want to be a Bayesian, MLE is the estimater that minimizes the sum of the probability of errors under each hypothesis:
$$
\widehat \iota(\cdot) = \arg\min_{\delta(\cdot)} \sum_{i=1}^4 \mathbb P_i \big( \delta(x) \neq i\big).
$$
where the minimization is over all possible decision rules $\delta(\cdot)$.
A: 
since I have only one observed value, I can't correctly find the
distribution with two parameters that maximizes the likelihood

There is a risk of confusion in the question, namely between hypothesis testing and point estimation. Usually, the likelihood function is understood as a function of a parameter that indexes the distribution of the observation or of the sample. Here, there is no unknown parameter but a collection of four known distributions that can be compared by their density values at the observation, $x=5$. The largest density corresponds to $f_3$ but I would not call $\widehat\iota=3$ a maximum likelihood estimator. This result is obtained with a single observation.
When estimating a parameter by maximum likelihood, e.g., mean and variance of a Normal distribution, the maximum may be infinity if there are not enough observations in the sample and this is the case here, when $x=5$ is the single observation. While one can compare a few Normal distributions, one cannot estimate the "best" Normal distribution with a single observation.

but since intuitively I can say 2 and 3 are better options than 1 and
4, I believe should be able to find a 1-dimensional family of
distributions that achieves this. Is this correct? and if so, how can
it be done?

It is indeed feasible to maximise the likelihood for most one-dimensional families of distributions, for instance the Exponential or the Poisson distribution, when only observing only $x=5$. But since there is an infinity of such one-dimensional families, if the family is not justified by an initial argument, the choice of such a family is arbitrary and the maximisation meaningless.
A: Thanks everyone for your inputs, I just want to summarize what I've learned here.
A1. The likelihood function in this case is indeed equal to the PDF of those distributions, so the one with maximum PDF value at $x = 5$ is the best estimation among the four. In this case

*

*PDF(x=5, N(0,1)) = 0.00

*PDF(x=5, N(0,3)) = 0.03

*PDF(x=5, N(4,1)) = 0.24

*PDF(x=5, N(4,3)) = 0.13

A2. I can find a 1-dimensional family of normal distributions that has the same likelihood at $x=5$ as any of the four distributions by finding the family that its PDF equals the PDF of that distribution. For example, for the 4th distribution all of the followings are as good (equivalently as bad) as a distribution:

A3. Finally, since I want to estimate two parameters but I have only one observation, I can't find "the" best (i.e. the one with maximum likelihood among "all" normal distributions, not just these four), or looked another way, the maximum likelihood one is indeed $N(5,0)$! If I had two or more iid observations, say $x = 5$ and $x = 6$, then I could find it by maximizing of $PDF(5, \mu, \sigma) \cdot PDF(6, \mu, \sigma)$ for $\mu \in (-\infty,\infty)$ and $\sigma \in (0, \infty)$.
Am I understanding these correctly?
