Why does the Null Hypothesis have to be "equals to" and not "greater than or equal to"? In an online lecture I saw an example of hypothesis testing:

The mean (of something) is at least 12
Claim: $\mu \geq 12 \implies H_0: \mu = 12$
Opposite: $\mu \lt 12 \implies H_A: \mu < 12$

And the lecturer insisted that we must put the $=$ and not $\geq$ in the $H_0$. But why?
Furthermore, if we end up rejecting $H_0$, sure it could mean that $\mu < 12$, but doesn't that also mean the it's possible that $\mu > 12$ ?
 A: The reason for using $H_0: \mu = 12$ is that, among the set of values that correspond to $\mu \geq 12$, $\mu = 12$ is the most conservative (also called least favorable) configuration.
Let us be more precise what conservative means here. Say we set certain value of the observed statistic $\hat{\mu}$ at which we are willing to consider the null hypothesis as false (also called critical value $c$). $c$ should naturally be smaller than 12 to provide evidence against the null. Since $\hat{\mu}$ is just one of many possible realizations of the statistic, there is always some possibility of observing a value of $\hat{\mu}$ even if $\mu \geq 12$. Luckily, if we know the distribution of the test statistic, wecan compute the probability of observing a value of $\hat{\mu}$ that is smaller than or equal to $c$. This probability is called the probability of Type 1 error.
You can compute the probability of Type 1 error for all configurations that correspond to the hypothesis $\mu \geq 12$. In the figure I plot the distributions of the test statistics under two different such configurations $C_1: \mu = 12$ and $C_2: \mu = 13$. I also plot the probabilities of Type 1 error under the critical value $10.36$ for the two hypotheses (the shaded area under the respective curve) . It is easy to see that the probability of Type 1 error is always bigger for the configuration $C_1: \mu = 12$ than for any other $C_2$ that would also correspond to the hypothesis $\mu \geq 12$. I assumed normality here, but this result holds for any distribution that the test statistic can take. 
To sum up, the common practice (which also makes a lot of sense!) is to choose, within the set of configurations of the test that correspond to the null hypothesis, the one that gives you the highest probability of Type 1 error.

