Maximum likelihood is not re-parametrization invariant. So how can one justify using it? There is something that is confusing me about max-likelihood estimators. Suppose my I have some data and the likelihood under a parameter $\mu$ is
$$
L(D|\mu) = e^{-(.7-\mu)^2}
$$
which is recognizable as the likelihood of Gaussian upto scaling. Now my max likelihood estimator will give me $\mu=.7$. 
Now suppose I didn't know that and instead was working with a parameter $t$ such that $\mu=\sin(t)$. Also suppose all this were numerical and so I wouldn't immediately see how silly the following likelihood looks like
$$
L(D|t) = e^{-(.7-\sin(t))^2}
$$
Now I would solve for the max likelihood and get additional solutions. To help see this I plot it below.

So from this point of view max-likelihood seems like a silly thing to do as it is not re-parametrization invariant. What am I missing?
Note that a Bayesian analysis would naturally take care of this since the likelihoods would always come with a measure
$$
L(D|\mu) P(\mu) d\mu = L(D|\mu(t)) P(\mu(t)) \frac{d\mu}{dt} dt
$$
Added part after responses and comments (added on 3/16/2018)
I realized later that my example above is not a good one because the two maxima in $t_1,t_2$ correspond to $.7=\sin(t_1)=\sin(t_2)$. So they are identifying the same point. I have kept the above for the discussion and responses below to make sense. However, I think the following is a better example of the issue I am trying to figure out.
Take 
$$
L(D|\mu) = e^{-(a-\mu)^2}
$$
Now suppose I reparameterize $\mu=\mu(t)$ then do a max-likelihood with respect to $t$ I get
$$
\frac{\partial L}{\partial t} = \frac{\partial L}{\partial \mu} \frac{\partial \mu}{\partial t}
$$
If I want a maxima at a location other than from the one I get from maximizing with respect to $\mu$ I require 
$$
\frac{\partial L}{\partial \mu} \ne 0
$$
and
$$
\frac{\partial \mu}{\partial t} =0, \qquad 
\frac{\partial L}{\partial \mu} \frac{\partial^2 \mu}{\partial t^2} < 0
$$
Thus, I can take a simple example 
$$
\mu = b - (a-b)t^2+t^3
$$
I plot the results below. We can clearly see that $\mu=a$ is the global maxima (and only one when maximizing with respect to $\mu$) but we also have another local maxima at $t=0$ when we maximizing with respect to $t$.

Note the map $\mu(t)$ is not bijective but I do not see why it has to be. Also, at least in this example, the global maxima will always be the one at $\mu=a$ but from a frequentist point of view wouldn't I be oblidged to take some kind of weighted average of 1/1.6 of $\mu=a$ and .6/1.6 of $\mu=b$ (that corresponds to $t=0$) if I completely worked in the $t$ space? 
 A: Looking at your graph, it appears that $\hat{t} \in \{0.7753975, 2.346194\}$ is a pretty reasonable guess at the MLE(s) of $t$.  Running those values through the $\sin$ function to get back to $\mu$ results in $\hat{\mu} = \{0.7, 0.7\}$ or $0.7$, just as it should.  So, there is no disagreement between the MLE of $\mu$ and the MLE(s) of $t$.
What is happening is that you have created a map from $\mu \to t$ that is not 1-1.  In this case, the true value of $\mu$ maps to multiple values of $t$, so not surprisingly you will have multiple maxima when working with $t$.  Note, however, that this would be the same if you were doing a Bayesian analysis, unless your prior restricted $t$ to the interval $[-\pi/2, \pi/2)$ or some such.  If you did so, for comparability, you should restrict the range of the MLE of $t$ to the same range, in which case you won't get multiple maxima for the likelihood function any more.
ETA:  In retrospect I focused too much on explanation-by-example and not enough on the underlying principle.  One can hardly do better than @whuber's comment in response to the OP in this regard.
In general, if you have a parameter $\theta$ and an associated MLE $\hat{\theta}$, and you construct a function $\theta = f(t)$, you've effectively created an alternate parameter $t$.  The MLE of $t$, label it $\hat{t}$, will be those values of $t$ such that $f(t) = \hat{\theta}$, i.e., $f(\hat{t}) = \hat{\theta}$.  
A: As my previous answer wasn't completely clear about the need for bijectivness
or not (one could argue my answer was just plain wrong). I did some
research about the whole reparametrizing thing and here is what I
found out. Both @whuber and @jbowman touch upon some of the same things.
Theory
So, in theory, the maximum likelihood estimator $\hat{\theta}$ of
the likelihood function $L\left(\theta\right)$, is invariant to re-parametrization.
So, say you have some known function $g$, which re-parametrizes $\theta$
into $\lambda=g(\theta)$ (where the dimensions of $\theta$ and $\lambda$
are not necessarily the same). Then two facts hold true,


*

*Maximizing $L\left(\theta\right)$ wrt. $\theta$, that is, finding the MLE, $\hat{\theta}$, and then reparametrizing it, $g(\hat{\theta})$,
yields the MLE of $\hat{\lambda}$. In short, $\hat{\lambda}=g\left(\hat{\theta}\right)$.

*Further, if $g$ has an inverse, maximizing $L\left(g^{-1}(\lambda)\right)$
wrt. $\lambda$, that is, finding the MLE $\hat{\lambda}$ yields
the same maximum as $\hat{\theta}$. So the MLE of $\theta$ is $\hat{\theta}=g^{-1}\left(\hat{\lambda}\right)$.


Splitting the invariance in these two sub-cases can seem a bit artificial,
but I find it useful since they represent two different use-cases
of re-parametrization.
In practice
The first use-case is where you somehow can identify the MLE for some
parameter, but you actually need a certain transformation of that
variable. For example you have an estimator, $\hat{\sigma},$ for the
parameter $\sigma$ in the normal distribution, but you are actually
interested in the MLE for the variance $\sigma^{2}$. Then you can
use the invariance principle and simply square the $\sigma$-MLE,
$\hat{\sigma^{2}}=(\hat{\sigma})^{2}$. 
An example for the second use-case, is that you have a numerical algorithm,
like gradient descent or Newton-Raphson, to maximize the likelihood
function. Say, you want to estimate the parameter $\sigma^{2}$ from
a normal distribution. The parameter is strictly positive by definition,
but the numerical procedure doesn't allow you to make constraints.
Well, you can use the invariance property to set $\sigma^{2}=\exp(\lambda)$
and let the algorithm vary $\lambda$ instead of $\sigma^{2}$, this
way ensuring that $\sigma^{2}$ stays positive. The exponential is
bijective, but this is not strictly required. We could have used $\sigma^{2}=\lambda^{2}$
instead, which is not bijective. But using a bijection is more practical,
since we can go from $\sigma^{2}$ to $\lambda$ and back in a unique
fashion.
The formalities
To define the MLE of $\lambda$ more formally we need to define what is called the profile likelihood
function as,
$$L^{\ast}(\lambda)=\sup_{\theta\vert\lambda=g\left(\theta\right)}L\left(\theta\right).$$
So, for a given $\lambda$-value the profile likelihood value, is
the supremum over all $\theta$'s which ensure that $g\left(\theta\right)$
equals $\lambda$.
With the profile likelihood defined we can then define the MLE for
$\lambda$, denoted $\hat{\lambda}$, as the value which maximizes
$L^{\ast}\left(\lambda\right)$.
With these definitions in place the invariance of re-parametrization
boils down to,
$$
L^{\ast}\left(\hat{\lambda}\right)=L\left(\hat{\theta}\right)
$$
which can be proved by, 
$$L^{\ast}\left(\hat{\lambda}\right)=\max_{\lambda}L^{\ast}\left(\lambda\right)=\max_{\lambda}\sup_{\theta\vert\lambda=g\left(\theta\right)}L\left(\theta\right)=\sup_{\theta}L\left(\theta\right)=\max_{\theta}L\left(\theta\right)$$
where I have assumed that $L\left(\theta\right)$ has a maximum.
If the re-parametrization is a bijection i.e. it is invertible, then
$L^{\ast}\left(\lambda\right)$ is simply $L(g(\theta))$ since each
$\theta$ uniquely maps to a $\lambda$, and hence the supremum over
``all'' $\theta$'s just collapses to the unique $L(\theta)$. So,
we get that,
\begin{align*}
L^{\ast}\left(\lambda\right) & =L\left(g\left(\theta\right)\right)\\
L^{\ast}\left(g^{-1}(\lambda)\right) & =L\left(\theta\right)
\end{align*}
and hence,
$$\hat{\theta}=g^{-1}\left(\hat{\lambda}\right).$$

References:

Invariance property of MLE: what is the MLE of $\theta^2$ of normal, $\bar{X}^2$?
http://www.stats.ox.ac.uk/~dlunn/b8_02/b8pdf_6.pdf
http://www.stat.unc.edu/faculty/cji/lecture7.pdf
https://en.wikipedia.org/wiki/Maximum_likelihood_estimation#Functional_invariance
