You can rewrite the prior as a Gaussian hyper prior:
$$
p(\mu|i) = \mathrm{N}(i,1), \text{where } i \in \{0,1\}
$$
and set a prior for $i$, say
$$
p_0(i) = \begin{cases}a, & i=0, \\
b, & i=1. \end{cases}
$$
First you could ignore the prior on $i$ and compute the posterior measure analytically parameterised in $i$. In this case, you would have just a Gaussian prior and a Gaussian likelihood, so you can compute the posterior
$$
p(\mu|x,i) = \frac{p(x|\mu)p(\mu|i)}{p(x|i)} \propto p(x|\mu)p(\mu|i)
$$
analytically using Bayes' formula (and the cheat sheet Wikipedia entry on conjugate priors).
Note that $p(\mu|x,i)$ is still Gaussian. To get to the actual posterior $p(\mu|x)$ we want to get rid of the $I$. This can be achieved using the following formula:
$$
p(\mu|x) = \sum_{i=0}^1 p(i|x)\cdot p(\mu|x,i), \tag 1
$$
where
$$
p(i|x) = \frac{p_0(i)\cdot p(x|i)}{\sum_{j=0}^1 p_0(j)\cdot p(x|j)}.
$$
is the posterior measure over $i$ given the data $x$.
In any case, the posterior $p(\mu|x)$ will be again a Gaussian mixture with two components. This you can see in equation (1): We have a convex combination of two Gaussian measures there.
The question about the regularisation is somewhat more tricky. You get an $L^2$ regularisation in the Gaussian prior case, since the negative log-prior is an $L^2$-norm.
Here, we have the regulariser $R(\mu)$ given by
$$
R(\mu) = -\log(p(\mu)) = -\log\left(a \frac{1}{\sqrt{2\pi}} \exp(-\frac12\|\mu\|^2) + b \frac{1}{\sqrt{2\pi}} \exp(-\frac12\|\mu-1\|^2)\right).
$$
I do not actually see a way to simplify this term. It is definitely something non-standard to do as a regularisation.
