Why do the terms in L2 Regularized likelihood represent Gaussian distributions?

Question

Studying machine learning, I've made it to the point where I've exponentiated my L2 Regularization loss function. We began with a simple ordinary least squares loss function, and added a penalty term proportional to the squared weights of the coefficients, as seen below:

Because minimizing the least squared loss function is equal minimizing the negative log-likelihood, we flipped the signs so that maximizing the negative loss function is equal to maximizing the log likelihood. Then we've exponentiated to get rid of the log function, yielding the $\exp{\{-J\}} $term below.

Now I'm told that these represent two gaussians. I have the following two expressions:

I know that $J = $ ($-$ log likelihood), thus $ -J =$ (log likelihood), thus $\exp{\{-J\}} = $ likelihood. What I'm confused by is how do the expressions in the first image represent the gaussians below? Or rather, why do/can I add $\frac{1}{2\sigma^2}$ inside the exponentiation and normalize by the constant? I'm missing the connection between them.

Side note: as I was following my material I thought I had the connection, but I got bogged down in some computations and believe I lost sight of the connection here.

Note: This question follows from my other post in which I was trying to prove what the instructor states, which is that the second expression in the $\exp{\{-J\}}$ term represents a Gaussian with $\mu = 0$ and $\sigma^2 = \frac{1}{\lambda}$, but I was getting a different answer. I can prove this about the Prior probability at the bottom, but the instructor said it referring to the second expression in $\exp{\{-J\}}$

Answer 1

These expressions, for the posterior and the (exponentiated) loss function, are not mathematically identical, but in this case they are computationally equivalent, since you are only interested in finding the maximum of the posterior (or minimum of the loss function). Because it doesn't matter whether you find the maximum of $f(x)$ or of $a\times f(x)$: both functions have their maximum at exactly the same value of $x$ (or one will have a minimum, if $a$ is negative), and you typically don't care how high that maximum is.

This also means that if you only know $a\times f(x)$ (i.e. you only know your target function up to a constant of multiplication), that's fine. You don't need to worry about getting the scaling constant right.

This explains why, for the purposes of optimization w.r.t. $w$, we can ignore the normalization constants $\frac{1}{\sqrt{2\pi\sigma^2}}$ and $\sqrt{\frac{\lambda}{2\pi}}$, and so they do not appear in the loss function (because why compute them if they don't matter), but they do appear in the strict mathematical definition of a likelihood and a prior.

That leaves one more discrepancy, which is the appearance of $\frac{1}{2\pi\sigma^2}$ in the likelihood (but not in the loss function) and of $\frac{\lambda}{2}$ in the prior (vs. $\lambda$ in the loss). This has a similar reason, because these values are weights that determine the importance of the prior and of the data points in the likelihood. But you only care about the relative importance of these things w.r.t. one another. And since $\sigma$ is constant for all data points, that means the only relevant parameter that we're left with is the strength of the prior, relative to the likelihood, which we can describe with a single scalar $\lambda$.

So in summary, the loss function retains only the relevant bits of the prior and likelihood, which is why you cannot fully reconstruct the prior and likelihood from the loss. It actually makes more sense to go in the other direction: take the log of the posterior (i.e. the product of likelihood $\times$ prior) and then drop all terms that you don't need for the optimization problem, and you should end up with the loss function you're familiar with.