I am considering a Gaussian distribution:

\begin{equation} y \sim N(\text{net}(x,w), \sigma^2). \end{equation}

where $\text{net}()$ is just the output of some neural net with weights $w$ and input $x$. The log-likelihood is

\begin{equation} \log L = -\frac{n}{2} (\log(2\pi) + \log(\sigma^2)) - \frac{1}{2\sigma^2} \sum_i (y_i - \text{net}(x_i,w_i))^2 \end{equation}

and the BIC is

\begin{align} BIC &= -2 \log L + \log(n) \cdot d \\[12pt] &= n(\log(2\pi) + \log(\sigma^2)) + \frac{1}{\sigma^2} \sum_i (y_i - \text{net}(x_i,\hat{w}_i))^2 + \log(n) \cdot d \\[6pt] &\approx \frac{n}{\sigma^2} \bigg( \text{MSE} + \frac{\log(n)\cdot d \cdot \sigma^2}{n} \bigg), \end{align}

where $d$ is the number of parameters.

Now, I want to estimate $\sigma^2$. My intuition was to estimate it with usual MLE which is the MSE, i.e.

$$\hat{\sigma}^2 = \frac{1}{n} \sum_i (y_i - \text{net}(x_i, \hat{w}_i))^2$$

but then the first term would just cancel out. Does the variance count as a parameter in $d$?

How do I estimate $\sigma^2$?