Inconsistent Empirical Risk Minimization procedure, but why?

Question

Given a random variable $Y$ and the typical squared loss function:

$$L(Y,\hat{Y}) = (Y-\hat{Y})^2$$

the minimizer for expected loss $E[L(Y,\hat{Y})]$ is know to be the mean, $\hat{Y} = E[Y] = \mu$.

If we take $n$ $IID$ samples from the distribution of $Y$, we can describe an Empirical Risk Minimization(ERM) procedure:

$$\hat{Y} = \arg\min_{Y^*} \sum_{i=1}^n (Y_i - Y^*)^2$$

$$\implies \hat{Y} = \frac{1}{n}\sum_{i=1}^nY_i$$

$$E[\hat{Y}] = \mu$$

hence, it is consistent.

Now let's assume that $Y \sim N(0,\sigma^2)$ and our loss function is as follows:

$$L(Y,\hat{Y}) = e^{2(Y-\hat{Y})} - 2(Y-\hat{Y}) - 1$$

The mimizer for expected loss $E[L(Y,\hat{Y})]$ can be shown to be $\hat{Y} = \sigma^2$ using the fact that $e^{2Y}$ is lognormal with $E[e^{2Y}] = e^{2\sigma^2}$.

If we again apply ERM procedure:

$$\hat{Y} = \arg\min_{Y^*} \sum_{i=1}^n L(Y_i,Y^*)$$

$$\implies \hat{Y} = \frac{1}{2} \, \ln \left( \frac{1}{n} \sum_{i=1}^n e^{2Y_i} \right)$$

$$E[\hat{Y}] \not\rightarrow \sigma^2$$

I would like to understand why the procedure is not consistent in this case. Which assumptions of ERM am I violating?

Answer 1

The estimator is strongly consistent, since $$ \hat{\theta}_n = \frac{1}{2} \, \ln \left( \frac{1}{n} \sum_{i=1}^n e^{2Y_i} \right) \to \frac{1}{2} \, \ln \left( \mathbb{E}[e^{2Y}]\right) = \sigma^2, $$ almost surely.

Continuing the discussion started in the comments about the distribution of the estimator.

sim <- function(true_var, n, N) {
    y <- matrix(rnorm(N * n, mean = 0, sd = sqrt(true_var)), ncol = n)
    M <- apply(y, 1, max)
    dy <- sweep(y, 1, M, "-")
    est <- M + apply(dy, 1, function(row) 0.5 * log(mean(exp(2 * row))))
    hist(est, freq = TRUE, breaks = "FD", col = "cyan")
    abline(v = true_var, col = "red", lwd = 2)
}

set.seed(1234)

sim(true_var = 2, n = 10^4, N = 10^3)

sim(true_var = 10, n = 10^4, N = 10^3)

Note: in the function, I subtracted the maximum to prevent overflows: $$ \hat{\theta}_n = M + \frac{1}{2} \, \ln \left( \frac{1}{n} \sum_{i=1}^n e^{2(Y_i-M)}\right), $$ in which $M = \max_{1\leq i\leq n} Y_i$.