# If $Y$ is a sub-Gaussian random variable, must $Y-E(Y\mid X)$ be also sub-Gaussian?

Question. If $$Y$$ is a sub-Gaussian random variable, must the regression residual $$\varepsilon=Y-E(Y\mid X)$$ be also sub-Gaussian? That is, whether there exists some constant $$\sigma>0$$ such that $$E\big( e^{\lambda \varepsilon} \big) \leq e^{\lambda^2\sigma^2/2}$$ for any $$\lambda\in\mathbb R$$.

In terms of the variance, it is well known that $$\mathrm{var}(\varepsilon)\leq \mathrm{var}(Y)$$. However, I can't find out a way to prove (or disprove) the sub-Gaussianity of $$\varepsilon$$ when $$Y$$ is sub-Gaussian. I wonder if $$\varepsilon$$ is always sub-Gaussian, by the sub-Gaussianity of $$Y$$, and maybe plus some additional requirements on $$X$$ (such as $$X$$ is also sub-Gaussian) or $$E(Y\mid X)$$ (such as $$E(Y\mid X)$$ is linear).

• The question isn't well-defined, because $\varepsilon$ has a bivariate distribution. If $X$ is not subgaussian, then in what sense would you hope for $\varepsilon$ to be subgaussian?
– whuber
Commented Nov 22, 2022 at 3:34
• @whuber I have updated the question by providing the definition of the sub-Gaussianity of $\varepsilon$. Actually, although $\varepsilon$ is generated by a bivariate distribution, I suppose it can be treated as a random variable with a univariate distribution. Commented Nov 22, 2022 at 4:19
• It turns out the statement is true, with a proof provided in math.stackexchange.com/questions/4581889/…. Commented Nov 22, 2022 at 11:58

If you're dealing with linear regression - the answer is yes, $$\epsilon$$ is indeed subG.

First and most obvious, $$E[\epsilon]=E[Y]-E[E[Y|X]]=0$$ by law of total expectation. Then, according to Vershynin, all you need to prove is the bound on the MGF of $$\epsilon$$.

Now, if you're modelling linear regression then $$E[Y|X]=\hat{Y}\sim\mathcal{N}$$, which means this is a subG variable as well and by Hoeffding's lemma we get that their sum is also subG.

This won't hold for Poisson regression, as the Poisson distribution is not subG.

Edit:

The thing with subG property is this: you need the distribution to fulfill that, from a point $$\lambda>0$$, the probability tail $$P(|Y|>\lambda)$$ is lighter than the Gaussian tail. This is usually the case when $$Y$$ is some variant of Gaussian or when it is bounded. If $$Y$$ is subG with variance proxy $$\sigma^2$$ then by definition of regression (classic, not Bayesian) we get that $$\hat{Y}$$ has subG distribution with the same proxy, hence the residual is subG with this proxy as well. The only condition here is that $$E[Y|X]=E[Y]$$ - that is, $$\hat{Y}$$ should be an unbiased estimator.

• Thanks for your reply. In your Poisson regression example, $Y$ itself is not sub-Gaussian. But I wonder if it is possible $\varepsilon$ is not sub-Gaussian when $Y$ is sub-Gaussian. Commented Nov 22, 2022 at 8:34
• see edit regarding tails and condition Commented Nov 22, 2022 at 8:57
• The statement "by definition of regression (classic, not Bayesian) we get that $\widehat{Y}$ has sub-Gaussian distribution with the same proxy" seems less obvious to me. Would you please provide a proof or a reference for this? Commented Nov 22, 2022 at 9:17
• When constructing a GLM model, the most basic assumption is $\hat{Y}_i=E[Y_i|X_i]$. If, for example, we discuss linear regression then the general model is $y_i=x^T_i\beta+\epsilon_i$, where $\epsilon_i\sim\mathcal{N}(0,\sigma^2)$. It then follows that due to the unbiasedness nature of $\hat{y}_i$, we get $y_i-\hat{y}_i=e_i\sim\mathcal{N}(0,\sigma^2)$. Commented Nov 22, 2022 at 10:52