Return to Answer

The Gaussian assumption, if any, is on the errors of the process, not the regressors. In the simplest case the model is $$y_i=\alpha+\beta x_i+\varepsilon_i$$, where $\varepsilon_i\sim\mathcal N(0,\sigma^2)$, i.e. errors are Gaussian.

The errors are not observed, so we can estimate them as residuals $\hat \varepsilon_i=r_i=y_i-(a+b x_i)$. Then we write the MLE equation and estimate $a,b,\hat\sigma$ from it. The estimates are of true parameters $\alpha,\beta,\sigma$ that you denoted collectively as $\theta$.

Actually, in many cases we don’t need the errors to be Gaussian IID, because it is enough that they have finite variance and are uncorrelated, plus some other assumptions. We then show that for large enough sample due to CLT the parameter estimates are Gaussian around true parameters. This way the requirements on the dataset are less restrictive and more realistic. Gaussian iid requirement on errors is often unrealistic, and it's a very strong assumption that is hard to test.

Requiring Gaussian assumption on regressors would render the model almost useless in most applications. Imagine you are conducting an experiment and setting X at a few chosen values, e.g. a temperature of the oven at $x=100,200,300,400F$. First, X is not random at all. Second, it’s distribution is not Gaussian. However, the regression can be applied in these type of settings.

The Gaussian assumption, if any, is on the errors of the process, not the regressors. In the simplest case the model is $$y_i=\alpha+\beta x_i+\varepsilon_i$$, where $\varepsilon_i\sim\mathcal N(0,\sigma^2)$, i.e. errors are Gaussian.

If you knew true parameters of the model you could infer the errors $\varepsilon_i=y_i-(\alpha+\beta x_i)$. You could interpret this as measured value $y_i$ being a random variable with mean $\alpha+\beta x_i$, i.e. $y_i\sim\mathcal N(\alpha+\beta x_i,\sigma^2)$. Notice how we replace a constant $\mu$ in a textbook Gaussian distribution expression with the model value $(\alpha+\beta x_i)$.

The errors are not observed, so we can estimate them as residuals $\hat \varepsilon_i\equiv r_i=y_i-(a+b x_i)$. Then we write the MLE equation and estimate $a,b,\hat\sigma$ from it. The estimates are of true parameters $\alpha,\beta,\sigma$ that you denoted collectively as $\theta$.

Actually, in many cases we don’t need the errors to be Gaussian IID, because it is enough that they have finite variance and are uncorrelated, plus some other assumptions. We then show that for large enough sample due to CLT the parameter estimates are Gaussian around true parameters. This way the requirements on the dataset are less restrictive and more realistic. Gaussian iid requirement on errors is often unrealistic, and it's a very strong assumption that is hard to test.

Requiring Gaussian assumption on regressors would render the model almost useless in most applications. Imagine you are conducting an experiment and setting X at a few chosen values, e.g. a temperature of the oven at $x=100,200,300,400F$. First, X is not random at all. Second, it’s distribution is not Gaussian. However, the regression can be applied in these type of settings.

The Gaussian assumption, if any, is on the errors of the process, not the regressors. In the simplest case the model is $$y_i=\alpha+\beta x_i+\varepsilon_i$$, where $\varepsilon_i\sim\mathcal N(0,\sigma^2)$, i.e. errors are Gaussian.

The errors are not observed, so we can estimate them as residuals $\hat \varepsilon_i=r_i=y_i-(a+b x_i)$. Then we write the MLE equations and estimate $a,b,\hat\sigma$, that approximate true parameters $\alpha,\beta,\sigma$ that you denoted collectively as $\theta$.

Actually, in many cases we don’t assume that the errors are Gaussian either, it is enough that they are of finite variance and uncorrelated plus some other assumptions. We then show that for large enough sample due to CLT the parameter estimates are Gaussian around true parameters. This way the requirements on the dataset are less restrictive and more realistic. Gaussian iid requirement on errors is often unrealistic.

Requiring Gaussian assumption on regressors would make the model very limited in applications. Imagine you are conducting an experiment and setting X at a few chosen values. First, X is not random at all. Second, it’s distribution can be Bernoulli or anything you choose. For instance you set temperature at set levels on the grid x=10, 20, 30, 40, ..., 100F. This is nothing like Gaussian yet the regression can be applied.

The Gaussian assumption, if any, is on the errors of the process, not the regressors. In the simplest case the model is $$y_i=\alpha+\beta x_i+\varepsilon_i$$, where $\varepsilon_i\sim\mathcal N(0,\sigma^2)$, i.e. errors are Gaussian.

The errors are not observed, so we can estimate them as residuals $\hat \varepsilon_i=r_i=y_i-(a+b x_i)$. Then we write the MLE equation and estimate $a,b,\hat\sigma$ from it. The estimates are of true parameters $\alpha,\beta,\sigma$ that you denoted collectively as $\theta$.

Actually, in many cases we don’t need the errors to be Gaussian IID, because it is enough that they have finite variance and are uncorrelated, plus some other assumptions. We then show that for large enough sample due to CLT the parameter estimates are Gaussian around true parameters. This way the requirements on the dataset are less restrictive and more realistic. Gaussian iid requirement on errors is often unrealistic, and it's a very strong assumption that is hard to test.

Requiring Gaussian assumption on regressors would render the model almost useless in most applications. Imagine you are conducting an experiment and setting X at a few chosen values, e.g. a temperature of the oven at $x=100,200,300,400F$. First, X is not random at all. Second, it’s distribution is not Gaussian. However, the regression can be applied in these type of settings.

The Gaussian assumption, if any, is on the errors of the process, not the regressors. In the simplest case the model is $$y_i=\alpha+\beta x_i+\varepsilon_i$$, where $\varepsilon_i\sim\mathcal N(0,\sigma^2)$, i.e. errors are Gaussian.

The errors are not observed, so we can estimate them as residuals $\hat \varepsilon_i=r_i=y_i-(a+b x_i)$. Then we write the MLE equations and estimate $a,b,\hat\sigma$, that approximate true parameters $\alpha,\beta,\sigma$ that you denoted collectively as $\theta$.

Actually, in many cases we don’t assume that the errors are Gaussian either, it is enough that they are of finite variance and uncorrelated plus some other assumptions. We then show that for large enough sample due to CLT the parameter estimates are Gaussian around true parameters. This way the requirements on the dataset are less restrictive and more realistic. Gaussian iid requirement on errors is often unrealistic.

The Gaussian assumption, if any, is on the errors of the process, not the regressors. In the simplest case the model is $$y_i=\alpha+\beta x_i+\varepsilon_i$$, where $\varepsilon_i\sim\mathcal N(0,\sigma^2)$, i.e. errors are Gaussian.

The errors are not observed, so we can estimate them as residuals $\hat \varepsilon_i=r_i=y_i-(a+b x_i)$. Then we write the MLE equations and estimate $a,b,\hat\sigma$, that approximate true parameters $\alpha,\beta,\sigma$ that you denoted collectively as $\theta$.

Actually, in many cases we don’t assume that the errors are Gaussian either, it is enough that they are of finite variance and uncorrelated plus some other assumptions. We then show that for large enough sample due to CLT the parameter estimates are Gaussian around true parameters. This way the requirements on the dataset are less restrictive and more realistic. Gaussian iid requirement on errors is often unrealistic.

Requiring Gaussian assumption on regressors would make the model very limited in applications. Imagine you are conducting an experiment and setting X at a few chosen values. First, X is not random at all. Second, it’s distribution can be Bernoulli or anything you choose. For instance you set temperature at set levels on the grid x=10, 20, 30, 40, ..., 100F. This is nothing like Gaussian yet the regression can be applied.