I often see regression expressed in two ways.
- The covariates are random: In this scenario, we have $(x_i,y_i) \sim G$ for some distribution $G$ and are i.i.d. for $i = 1, \cdots, n$. We then posit $y_i \mid x_i \sim A$ for some distribution $A$, and we want to successfully parameterize $A$ using maximum likelihood or some other method.
- The covariates are fixed: In this scenario we have associated points $(x_i, y_i)$. We assume $y_i \sim A$ for some distribution $A$, and we assume $x_i$ parameterizes the distribution $A$ in some way. We then use maximum likelihood to find the best values of the other parameter(s) of $A$.
My question is, is this distinction between random covariates $x_i$ or non-random covariates $x_i$ important? At the end of the day it seems that $x_i$ figures in the pmf/pdf of our distribution $A$ anyhow.
