$X$ fixed in a linear regression Let $y=x^t\beta+e$ be a linear regression where $x$ is fixed and $e$~$N(0,I\sigma^2)$. What are the consequences of having fixed regressors instead of stochastic regressors? How does the estimation or inference of the regression change?
 A: The first difference is conceptual. The idea of fixed regressors is usually traced back to developments in statistics for experiments. In a highly controlled trial, regressors can sometimes be fixed, chosen, or controlled by the experimenter and are thus not stochastic. In domains such as social sciences, or really most observational sources of data, it is often not realistic to think of regressors as being fixed quantities, they are typically realizations from probabalistic processes and just as random as the outcome variables.
Now when does this matter, do you need to treat your estimation different in one case or the other?
There are two perspectives that I think can be useful to bear on this question.

*

*Conditioning on stochastic regressors.

You may be familiar with standard results from OLS, such as conditions for unbiasedness, consistency, and BLUE (Gauss-Markov theorem). Some introductory texts show these results under the assumption that the regressors are fixed.
Take unbiasedness as an example
Treating $X$ as fixed, we can prove that $\hat{\beta} = (X^TX)^{-1}X^TY$ is unbiased as long as $E[e] = 0$.
\begin{align}
E[\hat{\beta}] &= E[(X^TX)^{-1}X^TY]\\
&= E[(X^TX)^{-1}X^T(X\beta + e)]\\
&= \beta + E[(X^TX)^{-1}X^Te]\\
&= \beta + (X^TX)^{-1}X^TE[e]
\end{align}
When the regressors are fixed, they can be taken outside of the expectation. However, we could prove unbiaseness in the stochastic case with the assumption that $E[e|X] = 0$. This says given the information in the random variable $X$, the error is mean 0. See
$E(e|X)=0$ assumption implies that the linear regression model is correctly specified?.
We can similarly show consistency using some conditioning arguments and there is a version of the Gauss-Markov theorem, sometimes called the conditional gauss-markov theorem, that states if $E[e|X]$ and $E[ee^T|X] = \sigma^2I$ that OLS is more efficient than any other linear unbiased estimator for $\beta$.
It turns out that with linear regression, many of the properties that we want are preserved when the regressors are stochastic by making assumptions that are conditional or by translating the results to conditional forms (See Davidson and Mackinnon for example, but many books derive all of their results this way). The conditioning can roughly be thought of as taking the information in $X$ as given and loosely once we condition on this information we are back conceptually closer to the experimental case where things are fixed.


*Nuisance parameter

This useful perspective comes from the semi-parametric literature. The idea is that sometimes we can partition all of the parameters which generated our data into two parts, the first part is the part we care about making inferences about. In linear regression, this is $\beta$. The second part we don't care about in itself, but we will have to account for in some way to correctly make inferences about $\beta$ called the nuissance parameter, denoted $\eta$. (See Tsiatis (2006) on Restricted Moment Estimators for a deeper look at this)
The observed data that we get in the linear regression example are the vectors $(Y,X)$. And we want to infer something about $\beta$.
We can break the joint density of these observations, $p_{Y,X}$ into the following:
\begin{align}
P_{Y,X}(y,x,|\beta,\eta) = P_{Y|X}(y|x,\beta, \eta_1)P_X(x|\eta_2)
\end{align}
Where we break the nuissance parameter into two parts $(\eta = (\eta_1, \eta_2))$, one that is indexes the conditional distribution of $Y|X$ ($\eta_1$) and the other which indexes the distribution of $X$ ($\eta_2$).
Notice that if we were to find the maximum likelihood solution for $\beta$ of this joint density, that it would have nothing to do with $P_X(x|\eta_2)$, since this part is not a function of $\beta$. In other words, it wouldn't matter what the distribution of $X$ is, it could be completely arbitrary but we could still make inference about $\beta$.
In this particular case, we do have to place restrictions on $\eta_1$ or the distribution of $P_{Y|X}$. In Tsiatis (2006), it is shown that for a broader class of models which encompasses linear regression that the necessary restrictions are that error distribution given X satisfies $E[e|X] = 0$, just like above. Thus we could estimate the thing we care about $\beta$ treating the regressors as random quantities with distributions and in the is case we wouldn't even have to make any assumptions about what that distribution looks like.
Your question was about linear regression and to understand stochastic regressors in this context, the first point about conditioning is more or less sufficient. However, if you want to go deeper into the question of when or how does the regressors being stochastic change our approach to estimation and inference in a broader class of models, then nuissance parameters is the way forward.
A: I guess this would be equivalent to $y=\mu+e$, where $\mu$ is the sample mean.
