The linearity condition states that $y_i=(\vec{x}_i)^{T}\vec{\beta}$$\mathbb{E}[y_i]=(\vec{x}_i)^{T}\vec{\beta}$ for all $i$. Now, if we have fixed regressors, $\{\vec{x}_1,\vec{x}_2,\cdots\}$, our linearity condition only says for those $\vec{x}_i$, there is a linear relationship. So it does not assume linearity when we have some $\vec{x}_j$ that is not in the set of fixed regressors.
However, if the regressors are random as well, the linearity condition says for all $\vec{x}$ there are, linearity holds. Therefore, if we have an experimental design and had fixed regressors, we can only infer $\vec{x}$ that has happened in the fixed regressor during the experiment and cannot infer otherwise. However, with stochastic regressors, we can infer any $\vec{x}$. Is that right?