A regression model (whether linear or polynomial) is said to be represented as hyperplane through the data space:

On the other hand, a probability distribution model fits an entire distribution through the data space:

My Question: Compared to the hyperplanes generated by regression models, do probability distributions inherently have the ability to capture more information and make more informative inferences about the same data? Of course, this would depend on how well both types of model fit the same data - but in general, can such a conclusion be made comparing the expressiveness of both models? Or is this simply not true?

Thanks!

• "Hyperplanes" and "distributions" are not comparable. However, most statistical regression procedures estimate the full distribution of the conditional responses.
– whuber
Nov 24, 2021 at 14:54
• @ Whuber: thank you for your reply! I had a question about your comment "statistical regression procedures estimate the full distribution of the conditional responses" : in this context, since a regression model is technically a conditional distribution P(y|x1, x2, .. xn) : should a regression model be considered as a probability distribution or a hyperplane? In what context do we interpret a statistical model as a hyperplane? Thank you! Nov 24, 2021 at 15:14
• When the model supposes some predefined function of the location parameter of the conditional distribution is a linear combination of $x_1, \ldots, x_n,$ it both determines a hyperplane (the locus of possible locations) and a joint distribution of the responses. This is the standard setting of ordinary least squares regression (OLS; the predefined function is the identity) and of the generalized linear model (GLM; the predefined function is called the "link function").
– whuber
Nov 24, 2021 at 15:25
• @ Whuber: thank you for your reply! just to clarify - when creating a regression model y = b_o + b_1*x1 + b_2 *x_2 : in the frequentist setting, there is no multivariate probability distribution associated with this model? Nov 24, 2021 at 22:11
• There certainly is! The model, fully written out, is $$y_i = \beta_0 + \beta_1x_{1i} + \beta_2 x_{2i} + \varepsilon_i$$ and the standard minimal assumptions are that the vector $E=(\varepsilon_1, \ldots, \varepsilon_n)$ has a multivariate distribution with zero mean and covariance matrix $\sigma^2\mathbb{I}_n$ ($\sigma$ is a "nuisance parameter" in the model). Note that these are assumptions about the distribution of $Y=(y_1,\ldots, y_n)$ conditional on the design matrix (of values of the explanatory variables).
– whuber
Nov 24, 2021 at 22:20