0
$\begingroup$

Suppose I am interested in studying the relationship between the age of giraffes in relation to the height and weight of giraffes. Given some measurements on the age, height and weight of giraffes - the most common approach would be to create a regression model age ~ f(height, weight) . For some height and weight measurements, I could predict the age of a giraffe having these measurements.

However, I could also fit a probability distribution function to my data. For instance, I could try to fit a joint multivariate normal probability distribution : P(height, weight, age). Doing so, I think you can answer more questions (relating to statistical inference) compared to the regression model. For example:

1) As in the regression model, we can find out the "most probable age" of a giraffe measuring 17ft and weighing 2000 lbs. This would involve randomly sampling the conditional distribution of the joint multivariate distribution - and then taking the average value of these samples.

enter image description here

2) Unlike the regression model, I can now find out the "most probable age" of a giraffe measuring 17ft and weighing more than 2000 lbs. This would involve adjusting the conditional distribution from 1) and taking the average of random samples (e.g. MCMC) from this conditional distribution.

3) Unlike the regression model, I can also perform conditional inference on the covariates in the probability distribution. For example, given a giraffe is taller than 17ft and between 20-25 years old : what is the most probable weight of this giraffe?

4) And finally, (unlike the regression model) I can also calculate probabilities - for example: What is the probability of observing a giraffe that is younger than 20 years and weighs more than 2000 lbs?

enter image description here

My Question: Can someone please tell me if the points I have listed above can truly be answered using probability distributions (and not answered using regression models) - and if they are in fact "relevant" in statistical research studies? (I imagine that fitting probability distributions to data can be more computationally expensive compared to regression models, and more difficult to explain to non-statisticians.)

Thanks!

Note: If you decide to fit a probability distribution to some data and decide to place priors on the mean, variances and covariances of the model parameters - in some sense, this is analogous to regularization terms in regression models.

Improve this question
$\endgroup$
6
  • 2
    $\begingroup$ Regression models are intertwined with distribution functions. If I model $Y = f(X)$ I am also modelling the conditional expectation $\mathbb{E}[Y|X]$. $\endgroup$
    – Galen
    Nov 9, 2021 at 18:24
  • $\begingroup$ @ Galen: thank yoy for your reply! this is a great observation! but I don't think that regression models can answer questions about "probabilities"? E.g. What is the probability of observing a giraffe that is younger than 20 years and weighs more than 2000 lbs? Thank you! $\endgroup$
    – stats_noob
    Nov 9, 2021 at 18:31
  • 2
    $\begingroup$ You can always represent the regression model as a model for the conditional distribution of Y given the Xs. That is the point of this book: routledge.com/… $\endgroup$
    – BigBendRegion
    Nov 9, 2021 at 18:42
  • $\begingroup$ Thank you! I will look into this book! $\endgroup$
    – stats_noob
    Nov 9, 2021 at 18:46
  • $\begingroup$ In general: is it easier to do this with probability distribution functions compared to regression models? do probability distribution models have the potential to capture more information and complex patterns in big data compared to regression models? are the 4 points i raised "relevant" in the field of statistical inference? thank you! $\endgroup$
    – stats_noob
    Nov 9, 2021 at 18:47

1 Answer 1

Reset to default
2
$\begingroup$

What you are describing with probability distributions is called joint modeling and can be expressed using a regression model. This is the same machinery used to model repeated measures in, say, a clinical trial. The clinical trial panel data represents measuring the same endpoint on the same subjects longitudinally. In your case you are measuring different endpoints on the same giraffes cross-sectionally. In both cases, though, we have correlated measurements.

Bradley Efron and others have written about the idea of division of labor. Rather than building an all-encompassing model it is more practical to build separate models to address different questions. If each giraffe measurement follows a different type of probability model it may become unwieldy to jointly model all three distributions, so we might use semi-parametric methods to jointly model just the means. If we wanted to condition the joint model on two of the endpoints to explain a third endpoint we could fit a separate standard issue regression model for this task. Since this is a univariate model with multiple regressors we could opt for a parametric model that describes the entire endpoint distribution rather than focusing exclusively on the mean. While there are certainly efficiency gains in terms of smaller standard errors by jointly modeling endpoints, it is by no means a requirement. Simply modeling each endpoint separately still allows you to make estimated probability/proportion statements about the population, form tolerance intervals that cover the true population proportions, and construct prediction intervals for predicting future samples. The coverage rates of such intervals would be on a per-endpoint basis, rather than a joint coverage rate. Keeping in mind that all of this has to eventually be disseminated to non-statisticians the idea of division of labor is not just a pragmatic matter for the analyst.

Improve this answer
$\endgroup$
9
  • $\begingroup$ Thank you so much for your answer! I will have to read more about this topic "the division of labor". But do you think that creating a joint probability model has any advantages than the "division of labor" strategy that you described? Perhaps in the case of high dimensional data - during the time of Bradley Efron, it would have been computationally infeasible to fit high dimensional probability distributions to the data, and "the division of labor" strategy would have been more pragmatic? But now with modern computers, is fitting a joint probability distribution to the data an "ok" strategy"? $\endgroup$
    – stats_noob
    Nov 9, 2021 at 19:06
  • 1
    $\begingroup$ Neither approach is bad or wrong. Exploring joint models certainly sounds like a fun task for someone interested in probability and statistics and I would not discourage it. The advantage I see for joint modeling is having slightly smaller standard errors when constructing p-values or confidence intervals. You mentioned priors in your question. If you are using Bayesian methods then jointly modeling will have a similar effect of tightening the credible intervals compared to modeling separately. I say explore it all and have fun. $\endgroup$
    – Geoffrey Johnson
    Nov 9, 2021 at 19:12
  • $\begingroup$ @ Geoffrey Johnson: Thank you for your reply! Another point that I constantly think of : suppose we wanted to choose the bayesian modelling approach. if we have access to historical data about giraffes (e.g. mu and sigma for height) - it is easier to use this historical data when defining priors in a joint probability distribution compared to a regression model. in a joint probability model, the priors are on mu and sigma for each variable (which correspond to real world data). in a bayesian regression model, we define priors on the beta regression coefficients, and it is less clear $\endgroup$
    – stats_noob
    Nov 9, 2021 at 19:29
  • $\begingroup$ and it less clear how to take the real world historical data we have and define priors on the bayesian regression model (compared to a probability distribution function). joint probability distribution models might be able to naturally make better use of real world historical information compared to regression models when defining bayesian priors. I posted a detailed question about this in the past - maybe this might interest you: stats.stackexchange.com/questions/549786/… :) $\endgroup$
    – stats_noob
    Nov 9, 2021 at 19:29
  • 1
    $\begingroup$ I will take a look. $\endgroup$
    – Geoffrey Johnson
    Nov 10, 2021 at 3:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.