1
$\begingroup$

How do I best find a best fit line if my y data is probability functions rather than single points? Obviously, in every regression problem you can include uncertainty on the xs and ys, but I have a dataset where I have a single point for each of my xs but my y data is probabilistic (that is for each x, I have some function representing the likelihood of y being a range of value). I could sample n points from each of the probability functions and then do a least squares line but that seems slow and not quite right. Are there any methods of doing this efficiently/in a particularly good way (I am also interested in any particularly useful metrics or data visualization techniques that are applied to this kind of data--the obvious solutions, like using colors to represent the probability on an x-y plot, which I am using so far don't seem like the easiest to read solution).

As a related question, what if x were not just a single number but rather coordinates (x1,x2,x3). Given probability functions at some set of points, is there a method to find a best fit function over space in general?

$\endgroup$
9
  • 3
    $\begingroup$ How is the distribution specified? $\endgroup$
    – Glen_b
    Aug 12, 2021 at 9:23
  • $\begingroup$ At the moment, its a gaussian mixture model. $\endgroup$
    – Sam H
    Aug 12, 2021 at 9:26
  • 2
    $\begingroup$ So the conditional expectation and variance should be simple enough to calculate $\endgroup$
    – Glen_b
    Aug 12, 2021 at 9:38
  • 1
    $\begingroup$ @Glen_b That use of weights doesn't seem indicated here: although it will work with Normal and approximately Normal conditional distributions, I doubt it is most appropriate for, say, a logistic regression (with linear link). Why not go straight to the MLE? $\endgroup$
    – whuber
    Aug 12, 2021 at 13:34
  • 2
    $\begingroup$ @Glen_b I was offering logistic regression to illustrate why my question is legitimate. The OP immediately indicates they think the least squares line "is not right," so we shouldn't assume that's part of the solution. $\endgroup$
    – whuber
    Aug 12, 2021 at 19:10

1 Answer 1

2
$\begingroup$

Seems like you are in an enviable position as compared to others who want to use regression. Whereas most users of regression would love to know the conditional distributions of $Y$ given $X=x$, you actually have them.

To graph the conditional mean function, simply calculate $E(Y|X=x)$ from your mixtures, for a grid of $X$ values (e.g. from a lower value of $X$ to an upper value, perhaps the 5th and 95th percentiles, with 100 equally spaced points), and plot the results, connecting the points with a smooth curve (or just a line connect will work with a sufficiently dense grid).

While you are at it, you can play the same game with other aspects of the conditional distributions, like standard deviations, medians and quantiles.

As far as use of multiple $X$'s, just set all $X$'s other than the one of interest to a constant (e.g. median), and then draw the same graphs. If there are interactions, use low and high values of the interacting variable, and overlay the plots to see the interacting effect.

You can also see the effects of two $X$'s simultaneously, while holding others constant, by using a 3-D surface plot.

These are the same kinds of graphs that can and should be used for all regressions. In the classic model, they are somewhat unrealistic due to violated assumptions (linearity, normality, homoscedasticity), but the plots still have benefit for assessing how badly violated are the assumptions, and whether remedial actions are needed.

$\endgroup$
2
  • $\begingroup$ This seems like it leaves out information. I know the answer might be "graphics are limited and this is the right way to do things", but if some of the measurements provide no data (that is, they have a probability function that is just a constant) then I don't really want to be using their expected value (which would simply be the midpoint of the sampled range) to constrain my best fit line. Is there a way to account for the entire set of data rather than simply taking these single number metrics and using them to fit a best fit line? $\endgroup$
    – Sam H
    Aug 13, 2021 at 4:14
  • $\begingroup$ Then draw quantile curves, as I suggested, overlayed on the same set of axes. Look into quantile regression to see examples. These show better representations of the conditional distributions than do single number summaries. $\endgroup$ Aug 13, 2021 at 10:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.