Best fit line with probability data

Question

How do I best find a best fit line if my y data is probability functions rather than single points? Obviously, in every regression problem you can include uncertainty on the xs and ys, but I have a dataset where I have a single point for each of my xs but my y data is probabilistic (that is for each x, I have some function representing the likelihood of y being a range of value). I could sample n points from each of the probability functions and then do a least squares line but that seems slow and not quite right. Are there any methods of doing this efficiently/in a particularly good way (I am also interested in any particularly useful metrics or data visualization techniques that are applied to this kind of data--the obvious solutions, like using colors to represent the probability on an x-y plot, which I am using so far don't seem like the easiest to read solution).

As a related question, what if x were not just a single number but rather coordinates (x1,x2,x3). Given probability functions at some set of points, is there a method to find a best fit function over space in general?

Answer 1

Seems like you are in an enviable position as compared to others who want to use regression. Whereas most users of regression would love to know the conditional distributions of $Y$ given $X=x$, you actually have them.

To graph the conditional mean function, simply calculate $E(Y|X=x)$ from your mixtures, for a grid of $X$ values (e.g. from a lower value of $X$ to an upper value, perhaps the 5th and 95th percentiles, with 100 equally spaced points), and plot the results, connecting the points with a smooth curve (or just a line connect will work with a sufficiently dense grid).

While you are at it, you can play the same game with other aspects of the conditional distributions, like standard deviations, medians and quantiles.

As far as use of multiple $X$'s, just set all $X$'s other than the one of interest to a constant (e.g. median), and then draw the same graphs. If there are interactions, use low and high values of the interacting variable, and overlay the plots to see the interacting effect.

You can also see the effects of two $X$'s simultaneously, while holding others constant, by using a 3-D surface plot.

These are the same kinds of graphs that can and should be used for all regressions. In the classic model, they are somewhat unrealistic due to violated assumptions (linearity, normality, homoscedasticity), but the plots still have benefit for assessing how badly violated are the assumptions, and whether remedial actions are needed.