0
$\begingroup$

I'm a little confused about how to specify a regression model in Bayesian terms. I understand that for the simple case of plain linear regression, the response variable is normally distributed about the mean, which varies linearly, like so: $$ Y \sim N(\beta X, \sigma) $$ where $\beta$ and $\sigma$ are drawn from priors and an intercept can also be added.

My data (sample shown below) are roughly exponentially shaped. Values can be zero, but not negative. My first impulse was to try a model with $Y \sim N(\lambda e^{-\lambda X},\sigma)$, but I'm not sure if this makes sense and, anyway, it would allow for negative values but shouldn't.

Ultimately, my goal is to estimate the variance along this curve, getting a distribution as a function of the variable on the horizontal axis.

Is there a standard approach?

enter image description here

$\endgroup$
4
  • $\begingroup$ why are the circles filled with different colours and have different diameters? Do you perhaps have more than one feature? $\endgroup$
    – utobi
    Nov 4, 2022 at 10:14
  • 1
    $\begingroup$ Yes I do. Markers are colored by the same feature as the horizontal axis, so that's no issue. Marker sizes represent another feature, but I don't necessarily need to include it in the model. $\endgroup$
    – qsfzy
    Nov 4, 2022 at 15:28
  • $\begingroup$ Great. Another thing, you are asking about two different things (1) Bayesian regression, (2) regression for the variance. In (1) Iguess you want to model the mean. Do you have any prior information? Is there any motivation about (2) ? $\endgroup$
    – utobi
    Nov 4, 2022 at 16:26
  • 1
    $\begingroup$ Yes I want to model the mean and estimate the variance as a function of the same feature on the horizontal axis. I know that the response variable can never be negative. $\endgroup$
    – qsfzy
    Nov 4, 2022 at 18:16

1 Answer 1

2
$\begingroup$

You can always apply transformations to your x and y values before doing Bayesian Linear Regression. This looks a lot like power-law to me, so I would try taking the logarithm of both x and y. Here the assumption is that y is log-normally distributed for a given x. If that isn't enough you can first transform your y data to cover the entire reals and then apply a Gaussian Processes, which assumes that the transformed ys are joint normally distributed for a set of xs.

$\endgroup$
4
  • $\begingroup$ Ok that makes sense. I'm still in the process of understanding what particular distribution Y assumes under these transformations. $\endgroup$
    – qsfzy
    Nov 3, 2022 at 14:08
  • $\begingroup$ For Bayesian Linear Regression and Gaussian Processes you always assume that your ys are in a multivariate normal distribution for a set of xs when applying the model (so after the transformation). To check whether such a transformation even exists, you can potentially check whether copula of different Ys is Gaussian. $\endgroup$
    – Jannis
    Nov 3, 2022 at 16:02
  • $\begingroup$ It might also be interesting to check out these papers: papers.nips.cc/paper/2003/hash/… proceedings.neurips.cc/paper/2010/file/… $\endgroup$
    – Jannis
    Nov 3, 2022 at 16:03
  • $\begingroup$ Does it make sense to use a model in the form: $\log Y \sim N(\beta X + \alpha, \gamma X)$ where $\beta$, $\alpha$, and $\gamma$ are the parameters to estimate $\endgroup$
    – qsfzy
    Nov 3, 2022 at 20:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.