3
$\begingroup$

I have a simple linear regression model with the constraint that my dependent variable y (response time) has to be greater than zero. I want to specify priors for intercept, slope and sigma (the standard deviation of observed values around the predictions) but I don't know how those priors should look like. My model, evaluated with any triple taken from the priors, should result in values strictly greater than zero.

I know how to fit such a model using common software such as JAGS or Stan but I am particularly interested in finding appropriate priors. All parameters taken from those priors should result in plausible values for y.

$\endgroup$
4
  • $\begingroup$ There seems to be some confusion there. If your dependent variable is positive, this should be a feature of the sampling model, using for instance a truncated normal,$$y|x\sim\mathcal{N}_+(a+bx,\sigma^2)$$which does not require restrictions on the prior. $\endgroup$
    – Xi'an
    Commented Aug 9, 2015 at 13:13
  • $\begingroup$ Thank you. What I want to do is compute an "average likelihood" by sampling repeatedly parameters from the priors and compute the likelihood. Finally, I compute the mean of all those likelihoods. Imagine I use your proposed sampling model, what happens if I choose parameters for slope and intercept so that my model predicts a value of say -10 with sigma of say 2? $\endgroup$
    – beginneR
    Commented Aug 9, 2015 at 14:03
  • $\begingroup$ Following up: But isn't it the case that priors which would lead to negative model predictions in many cases can be considered as a poor choice? $\endgroup$
    – beginneR
    Commented Aug 9, 2015 at 14:19
  • $\begingroup$ Have you considered applying a logarithmic transformation to your dependent variable? $\endgroup$
    – Cyan
    Commented Aug 10, 2015 at 21:44

2 Answers 2

2
$\begingroup$

The prior is not (yet) at issue: A simple linear regression doesn't appear an appropriate choice for your data. More plainly, that the model could produce negative predictions of positive data shows an improper choice of likelihood.

What it seems you're looking for is a generalized linear model. In brief, a simple linear regression assumes that $y|x \sim N(X\beta, \sigma)$; in words, that $y$ is normally distributed around some linear function of $X$. A generalized linear model treats the data as produced by some other distribution, and transforms this linear function to produce a parameter for that distribution. Logistic regression is a familiar example: The data are Bernoulli generated, with $p = S(X\beta)$, where $S$ is the sigmoid function.

For more on this Andrew Ng's introduction is nicely approachable, and a fuller treatment is available here. For continuous, positive data, typical choices are the gamma and exponential distributions. From Ng's tutorial, emphasis mine:

The multinomial (which we’ll see later), the Poisson (for modeling count-data; also see the problem set); the gamma and the exponential (for modeling continuous, non-negative random variables, such as time intervals); the beta and the Dirichlet (for distributions over probabilities); and many more.

Modeling your data using an appropriate generalized linear model with appropriate priors, rather than a simple linear regression, would solve your problem of negative predictions: All your $y$ values would be positive, because they would all be sampled from a distribution with strictly positive support.

$\endgroup$
1
$\begingroup$

If you need to limit the responses to positive values, you can construct a "tobit" model with the survreg function in Therneau's survival package for R. There is a worked example in the help page for ?survreg. This comes about because censoring and truncation have some commonalities. I realize this isn't the Bayesian answer you were looking for but it might be useful to have a frequentist alternative. It may also be helpful to use "tobit" as an additional search term.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.