3
$\begingroup$

I wonder if there some existing work of Linear Regression or Logistic Regression with partially known coefficients ($\beta$).

For a linear regression, $Y=X\beta$, when we already have knowledge about some coefficient of $\beta$, say $\beta_{0,...,j}$ and still need to solve for $\beta_{j,..., p}$.

This should be trivial for linear regression case since we can directly deduct the known part out of $Y$, but how about Logistic Regression?

And what if we don't know the actual values of $\beta_{0,...,j}$, but only know that they should be non-zero.

I believe these could be solved with some constrained optimization, so my question is that if there are some published manuscripts that have studied this problem? I wonder because I am interested in whether there are some interesting theoretical aspects of this more than a constrained optimization problem.

$\endgroup$
1
$\begingroup$

There are a few options.

If analyzing data using R (other systems may have the same functionality) you can include an offset in a formula for setting a know relationship as part of the formula.

For linear regression, minimizing the squared residuals is just a quadratic programming problem, so adding a non-negative constraint is straight forward if you have a quadratic programming tool.

You can fit the unconstrained model, then if any of the slopes are negative you set them to 0 (leave that term out of the model) then refit. Inference from this method can be a bit iffy, you should do something like bootstrapping the whole process to get the inference rather than trusting the final model.

You can use a non-linear regression model tool and use exp(beta)*x to force a positive slope.

You can use a Bayesian model and choose a prior for your slopes that restrict the possible values, e.g. use a gamma or half-normal prior on slopes that you want to restrict to be non-negative.

The Bayesian approach is probably the most general and lets you include more prior information than just something needs to be non-negative. But also remember that signs on coefficients in multiple regression cases can be non-intuitive and exploring why a sign is different from what you expect may gain you more insight than just forcing it to match preconceptions.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.