linear regression with partially known coefficients

Question

I wonder if there some existing work of Linear Regression or Logistic Regression with partially known coefficients ($\beta$).

For a linear regression, $Y=X\beta$, when we already have knowledge about some coefficient of $\beta$, say $\beta_{0,...,j}$ and still need to solve for $\beta_{j,..., p}$.

This should be trivial for linear regression case since we can directly deduct the known part out of $Y$, but how about Logistic Regression?

And what if we don't know the actual values of $\beta_{0,...,j}$, but only know that they should be non-zero.

I believe these could be solved with some constrained optimization, so my question is that if there are some published manuscripts that have studied this problem? I wonder because I am interested in whether there are some interesting theoretical aspects of this more than a constrained optimization problem.

Answer 1

There are a few options.

If analyzing data using R (other systems may have the same functionality) you can include an offset in a formula for setting a know relationship as part of the formula.

For linear regression, minimizing the squared residuals is just a quadratic programming problem, so adding a non-negative constraint is straight forward if you have a quadratic programming tool.

You can fit the unconstrained model, then if any of the slopes are negative you set them to 0 (leave that term out of the model) then refit. Inference from this method can be a bit iffy, you should do something like bootstrapping the whole process to get the inference rather than trusting the final model.

You can use a non-linear regression model tool and use exp(beta)*x to force a positive slope.

You can use a Bayesian model and choose a prior for your slopes that restrict the possible values, e.g. use a gamma or half-normal prior on slopes that you want to restrict to be non-negative.

The Bayesian approach is probably the most general and lets you include more prior information than just something needs to be non-negative. But also remember that signs on coefficients in multiple regression cases can be non-intuitive and exploring why a sign is different from what you expect may gain you more insight than just forcing it to match preconceptions.