How to introduce a-priori knowledge about the factors in linear regression? There has been a lot of research about weighted linear regression. Most of the discussions are about weight of samples though, i.e., weight on $y$, rather than weight on $X$,
\begin{align}
X &= [X_{0}, X_{1}, ... X_{n}]  \\
y &= \beta X 
\end{align}
In most engineering problems, there exist some a-priori knowledge about the factors $X_{i}$. We know that a few factors are 'primary' factors, with strong business logic reasons. For instance, house price ($y$) and median income of zip-code.
I am wondering if there is an approach similar to 'weight of factors' in the regression model, which can introduce above a-priori knowledge.
This should be very useful when we deal with multicollinearity. For a set a correlated factors, we can give higher weight to the factor that we know is important. However, such a-priori knowledge is not really a distribution about $\beta$, but just a weight
Can anyone give any insights here?
 A: As @DeltaIV hinted in his or her comment, Bayesian methods are a natural fit for this. Using linear regression in a Bayesian fashion means treating the parameters (viz., the coefficients and the SD of the error term) as random variables. If you think a certain coefficient is more likely to have certain values, you can choose a prior distribution for that random variable that puts more weight on the values in question.
A: I'm not aware of any method which takes hints like "Variable X1 is more important than Variable X3".
As the comments to the OP reflect, this isn't straightforwardly a Bayesian regression issue, since you're interested in weighting the inclusion of variables, not (directly) in weighting possible values of the coefficients. 
Three thoughts on what could be done:


*

*Use Bayesian Regression and give your preferred variables the usual priors, while giving non-preferred variables sparsity-inducing priors such as the Horseshoe. I don't know if this would actually work, but the idea would be to give the non-preferred variables more opportunity to have their coefficients be driven to zero.

*Modify a LARS-style regression to weight the angles inversely-proportionally to whether they are favored variables or not.

*Modify a stepwise regression (I don't like stepwise, but it would be easier to modify) such that it prefers to add the preferred variables first. Or perhaps starts with the preferred variables in and then adds or subtracts the other variables in the steps. (See this other posting.)
