Setting Multiple linear restrictions equal to some coefficient in R?

Question

I have a model of the form: $y_i = \beta_0 + \beta_1 x_1 +\beta_2 x_2 +\epsilon_i$ . I want to test the null hypothesis that $\beta_1 + \beta_2 =2$, by creating a restricted model imposing these restrictions. My idea was to create a new variable, call it $restrictions$ and make it the equation $\beta_1 + \beta_2 =2$ and then input this equation into a restricted regression, call it R,

R <- lm(y_i ~ restrictions, data = mydata)

This was just what occurred to me, I have never dealt with multiple linear restrictions equal to something non-trivial such as zero, which would make this a much more straightforward affair as I would be able to just exclude them, e.g. $\beta_1=0$

I also want to be able to fit my restrictions into the general form of $R \beta = r$ where R is a qx(k+1) with rank(R)=q<k+1 and r is qx1, and q is the number of restrictions.

Answer 1

There are two ways of doing this. One is to incorporate each kind of restriction using algebra, and the other is to derive a general formula into which specific cases can be fitted.

Let's start with the first. Your constrained model is $$y_i = \beta_0 + \beta_1 x_1 +\beta_2 x_2 +\epsilon_i \mbox{ where } \beta_1 + \beta_2 =2.$$

This can be rewritten as

$$y_i = \beta_0 + \beta_1 x_1 +(2-\beta_1) x_2 +\epsilon_i,$$

which is equivalent to

$$y_i - 2 x_2 = \beta_0 + \beta_1 (x_1 - x_2) +\epsilon_i.$$

This is the model you can fit after defining 2 new variables. To recover $\hat \beta_2$, you just need to calculate the linear function of $2 - \hat \beta_1$.

Now for the general solution for a linear equality constraint of the form $R\beta=r$, where $R$ is a $q \times k$ matrix of known constants, with $q<k$; r is a $q-$vector of known constants; $k$ is the number of variables in the model (including intercept) and $q$ is the number of restrictions. R and r constants come from the restrictions you want to impose. For example, in your simple model

$$R = \begin{bmatrix} 0 & 1 & 1 \end{bmatrix},\mbox{ } r=2, \mbox{ and } q=1.$$

To impose the restriction, we define a constrained sum of squares $$RSS=(y-Xb^*)'(y-Xb^*) - 2 \lambda'(Rb^*-r),$$ where $\lambda$ is a $q-$vector of Lagrange multipliers. From setting the FOCs with respect to $b^*$ and $\lambda$ to zero, you can derive that

$$b^*= b + (X'X)^{-1}R'[R(X'X)^{-1}R']^{-1}(r-Rb)$$

where $b$ is the usual OLS estimator $(X'X)^{-1}X'y$ and $b^*$ is the constrained coefficient vector.

In R, this can be done with glmc which will allow you to specify the constraint(s) and handles the rest.

Answer 2

By using the R bimets package you can activate restrictions directly into the model definition, so you can modify (or remove) the imposed linear restriction ( i.e. $\beta_1+\beta_2=2$ ) without changing the regression equation or code.

More details about the package at the rstudio blog: https://rviews.rstudio.com/2021/01/22/sem-time-series-modeling/

An example follows:

library(bimets)

#define the restricted model
model.txt <- "MODEL
BEHAVIORAL> Y
EQ> Y = b0 + b1*X1 + b2*X2
COEFF> b0 b1 b2
RESTRICT> b1 + b2 = 2
END"

#create regressors
modelData <- list(
  X1=ts(1:10,start=c(2000,1),frequency=1),
  X2=ts(sin(1:10),start=c(2000,1),frequency=1)
)

#create dependant
modelData <- within(modelData,
                 Y <- X1 + X2 + rnorm(1:10,0,1))

#load model and model data
model <- LOAD_MODEL(modelText = model.txt)
model <- LOAD_MODEL_DATA(model,modelData)

#estimate the model
model <- ESTIMATE(model,TSRANGE=c(2000,1,2009,1))

## Estimate the Model model.txt:
## the number of behavioral equations to be estimated is 1.
## The total number of coefficients is 3.
## 
## _________________________________________
## 
## BEHAVIORAL EQUATION: Y
## Estimation Technique: OLS
## 
## Y                 = -   0.1007733   
##                         T-stat. -0.2454512  
## 
##                     +   1.04065     X1
##                         T-stat. 15.65087    ***
## 
##                     +   0.9593502   X2
##                         T-stat. 14.42816    ***
## 
## RESTRICTIONS:
## b1+b2=2
## 
## RESTRICTIONS F-TEST:
## F-value            : 0.02947574     
## F-prob(1,7)        : 0.868543       
## 
## 
## STATs:
## R-Squared                      : 0.9635383   
## Adjusted R-Squared             : 0.9589806   
## Durbin-Watson Statistic        : 2.844719    
## Sum of squares of residuals    : 3.327863    
## Standard Error of Regression   : 0.6449674   
## Log of the Likelihood Function : -8.688112   
## F-statistic                    : 211.4085    
## F-probability                  : 4.904992e-07
## Akaike's IC                    : 23.37622    
## Schwarz's IC                   : 24.28398    
## Mean of Dependent Variable     : 5.758183    
## Number of Observations         : 10
## Number of Degrees of Freedom   : 8
## Current Sample (year-period)   : 2000-1 / 2009-1
## 
## 
## Signif. codes:   *** 0.001  ** 0.01  * 0.05  
## 
## 
## ...ESTIMATE OK

#test real coefficents restriction
print(model$behavioral$Y$coefficients[2] + model$behavioral$Y$coefficients[3])
## [1] 2

In order to test and to perform a regression with a different restriction you can easily change the RESTRICT> line, e.g.:

RESTRICT> b1 - 2.34*b2 + 0.99*b0 = -12

The package does not use the R formula function.