Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
Suppose I'm attempting to use linear regression but instead of
$$y_i = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3 + \beta_4 x_4 + \varepsilon_i,$$
for scientific reasons, I know that $β_2=\alpha\beta_1$ and $β_4=\alpha\beta_3$. So my formula is actually:
$$y_i = \beta_0 + \beta_1 x_1 + \alpha\beta_1 x_2 + \beta_3 x_3 + \alpha\beta_3 x_4 + \varepsilon_i$$
Note that the variable $\alpha$ is the same in both equations but unknown. How should I tackle something like this?
Since $\alpha$ is unknown, this reduces to a nonlinear regression with the model form:
$$y_i = \beta_0 + \beta_1 [x_1 + \alpha x_2] + \beta_2 [x_3 + \alpha x_4] + \varepsilon_i.$$
You can implement nonlinear regression using the nls function in R. For a nonlinear model you will need to specify a starting point for your parameters in order to implement the iterative fitting-method. A reasonable starting point for the parameters in this case could be obtained by fitting the unconstrained linear model (your first model) and then taking the starting values for the parameters to be:
$$\begin{align} \beta_0^* &= \hat{\beta}_0, \\[12pt] \beta_1^* &= \hat{\beta}_1, \\[12pt] \beta_2^* &= \hat{\beta}_3, \\[6pt] \alpha^* &= \frac{\hat{\beta}_2+\hat{\beta}_4}{\hat{\beta}_1+\hat{\beta}_3}. \\[6pt] \end{align}$$
(The values on the right-hand-side in these equations would be estimates from your first model.)
I am thinking of two different ways with different types of analysis.
A) you simply run the regression as if they are no relationship. In R, you can use model = lm(y ~ x1 + x2 + x3 + x4). Once you get the coefficient values from summary(model) or coef(model), you use linear equations, where:
$$
\hat{\beta_2}=\hat{\alpha}\hat{\beta_1}
$$
OR
$$
\hat{\beta_4}=\hat{\alpha}\hat{\beta_3}
$$
The only concern I have is that both values of $\hat{\alpha}$ is no similar.
B) Imposing conditions as you stated. I am not sure how to run this in R but I think Solver in Excel can help you.
My question to you: How does the coefficient has relationship with one another? I usually find the case where the "so-called" independent variables are actually not independent, meaning $x_1=f(x_2)$ and $x_3=f(x_4)$
This is essentially a non-linear least squares problem, and the R function nls may be of interest to you. Rather than describe the workings of the function though, let me briefly outline the relevant statistical theory.
The basic idea behind non-linear least squares is as follows. You have some regression function $f(X;\theta)$ which takes predictors $X$ and parameters $\theta$ and predicts the value of the outcome $y$. In particular, the assumption is that $\mathbb E[y|X] = f(X;\theta)$ for some $\theta$ in your case, $\theta = (\beta_1,\beta_3,\alpha)$ and $f(x_1,x_2,x_3,x_4;\theta) = \beta_1(x_1+\alpha x_2) + \beta_3(x_3 + \alpha x_4)$. The fact that $f(X;\theta)$ is the conditional expectation of $y$ given $X$ implies that in population, we have that the true value of the parameter, $\theta_0$ satisfies $$\theta_0 = \mathrm{argmin}_\theta\, \mathbb E[(y - f(X;\theta))^2]$$ The fact that $\theta_0$ solves this optimization problem immediately implies an estimation strategy: simply choose $\hat\theta$ to be the value of the parameter that minimizes the in-sample analogue of the above objective function, that is $$\hat\theta = \mathrm{argmin}_\theta\, \frac1n\sum_{i=1}^n (y_i - f(X_i;\theta))^2$$ This is the reason for @wzbillings's suggestion above to use the R optim function.
If you are interested in how, given this estimate, you should compute standard errors, you can consult previous posts, such as this one.
optim()in R. $\endgroup$