# Regression specification: what if one regressor is function of another

Consider the following regression model

$$Y_i=D_{i}(\alpha_1+\beta_1 X_i)+(1-D_i)(\alpha_2+\beta_2 X_i)+\epsilon_i$$ where $$D_i$$ is a binary variable. Suppose the researcher has an i.i.d. sample $$\{Y_i, X_i, D_i, Z_i\}_{i=1}^n$$ where $$Z_i$$ is an instrument such that $$E(Z_i^\top \epsilon_i)=0$$.

One way to estimate $$\theta\equiv (\alpha_1,\beta_1,\alpha_2,\beta_2)$$ is by GMM. In particular, note that we can write $$E(Z_i^\top\underbrace{(Y_i-D_{i}(\alpha_1+\beta_1 X_i)+(1-D_i)(\alpha_2+\beta_2 X_i))}_{g(Y_i, D_i, X_i; \theta)})=0$$ We assume identification, i.e., the true parameter $$\theta_0$$ is the unique minimizer of $$E(Z_i^\top g(Y_i, D_i, X_i; \theta) )^\top W E(Z_i^\top\ g(Y_i, D_i, X_i; \theta))$$ where $$W$$ is some positive definite matrix. The GMM estimator $$\hat{\theta}$$ is the solution of $$(1) \quad \min_{\theta}(\frac{1}{n}\sum_{i=1}^nZ_i^\top g(Y_i, D_i, X_i; \theta) )^\top W_n \frac{1}{n}\sum_{i=1}^n Z_i^\top\ g(Y_i, D_i, X_i; \theta)$$

Question: Now, suppose that each $$D_i = f_i(\theta)$$ is a parametric function of $$\theta$$ and I denote it as $$D_i(\theta)$$. Does the fact that I essentially observe $$D_i(\theta_0)$$ in the data (under correct model specification) allow me to solve (1) instead of $$(2) \quad \min_{\theta}(\frac{1}{n}\sum_{i=1}^nZ_i^\top g(Y_i, D_i(\theta), X_i; \theta) )^\top W_n \frac{1}{n}\sum_{i=1}^n Z_i^\top\ g(Y_i, D_i(\theta), X_i; \theta)$$ ? If not, which are assumptions are needed for being justified to solve simply (1)?

Consider the problem at the population level. Let $$\theta_0\in \mathbb{R}^4$$ be the true parameter value.

(i) The orthogonality condition is $$(3) \quad E\Big[Z_i^\top \Big(Y_i- f_i(\theta_0)(\alpha_{1,0}+\beta_{1,0} X_i)+(1-f_i(\theta_0))(\alpha_{2,0}+\beta_{2,0} X_i)\Big)\Big]=0$$ Given that $$D_i$$ that we observe is indeed $$f_i(\theta_0)$$ (under correct model specification), (3) is equivalent to $$(4) \quad E\Big[Z_i^\top \Big(Y_i- D_i(\alpha_{1,0}+\beta_{1,0} X_i)+(1-D_i)(\alpha_{2,0}+\beta_{2,0} X_i)\Big)\Big]=0$$ Correct?

(ii) The identification condition is $$(5) \quad E\Big[Z_i^\top \Big(Y_i- f_i(\theta)(\alpha_{1}+\beta_{1} X_i)+(1-f_i(\theta))(\alpha_{2}+\beta_{2} X_i)\Big)\Big]=0 \quad \text{ only if \theta=\theta_0}$$ From reading the answer below, I understand that (5) is equivalent to $$(6) \quad \theta_0\underbrace{=}_{\text{unique}}\text{argmin}_{\theta\in \mathbb{R}^4} E\Big[Z_i^\top \Big(Y_i- f_i(\theta)(\alpha_{1}+\beta_{1} X_i)+(1-f_i(\theta))(\alpha_{2}+\beta_{2} X_i)\Big)\Big]^\top W E\Big[Z_i^\top \Big(Y_i- f_i(\theta)(\alpha_{1}+\beta_{1} X_i)+(1-f_i(\theta))(\alpha_{2}+\beta_{2} X_i)\Big)\Big]\\ \text{ s.t. f_i(\theta)=D_i}$$ where $$W$$ is some positive definite matrix. Correct?

(iii) Now, suppose I solve the unconstrained problem, where I simply plug in $$f_i(\theta)=D_i$$ in the objective function and obtain $$(7) \quad \min_{\theta\in \mathbb{R}^4} E\Big[Z_i^\top \Big(Y_i- D_i(\alpha_{1}+\beta_{1} X_i)+(1-D_i)(\alpha_{2}+\beta_{2} X_i)\Big)\Big]^\top W E\Big[Z_i^\top \Big(Y_i- D_i(\alpha_{1}+\beta_{1} X_i)+(1-D_i)(\alpha_{2}+\beta_{2} X_i)\Big)\Big]$$ Can I get a solution for (7) that is different from the unique solution $$\theta_0$$ of (6)? I think I will get the same unique solution $$\theta_0$$. Therefore, at the population level, the constrained and unconstrained problems are equivalent. Could you confirm?

(iv) Let us now move to the sample level, where we replace $$E$$ with the sample average $$\frac{1}{n}\sum_{i=1}^n$$. Is the minimiser of the sample analogue of (6) the same as the minimiser of the sample analogue of (7)? If I solve (7) instead of (6), do I need to take care of some extra variation when computing standard errors (and this is perhaps what you refer to when saying "sensible" and "valid" estimator)?

• If $D_i$ is a parametric function of $\theta$, how does it still depend on $i$ (i.e., do we have $D_1 = \cdots = D_n = f(\theta)$ all the same)?
– Ben
Commented Mar 22, 2022 at 21:32
• Suppose it is $f$ can differ across $i$.
– Star
Commented Mar 22, 2022 at 21:43
• Do you know each of the functions $f_i$ or are they unknown?
– Ben
Commented Mar 22, 2022 at 21:55
• I know all those functions in principle, but they do not have closed form (I would need to simulate them), that is why I would like to avoid considering them in the minimisation routine.
– Star
Commented Mar 22, 2022 at 22:36

So long as the vector $$\mathbf{D}$$ is observed, you could still estimate by solving $$(1)$$. This estimate would not take account of the information in your functions, but it would still be "valid" in the sense that it is a sensible estimator in the absence of that information.

If you want to incorporate the new information in your functions, you wouldn't do it in the way you're proposing --- the $$D_i$$ values are observed, so they are fixed values that should not vary in the minimisation. Instead, you would just take account of the fact that when you observe $$\mathbf{D}=\mathbf{d}$$ your parameters are now known to fall within the subspace:

$$\Theta(\mathbf{d}) \equiv \{ (\alpha_1, \beta_1, \alpha_2, \beta_2) | (\forall i=1,...,n): d_i = f_i(\alpha_1, \beta_1, \alpha_2, \beta_2) \}.$$

Consequently, your new minimisation would be just like $$(1)$$, except that you should only search over the parameters $$\theta \in \Theta(\mathbf{d})$$. In your comments you note that you only have access to your functions $$f_1,...,f_n$$ "in principle" but they are not in closed form and you would need to simulate them. This is likely to make it extremely difficult to determine the set $$\Theta(\mathbf{d})$$ for your observed $$\mathbf{d}$$. As a fallback if this is impossible, you could instead obtain roughly the same result by using constrained minimisation via penalty methods.$$^\dagger$$

$$^\dagger$$ To do the constrained minimisation via penalty method, you would formulate a sequence of smooth penalty functions $$p_1,p_2,p_3,...$$ that satisfies the limiting requirement:

$$\lim_{k \rightarrow \infty} p_k(\theta) = \begin{cases} 0 & & & \text{if } \theta \in \Theta(\mathbf{d}), \\[6pt] \infty & & & \text{if } \theta \notin \Theta(\mathbf{d}). \\[6pt] \end{cases}$$

You would then solve a sequence of unconstrained minimisation problems where you minimise $$F(\theta) - p_k(\theta)$$ for $$k=1,2,3,...$$ (where $$F$$ is your objective function) and estimate using the result for some sufficiently large value of $$k$$.

• Thanks for your answer. I'm still confused and I have tried to outline my doubts at the bottom of my question. It would be great if you could address them. Essentially, my doubts are related to the words "valid" and "sensible" used in your sentence "but it would still be "valid" in the sense that it is a sensible estimator in the absence of that information." I would like to formally qualify the words "valid" and "sensible" within an identification and inference framework.
– Star
Commented Mar 23, 2022 at 10:09