2
$\begingroup$

Suppose that there is a data-generating process $$ y = \alpha + g(x) + \epsilon $$

which is to say that an outcome is some function of $x$. Suppose that $x$ is randomly assigned, so $\text{Cov}(x,\epsilon)=0$. But the functional form mappying $x$ to $y$ is unknown. Say that $y$ is continuous and the variance of $\epsilon$ is normal and iid. Think of $\epsilon$ not as measurement error, but rather as everything else that determines $y$ besides $x$. So, $g(x)$ represents the causal effect of $x$ on $y$.

Say that you've got reason to believe that a straight line is a bad model for the data. Your instinct might be to use a gam:

library(mgcv)
model = gam(y~s(x))

But smoothing is model selection (via a ridge penalty), and model selection biases coefficients. This is no good, because you want unbiased causal effects. You can represent the various splines that make up a gam as $\beta_j$ for a given spline basis. Given a non-zero smoothing parameter $\lambda$, $E[\hat\beta_j] \neq \beta_j$. The bias-variance tradeoff for a smoothing spline is often presented as

$$ PSS(\hat f, \lambda) = \displaystyle\sum_{i=1}^N\left( y_i-\hat f(x_i)\right)^2 - \lambda \int\left( \hat f''(x)\right)^2 $$

which is to say that as the smoothness increases, the bias increases.

This is the problem -- bias away from what? The bias is away from the true values of $\beta_j$. I don't care about $\beta_j$, I care about $g(x)$, which is roughly represented by a set of penalized splines.

So if I were to take the ridiculous position that my set of unpenalized splines (chosen by allocating them evenly across the quantiles of $x$) were the $true$ model, then indeed, my estimates of $g_{smooth}(x)$ would be "biased" away from my estimates of $g_{unpenalized}(x)$.

So I want to be able to quantify something like $$ E[\hat g_\hat{\lambda}(x)] = g(x) + \text{smoothing bias} - \text{misspecification bias} $$

so that I can discriminate whether bias is minimized using a gam or minimized using a parametric approximation.

Intuitively, if the true $g(x)$ looks roughly sigmoidal and I fit it with a straight line, then smoothing bias would be zero and mis-specification bias will be high. If I use a gam, then smoothing bias will be positive and mis-specification bias will be lower (though not zero, because the basis dimension is finite and $\lambda$ is estimated).

Basically, I want to be able to argue that the choice of a smooth functional form is more or less biased than a linear regression coefficient for a completely exogenous, continuously-valued treatment. So how do I quantify mis-specification bias?

One of the best forms of help here would be citations of papers to read that have thought about this. I'm not sure what terms to search for when scouring the literature.

Note: I'll have control variables as well as $x$, so lowess is out. In addition, I'd like to look at heterogeneous treatment effects via interactions with $x$. So I'd like to see if its not impossible to use gam

EDIT: no ideas, not even any comments?

$\endgroup$
1
$\begingroup$

Perhaps the answer to my question is that it is impossible to quantify mis-specification bias without knowing the true functional form.

Edit 5 years later Somebody should have told me to read Wahba 1983 https://www.jstor.org/stable/2345632?seq=1#page_scan_tab_contents And Nychka 1988 http://amstat.tandfonline.com/doi/abs/10.1080/01621459.1988.10478711#.WmnJlcpOlnE

$\endgroup$
  • $\begingroup$ Or, by having a big hold-out sample? $\endgroup$ – kjetil b halvorsen Dec 22 '17 at 9:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.