Usage of Heckman estimation for a random sample

Question

My colleague argues to use a Heckman Model in the following case (Agricultural economics):

I have a random sample of farmers (n) of the general population N. In n some observations apply a given technique to improve agricultural productivity whereas others don't. If they use this technique, they can choose to apply it on any given amount of area on their farm. I would like to test whether a given policy change influenced their choice to apply this technique and the subsequent amount of area where this technique is applied, once the choice was positive.

Without deeper econometric knowledge I would use a probit/or logit model to estimate the influence of the policy on choice and a separate OLS to estimate the influence of the policy on the total amount of area where this technique is applied. My colleague, however, argues that estimating a normal OLS might suffer from bias because of variables that determine the set of "technology adopters" in the first place.

My intuition tells me that I can just use these potentially relevant variables as controls in the OLS which will single out their effect. Furthermore, literature suggest, that Heckman is applied to the case where I have a non-random sample where those who are not selected are omitted from the sample in the first place (selection bias). The classical example is the one of incidental truncation where we do not observe the dependent variable y because of the outcome of another variable x. An often cited example is the wage offer function from labor economics where wages are only observed for the workforce and a model of e.g. education on wage suffers from bias (see Wooldrige, 2009. p.606ff).

Certo et al (2016) discuss the usage of Heckman models and argue:

"The first step in implementing a Heckman model involves considering the potential for sample selection bias. As we noted previously, our literature review suggests that scholars often conflate sample selection bias with endogeneity from other sources. A simple rule helps to distinguish between these two alternatives: If the dependent variable is observed for only a subsample of a larger population (e.g., wages for working women, stock market reactions to acquisition announcements, etc.), sample selection is a potential problem. In such cases, Heckman models (i.e., Stata’s “-heckman-”) may help to resolve potential sample selection bias. In contrast, if the dependent variable is available for all observations (e.g., ROI of acquirers versus nonacquirers), then sample selection is likely not an issue. Instead, in such cases the independent variable is likely endogenous, which would require either two-stage least squares (i.e., Stata’s “-ivreg-”) or treatment effects models (i.e., Stata’s “-etregress-”)"

Can anybody highlight me whether I am overseeing something essential here? Also alternative solutions to model my stated problem would be very welcome.

Answer 1

You need to distinguish between sample selection in terms of who is in the sample, and sample selection in terms of what those in the sample choose to do. In labour economics, the former refers to the fact that wage is only observed for those employed, but not the unemployed or inactive. This is the traditional example of Heckman. Meanwhile, the second type of selection refers, for instance, to the type of work workers sort into, e.g. self-employed versus employee. A classic example of this selection type is the paper by Card (1996), where selection is between union and non-union sectors.

In the RCT literature (which is commonplace in agricultural economics), first you produce a random sample of the population, and then you randomise the treatment, so you overcome both problems of selection. The first problem can also be solved using weighting, but the second is more pervasive. Another reference that might help you on this is Suri (2011).

In your case, your sample is random, so you do not face the first problem. What you do face is selection into whether to apply the technique or not. This is clearly not random, as those who apply it are those more likely to gain from it. This is the selection equation that you need to model together with your structural equation. Like in any other selection model, you need to think on variables that might affect the use of the technique but do not affect the outcome variable in terms of area.

Answer 2

I'm going to try and rephrase your first paragraph to make sure we're on the same page.

I have a random sample of farmers (n) of the general population N. In n some observations apply a given technique to improve agricultural productivity whereas others don't. If they use this technique, they can choose to apply it on any given amount of area on their farm. I would like to test whether a given policy change influenced their choice to apply this technique and the subsequent amount of area where this technique is applied, once the choice was positive.

We observe $n$ individuals, each of which has a farm with area $a_1, \cdots, a_n$ Then training is given to $m<n$ individuals, some of which choose to apply the new technique to a fraction $p_i$ of their farm. Ok?

Here's the thing: if training is given to individuals at random, then you merely have another random sample of the population and no sampling bias issues should matter. On the other hand, if the $m$ farmers were selected according to some criteria (possibly researchers wanted the farmers most likely to apply the technique), you definitely have a problem. But then, the first-stage equation estimated by heckman in Stata would show insignificant coefficients and second-stage estimates would remain close to those of pure OLS. The best practice is to publish coefficients from different estimating techniques when in doubt.

(Relevant analogy: a die with faces $D=\{1,2,3,4,5,6\}$ is a fixed set of integers. A die throw is a random variable, i.e. a function from $2^D \to [0,1]$ taking each subset of $D$ to a probability. Likewise, a population (with its measurements, assumed to have no noise, etc.) is a fixed set of vectors $\{v_1, \cdots, v_N\}$ -- but a random sample is a set of random variables, like $N$ dice thrown over the range of available individuals in the population. (I'm overlooking details about sampling with or without repetition.) It's important that these random variables are iid, otherwise basic OLS theory goes out the window.)