1
$\begingroup$

I am working with a two-step heckman selection model. In the first step the selection occurs based on a probit model, in the second step the mean equation is fitted with a linear model where the inverse mills ration is included.

Technically I assume the existence of a partly latent variable $Y\in \mathbb{R}$, where we only observe $\Delta Y \in \mathbb{R_+}$, with $\Delta=I(Y>0)$ beeing an indicator function that is one if $Y$ is greater than zero which is seen as equivalent to $Y$ beeing observed. Furthermore we assume that the latent variable $Y$ can be modelled using a linear model, using some covariates $X$: $Y=X\beta + \epsilon$ and that we can use covariates $Z$ in order to employ a probit model for the probability of response, i.e. $P(\Delta=1|Z)=\Phi(Z\gamma)$, with $\Phi(\cdot)$ denoting a standard normal CDF.

Using this, plus the assumption that the errors from both stages are jointly normal, as well as $Z\subset X$, we can arrive at $$E(\Delta Y|X,\Delta=1)=E(Y|X,\Delta=1)=X\beta + E(\epsilon|X,\Delta=1)=X\beta + \rho \sigma_\epsilon\lambda(Z\gamma) $$ with $\lambda(\cdot)$ representing the inverse Mill Ratio evaluated at $Z\gamma$.

The estimation is done based on maximum likelihood, hence I can compare different models based on the AIC and/or the likelihood ratio test. However I also want to evaluate the Goodness-of-Fit for both stages separately.

For the first stage, the probit model, I can employ ROC Curves and calculate the area under the curve (AUC). Additionally I could rely on Precision-Recall Curves. But what about the second stage? I cannot employ the measure $R^2$ since the mean equation is not a linear regression model if I am condition on the selection. At the moment I am relying on calculating the mean squared residuals of the mean equation given $\Delta=1$: $$MSR=n^{-1}\sum_{i=1}^n(\Delta Y_i -X_i \hat{\beta}-\hat{\rho}\hat{\sigma}_{\epsilon}\lambda(Z_i \hat{\gamma}) )^2$$ I have not seen this measure anywhere else but I would use it in the following way: if the $MSR$ of model A is smaller than the $MSR$ of model B, model A provides the better fit, conditional on the selection.

Alternatively I could probably use the $R^2$ based on the fitted latent model $Y=X\hat{\beta}+\hat{\epsilon}$ but then the idea of evaluating both stages separately, first the selection and then the volume given the selection, is gone. Do you have same suggestions or solutions?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.