3
$\begingroup$

I have a particular situation. I would like to build a probit model to predict a given outcome $Y=\{0,1\}$ based on a set of predictors $X$.

A probit model has the form:

$$\text{Pr}(Y=1\,|\,X)=\Phi(X^T\beta)$$

where $\beta$ is a vector of coefficients estimated in the model. It can also be written as a latent variable model with an auxiliary variable

$$Y^{*}=X^T\beta+\epsilon$$

where $\epsilon\sim N(0,1)$. $Y$ is then viewed as an indicator for whether the latent variable is positive

$$Y=\begin{cases} 1,\quad\quad Y^{*}>0\\ 0,\quad\quad \text{otherwise} \end{cases}$$

Now, this is all fine. However, in my circumstances I have the assumption that there is some level of right truncation in the data I have. Thus, I was wondering if the following is a valid approach.

Let's say we fit our latent variable regression above using a simple linear model. From this, we would have estimates $\hat{\beta}$ and $\hat{\sigma}$ i.e. the coefficients and variance term of the fitted model.

Essentially, this gives us the mean and variance parameters of a Normal distribution. We can adjust these parameters for truncation as follows:

$$\begin{align} \mathbb{E}[X \,|\,X < b] &= \mu_{a} - \sigma_{a} \frac{\phi(\alpha)}{\Phi(\alpha)}\notag\\ \text{Var}(X \,|\, X < b) &= \sigma_{a}^2\left( 1 - \alpha\frac{\phi(\alpha)}{\Phi(\alpha)} - \left(\frac{\phi(\alpha)}{\Phi(\alpha)}\right)^2 \right) \end{align}$$

i.e. this gives us the true $(\mu_{a},\sigma_{a})$ of our Normal distribution after adjusting for truncation. The truncation level is $\alpha$. Note that in the case of right-truncation $\mu_{a}>X^{T}\beta$.

From here we can convert the probit as follows:

$$\begin{align}\text{Pr}(Y=1\,|\,X)&=\text{Pr}(Y^{*}>0)\notag\\ &=\text{Pr}(N(\mu_{a},\sigma_{a}^2)>0)\notag\\ &=\Phi\bigg(\frac{\mu_{a}}{\sigma_{a}}\bigg)\notag\\ &=\Phi\bigg(\frac{X^{T}\beta+c}{\sigma_{a}}\bigg)\notag \end{align}$$

for some $c>0$.

Is the above reasoning correct?

Alternatively, the likelihood function for a probit model is:

$$\begin{align} \ln\mathcal{L}(\beta)&=\sum_{i=1}^{n}\bigg(y_{i}\ln\Phi(x_{i}'\beta)+(1-y_{i})\ln\big(1-\Phi(x_{i}'\beta)\big)\bigg) \end{align}$$

Where the right-truncated version would be: $$\begin{align} \ln\mathcal{L}(\beta)&=\sum_{i=1}^{n}\bigg(y_{i}\ln F_{\text{TN}}(x_{i}'\beta;\alpha)+(1-y_{i})\ln\big(1-F_{\text{TN}}(x_{i}'\beta;\alpha)\big)\bigg) \end{align}$$

where $F_{\text{TN}}(x;\alpha)$ is the cumulative distribution function of the standard right-truncated Normal distribution. Maximizing (3) provides the estimates of the coefficients, $\hat{\beta}$.

Is this also correct?

$\endgroup$
  • $\begingroup$ What exactly is truncated in your data? $\endgroup$ – Dimitriy V. Masterov Dec 8 '17 at 0:31
  • $\begingroup$ @DimitriyV.Masterov The assumption is that the binary dependent variable is truncated i.e. that there is some truncation of the yes/no outcomes we observe. $\endgroup$ – StatsPlease Dec 8 '17 at 0:40
  • $\begingroup$ Truncation occurs when some values of an underlying continuous variable are simply not observed. For example, the government might collect data only on firms above a certain size, in order to minimize regulatory burden from paper work. I don't really know what it means for a binary variable to be truncated. $\endgroup$ – Dimitriy V. Masterov Dec 8 '17 at 0:55
  • $\begingroup$ @DimitriyV.Masterov yes I understand the use of the term truncation is not exactly rigorous here. The application is that we know we haven't observed the worst parts of the economic cycle and therefore we expect a similar level of truncation in our dependent variable. $\endgroup$ – StatsPlease Dec 8 '17 at 1:03
  • $\begingroup$ I'm afraid you're not really explaining how a binary variable can be truncated. If you are observing 0 and 1 values, truncation would normally imply that you are not observing the 0 values or not observing the 1 values. If $X$ is "truncated" simply because you haven't observed the more extreme values of $X$, that's not a truncation of the dependent variable, that just means that observed $X$ is not a random sample from actual $X$. Your model will still work, conditional upon the observed $X$. It's always chancy to extrapolate beyond the range of the observed data, however. $\endgroup$ – jbowman Dec 8 '17 at 20:55
1
$\begingroup$

Here's my attempt to this problem. I hope this can answer your question.

It is easier to think of the the truncation problem for probit model as a latent variable model, i.e. $$Y_i=\begin{cases} 1, \quad Y_i^*>0\\ 0, \quad Y_i \leq 0 \end{cases}$$ And the latent linear model: $$ Y_i^*=X\beta+\epsilon_i,\quad \epsilon_i \sim i.i.d. \mathcal{N}(0,\sigma^2) $$ My thinking of the truncation is that, it is the latent variable that is being truncated, which obviously implies the truncation of the outcome variable. So the following pairs of data are not observable: $(Y_i, X_i)$ s.t. $Y^*_i>b$ with $b$ the truncation level. With this truncation, the original probit model becomes: $$Y_i=\begin{cases} 1, \quad 0<Y_i^*<b\\ 0, \quad Y_i \leq 0 \end{cases}$$ Which implies that: $$Y_i=\begin{cases} 1, \quad -X_i'\beta<\epsilon_i<b-X_i'\beta\\ 0, \quad \epsilon_i \leq -X_i'\beta \end{cases}$$ Conditioning on $\epsilon_i<b-X_i\beta$, the distribution of $\epsilon_i$ is independent truncated normal with a heterogeneous truncation level that depends on the $X_i$s.

The log-likelihood function is relatively easy: $$ \ln \mathcal{L}(\beta, \sigma;Y_i)=\sum_{i=1}^N\Big( Y_i\ln P(-X_i'\beta<\epsilon_i<b-X_i'\beta)+(1-Y_i)\ln P(\epsilon_i\leq -X_i'\beta) \Big)$$ And note that for arbitrary $x$ an $y$ with $x<y<b-X_i'\beta$: $$\ln P(x<\epsilon_i<y)=\ln(F(y)-F(x))= \ln( \Phi(y/\sigma)-\Phi(x/\sigma))-\ln(\Phi((b-X_i'\beta)/\sigma)) $$ Clearly it can be generalized to allow for truncation of $Y_i^*$ from both above and below.

The problem with this approach though, is that you don't know the truncation level $b$ for the latent data, and you have to treat it as a parameter and estimate it. Unfortunately it is not identified as your data is not continuous. Some grid-search algorithms are needed to maximize this likelihood function, but conditioning on $b$, there shouldn't be a problem to maximize the likelihood at all.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.