Ways to shrink standard errors in models for discrete dependent variables

Question

Consider a simple Probit model $$ Y_i=1\{X_i\beta+\epsilon_i\geq 0\} $$ where $\epsilon_i$ is standard normal independent of $X_i$.

(1) Cardinality of the support of $X_i$

Is it true that (and, if yes, why) when $X_i$ is a continuous regressor we typically get a confidence interval for $\beta$ that is tighter than when $X_i$ is discrete (for instance binary)?

My confusion: I know that in the classic linear model, if the variance of $X_i$ increases, then the standard error of the estimate for $\beta$ decreases. I suppose a similar mechanism takes place in the Probit model. However, here, we are talking about cardinality of the support of $X_i$ and not about its variance.

(2) Variance of $\epsilon_i$

Suppose that $\epsilon_i$ is distributed as a normal with variance $5$ (instead of $1$) as in the classic Probit model. Suppose that the researcher knows the variance of $\epsilon_i$ (it is not a parameter to estimate).

Is it true that (and, if yes, why) that with a higher variance of $\epsilon_i$ we get a confidence interval for $\beta$ that is tighter?

My thoughts: I know that in the classic linear model, if the variance of the error increases, then the standard error of the estimate for $\beta$ decreases. Essentially, this is because a higher variance of $\epsilon_i$ generates a higher variation in $Y_i$ for each given realisation of $X_i$, which improves the precision of our estimates. I suppose a similar mechanism takes place in the Probit model. Is it correct?

(3) Correlation among latent terms

Suppose now that $Y_i$ can take three values $y\in \{0,1,2\}$ (instead of being binary). Hence, instead of using a Probit model, we use a Multinomial Probit $$ Y_i=argmax_{y\in \{0,1,2\}} \beta X_{iy}+\epsilon_{iy} $$ where $\epsilon\equiv (\epsilon_{i0}, \epsilon_{i1}, \epsilon_{i2})$ is a tri-variate Normal with mean $0$ and variance-covariance matrix $\begin{pmatrix} 1 & \rho & \rho\\ \rho & 1 & \rho\\ \rho & \rho & 1 \end{pmatrix}$.

Suppose that the researcher knows the value of $\rho$ (it is not a parameter to estimate). In my simulations, I found that as $\rho$ increases from $0$ to $0.9$, the confidence interval for $\beta$ shrinks. Is that something that should be expected and, if yes, what is the intuition behind?

My thoughts: I think a possible explanation could be that, when $\rho$ is high, the errors are "partly" predictable. This improves our precision in getting $\beta$.

Answer 1

1. Yes (usually)

Let's say the true data-generating process is $Y_i=1\{X_i\beta+\epsilon_i\geq 0\}$, as you described, and $X$ is continuous. If you replace $X_i$ with $Z_i$, where $Z$ is obtained by binning $X$ (including a median split) you've thrown away relevant information. Less information means more uncertainty, means greater standard error.

Of course, if the true process really is $Y_i=1\{Z_i\beta+\epsilon_i\geq 0\}$ this doesn't apply.

2. No

$\epsilon$ is the noise term. Greater $\sigma_{\epsilon}$ (noise standard deviation) means more noise in the model. More noise means greater standard errors.

I know that in the classic linear model, if the variance of the error increases, then the standard error of the estimate for β decreases. Essentially, this is because a higher variance of ϵi generates a higher variation in Yi for each given realisation of Xi, which improves the precision of our estimates.

This is exactly wrong. Again, more noise variance means less certainty.

Note that the probit model doesn't work if both $\beta$ and $\sigma_{\epsilon}$ (the standard deviation of $\epsilon$) are both free parameters, since they trade off against each other: $ 1\{\beta X+\epsilon\geq 0\} = 1\{\frac{\beta}{k} +k\epsilon\geq 0\} $, where $k$ is any scaling factor. By setting $\sigma_{\epsilon} = 1$, we constrain the model so that $\beta$ is in units of standard deviation, making it a signal-to-noise ratio.

If you simulate data with, e.g. $\beta = 10, \sigma_{\epsilon} = 5$ and fit a probit model with $\sigma_{\epsilon}$ fixed to $1$ and $\beta$ estimated around $2$. Increasing simulated $\sigma_{\epsilon}$ will reduced $\beta$, but keep the standard error the same.

If you simulate data with $\beta = 10, \sigma_{\epsilon} = 1$, and fit a probit model where you fix $\sigma_{\epsilon}$ to $5$, beta will inflated to $50$, but so will its standard error, leaving your p-value unchanged.

3. Don't Know

I'm afraid I couldn't parse your formula or your question. Could you maybe clarify? Some of what you describe looks like multinomial probit, and some like multivariate probit (which isn't the same thing).