I have a sample of $N$ units and $T$ observations per unit. For each of these units $i$, I estimate a measure of dependence between two variables,$Y_{it}$ and $X_{it}$.

Call the estimated measure $\hat{\rho_i}$

At the unit level, I can use the standard error of $\hat{\rho_i}$ to carry out hypothesis tests and confidence intervals about the true dependence $\rho_i$

But how can I estimate the proportion of units with (say) a positive dependence, $\frac{1}{n}\sum_i 1(\rho_i > 0)$ and construct tests/confidence intervals for that?

Clearly, I can't just replace $\frac{1}{n}\sum_i 1(\rho_i > 0)$ with $\frac{1}{n}\sum_i 1(\hat{\rho_i} > 0)$ because $\hat{\rho_i}$ is never going to be exactly zero even if the true unit dependence is zero. The proportion estimator has got to take into account the (unit-varying) strength of evidence for a positive dependence at the unit level. One basic solution would be to look at the fraction of units with positive $\hat{\rho_i}$ for which the no-dependence can be rejected at confidence level $\alpha$%, but that's not really what I want.

It seems to me that if I would model $1(\rho_i > 0)$ as a Bernoulli random variable at the unit level, $\Pr(\rho_i > 0|Y_{it}, X_{it})$, I could just use the binomial distribution with different probabilities to do inference about the proportion of units with positive dependence.

But that seems to be a Bayesian approach. What's the general frequentist approach for this type of problem?

  • $\begingroup$ Try to get the results from raw data ($Y_{it}$ and $X_{it}$) directly. Introducing the intermediate statistics $\hat\rho_i$ will make the situation very complicated, even having no solution. $\endgroup$ – user158565 Jul 8 at 17:03
  • $\begingroup$ I'm open to any good estimator of $\frac{1}{n}\sum_i 1(\rho_i > 0)$, the proportion of units for which the two variables are positively associated, not just ones that use $\hat{\rho_i}$. But I don't know how to construct such an estimator. $\endgroup$ – CloseToC Jul 9 at 13:10
  • $\begingroup$ Even $\hat \phi_1 = \hat \phi_2 = 0.3$, $\hat \phi_1$ and $\hat \phi_2$ are different because they have different variance in most situations. $\endgroup$ – user158565 Jul 9 at 17:04
  • $\begingroup$ I agree with that, that's why the naive estimator that just replaces the true dependencies with their estimates to get the proportion is bad (despite being consistent as T goes to infinity), that's why you need to take into account the precision with which each unit dependency is estimated when aggregating. The question is how! $\endgroup$ – CloseToC Jul 10 at 9:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.