# Estimating and doing inference about proportions when individual indicators are themselves estimated

I have a sample of $$N$$ units and $$T$$ observations per unit. For each of these units $$i$$, I estimate a measure of dependence between two variables,$$Y_{it}$$ and $$X_{it}$$.

Call the estimated measure $$\hat{\rho_i}$$

At the unit level, I can use the standard error of $$\hat{\rho_i}$$ to carry out hypothesis tests and confidence intervals about the true dependence $$\rho_i$$

But how can I estimate the proportion of units with (say) a positive dependence, $$\frac{1}{n}\sum_i 1(\rho_i > 0)$$ and construct tests/confidence intervals for that?

Clearly, I can't just replace $$\frac{1}{n}\sum_i 1(\rho_i > 0)$$ with $$\frac{1}{n}\sum_i 1(\hat{\rho_i} > 0)$$ because $$\hat{\rho_i}$$ is never going to be exactly zero even if the true unit dependence is zero. The proportion estimator has got to take into account the (unit-varying) strength of evidence for a positive dependence at the unit level. One basic solution would be to look at the fraction of units with positive $$\hat{\rho_i}$$ for which the no-dependence can be rejected at confidence level $$\alpha$$%, but that's not really what I want.

It seems to me that if I would model $$1(\rho_i > 0)$$ as a Bernoulli random variable at the unit level, $$\Pr(\rho_i > 0|Y_{it}, X_{it})$$, I could just use the binomial distribution with different probabilities to do inference about the proportion of units with positive dependence.

But that seems to be a Bayesian approach. What's the general frequentist approach for this type of problem?

• Try to get the results from raw data ($Y_{it}$ and $X_{it}$) directly. Introducing the intermediate statistics $\hat\rho_i$ will make the situation very complicated, even having no solution. – user158565 Jul 8 at 17:03
• I'm open to any good estimator of $\frac{1}{n}\sum_i 1(\rho_i > 0)$, the proportion of units for which the two variables are positively associated, not just ones that use $\hat{\rho_i}$. But I don't know how to construct such an estimator. – CloseToC Jul 9 at 13:10
• Even $\hat \phi_1 = \hat \phi_2 = 0.3$, $\hat \phi_1$ and $\hat \phi_2$ are different because they have different variance in most situations. – user158565 Jul 9 at 17:04
• I agree with that, that's why the naive estimator that just replaces the true dependencies with their estimates to get the proportion is bad (despite being consistent as T goes to infinity), that's why you need to take into account the precision with which each unit dependency is estimated when aggregating. The question is how! – CloseToC Jul 10 at 9:38