# How does an influence function-based estimator estimate a target functional for an unknown distribution?

How exactly does a "1-step" influence function-based estimator estimate a target functional (like average treatment effect) for an unknown distribution?

As described in Aaron Fisher and Edward H. Kennedy's tutorial (2019), the aim is to find an unbiased estimate of a functional $$T(P)$$ and its variance for an unknown distribution $$P$$ by approximating with a smoothed parametric estimate (or possibly nonparametric estimate using machine learning models) $$T(\tilde{P})$$ from the observed data.

The plots above are from Aaron Fisher and Edward H. Kennedy's tutorial (2019) on influence functions. Panel B illustrates how a Taylor series approximation is used to estimate a hypothetical $$T(P)$$ by a linear extrapolation from the slope (calculated using an influence function) at $$T(\tilde{P})$$ (that is, the $$y$$-intercept), where the solid line shows the path of the target functional value $$P_{\epsilon}$$ as we vary the weight $$\epsilon \in [0,1]$$.

The Taylor series expansion is then:

$$T(P_0)=T(P_1)+\left(\frac{\partial}{\partial \epsilon}T(P_{\epsilon})\Big{|}_{\epsilon =1}\right)\times (0-1)-R_2$$

where $$R_2$$ is the second order term of the Taylor expansion.

The plots seem a little too convenient, where a step size of exactly $$\Delta \epsilon=1$$, from 1 to 0, magically gets you from $$T(P_1)=T(\tilde{P})$$ to $$T(P_0)=T(P)$$ even though you don't know in advance what the $$T(P_{\epsilon})$$ path looks like.

### About the convenient plot and the Taylor approximation

The plots seem a little too convenient, where a step size of exactly $$\Delta \epsilon=1$$, from 1 to 0, magically gets you from $$T(P_1)=T(\tilde{P})$$ to $$T(P_0)=T(P)$$ even though you don't know in advance what the $$T(P_{\epsilon})$$ path looks like.

$$T(P_0)=T(P_1)+\left(\frac{\partial}{\partial \epsilon}T(P_{\epsilon})\Big{|}_{\epsilon =1}\right)\times (0-1)-R_2$$

This remainder term $$R_2 = -\frac{1}{2} \frac{\partial^2}{\partial \epsilon^2}T(P_{\epsilon})\Big{|}_{\epsilon = \bar{\epsilon}}$$ is a consequence from Taylor's theorem.

It is not the second order term of the Taylor expansion, instead it is the remainder term. Note that for this term $$\epsilon \neq 1$$, but $$\epsilon = \bar{\epsilon}$$.

The value $$\bar{\epsilon}$$ is undetermined but it must be some value between the boundaries (0 and 1).

You can see it alternatively as following: the remainder term $$R_2$$ of the taylor approximation is limited by the lower and upper limits of the second derivative $$T^{\prime\prime}(P_\epsilon)$$

$$\frac{1}{2} \min_{0\leq\epsilon\leq1}T^{\prime\prime}(P_\epsilon) \leq R_2 \leq \frac{1}{2} \max_{0\leq\epsilon\leq1} T^{\prime\prime}(P_\epsilon)$$

and so there is some value $$\bar\epsilon$$ for the real value $$T^{\prime\prime}(P_\bar{\epsilon})$$ which is somewhere in between.

$$\frac{1}{2} \min_{0\leq\epsilon\leq1}T^{\prime\prime}(P_\epsilon) \leq \frac{1}{2} T^{\prime\prime}(P_\hat\epsilon) \leq \frac{1}{2} \max_{0\leq\epsilon\leq1} T^{\prime\prime}(P_\epsilon)$$

The exact path, the thick line remains unknown. We do not magically get it.

But we can know that the remainder term $$R_2$$, the difference between our linear estimate (the broken thin line thin) and the exact path, is relatively small (an error term that doesn't grow faster than the second derivative, which is for most smooth functions not so large).

### About an intuitive view of the robustness due to correction with influence curves

Hoping to see a clear, intuitive explanation of how efficient influence curves are applied to initial probability distribution estimates (perhaps using nonparametric machine learning models) to arrive at an unbiased estimate of a target function.

It is the first time that I read about 1-step estimators, to my intuition it seems like some form of scoring algorithm where the score and fisher information are based the influence functions and an empirical distribution (how a change in the observation changes the parameter estimate can be inverted to how a change in the parameter changes the probability of observations, and related the likelihood function).

### Example

The practical example and computation below may provide some intuition:

In this example the target is to estimate for the population distribution function $$f(x)$$ the integrated squared density:

$$T(f(x)) = E[f(x)] = \int_{-\infty}^{\infty} f(x)^2 dx$$

In the appendix C of the article from Fisher and Kennedy it is stated that in this case the influence function is

$$IF(x,f) = 2(f(x)-T(f))$$

In the code below we first estimate the distribution with a normal distribution in which case the initial estimate is $$T(\tilde{f}(x)) = 1/\sqrt{4 \pi \hat{\sigma^2}}$$. See in the image below that this estimate with a normal distribution is not a good one if the data is not normal distributed (in this case we generate the data according to a geometric distribution).

So we use the influence functions to correct the biased normal distribution estimate and shift that distribution with a first order approximation to the empirical distribution (a sum of delta functions).

The effect is a reduction of the bias from using the normal distribution as an approximation for the distribution. We get an estimate that is more robust than our estimate with a (potentially biased) parameterized distribution.

In this case the plugin solution $$\sum \hat{f}(x)^2$$ is actually doing pretty well, and even better than the 1-step estimator. This is because the computation is done with a sample size of $$n=100$$ in which case the mass distribution $$f(x)$$ can be well estimated. But for a small sample there will be only a small amount of cases in each bin and we would have $$\sum \hat{f}(x)^2 \approx \sum (1/n)^2 = 1/n$$ and that's when the approximation with a parametric distribution (and the 1-step estimator for robustness) are useful.

set.seed(1)

trueval <- sum(dgeom(0:300,0.1)^2)

onestep <- function(n = 20, plotting = FALSE) {
x <- rgeom(n,0.1)

# estimating distribution with normal distribution
# using method of moments
mu <- mean(x)
var <- var(x)*n/(n-1)
Test <- 1/sqrt(4*pi*var)

# computing influence functions
#
# for T(f(x)) = integral of f(x)^2 dx
#
IF <- 2*(dnorm(x,mu,var^0.5)-Test)

# making corrections
T1step <- Test + sum(IF)/n

#plotting
if (plotting) {
h<-hist(x, breaks = c(0:200)-0.5, xlim = c(-10,40), freq = FALSE,
main = "geometric data and normal estimate")
xs <- seq(-10,40,0.01)
lines(xs, dnorm(xs,mu,var^0.5))
} else {
h<-hist(x, breaks = c(0:200)-0.5, xlim = c(-10,40), freq = FALSE,
main = "geometric data and normal estimate", plot = FALSE)
}