As part of self-study, I am reviewing arguments I found tricky from notes from an introductory theoretical statistics course by Larry Wasserman. In particular, I have a number of queries concerning a heuristic argument to show that the interpretation of the likelihood function as "*that which contains all the information in the data*", is faulty.

**Context.**

Here is an extract of the argument:

> The likelihood function is a minimal sufficient statistic. That is, if we define the equivalence relation: $x^n \sim y^n$ when $L(\theta; x^n) \propto L(\theta; y^n)$, then the resulting partition is minimal sufficient.
>
> Does this mean that the likelihood function contains all the relevant information? Some people say yes it does. This is sometimes called the *the likelihood principle*. That is, the likelihood principle says that the likelihood function contains all the information in the data.
>
> This is FALSE. Here is a simple example to illustrate why. Let $\mathcal{C} = \{c_1, \dots, c_N\}$ be a finite set of constants. For simplicity, assume that $c_j \in \{0, 1\}$ (although this is not important). Let $\theta = (1 / N)  \sum^N_{j=1} c_j$. Suppose we want to estimate $\theta$. We proceed as follows. Let $S_1, \dots S_n \sim \text{Bernoulli}(\pi)$ where $\pi$ is known. If $S_i = 1$, you get to see $c_i$. Otherwise, you do not. (This is an example of survey sampling.) The likelihood function is
>
> $$\prod_i \pi^{S_i} (1 - \pi)^{1 - S_i}.$$
>
> The unknown parameter does not appear in the likelihood. In fact, there are unknown parameters in the likelihood! The likelihood function contains no information at all.
> But we can estimate $\theta$. Let
>
> $$\hat{\theta} = \frac{1}{N \pi} \sum^N_{j=1} c_j S_j.$$
>
> Then $\mathbb{E}[\hat{\theta}] = \theta$, Hoeffding's inequality implies that
>
> $$\mathbb{P}(\vert \hat{\theta} - \theta \vert > \epsilon) \leq 2 \exp(-2n \epsilon^2 \pi^2).$$
>
> Hence, $\hat{\theta}$ is close to $\theta$ with high probability.
>
> **Summary: The minimal sufficient statistic has all the information you need to compute the likelihood. But that does not mean that all the information is in the likelihood.**

**Queries.**

1\. *What is the "data", the distribution expectations are being computed with respect to, and the explicit steps to show $\hat{\theta}$ is unbiased*?

Whilst I understand that $\hat{\theta}$ is unbiased, I am struggling to show why it is true. In particular, in the computation of $\mathbb{E}[\hat{\theta}]$. I am uncertain as to what the "data" is here, and as to what the distribution expectations are being computed with respect to is. Therefore I am unsure as to how to proceed in this calculation:

$$\mathbb{E}[\hat{\theta}] = \frac{1}{N \pi} \mathbb{E} \left[ \sum^N_{j=1} c_j S_j \right]$$

I am guessing that independence of $c_j$ and $S_j$ can be assumed, and I know the above should be equal to $\theta$.

2\. *How does the fact that we can estimate $\hat{\theta}$ in spite of it not being present in the likelihood function, and the fact that $\hat{\theta}$ is close to $\theta$ with high probability relevant to the argument*?

Whilst I understand that the likelihood function $\prod_i \pi^{S_i} (1 - \pi)^{1 - S_i}$ does not contain the parameter $\theta$, I fail to see the relevance and importance of the counterpoint in the argument, "...*but we can estimate $\theta$.*" I am therefore unable to appreciate how showing that $\hat{\theta}$ is close to $\theta$ with high probability via Hoeffding's inequality fits into the argument. 

3\. *Are there any further references which discuss this heuristic argument and its the issues contained within more broadly?*

I ask because my instinct is that the claim that "*[the likelihood function contains all the information in the data] is FALSE*" may be more contentious than the author might be suggesting. And I am not sure if this is a pedagogical simplification, or if it's the author's personal position on a contentious issue in statisitics, or if this enjoys general consensus in the statistical community. Essentially, I would like to know how contentious this claim is, if at all.