Likelihood function and Fisher information of a coin experiment I'm trying to understand the notions of likelihood function and Fisher information with a simple example. Suppose I have a coin that gives 0 with probability $p$ and 1 with probability $(1-p)$. I've thrown the coin $n$ times and observed $m$ 0s.
My likelihood function is looking like
$$L(p) = p^m(1-p)^{n-m}$$
I'm now trying to estimate p using maximum log-likelihood:
$$\frac{d \ln L}{d p}(p)=0$$
That gives me $p=\frac{m}{n}$. Looks pretty good.
Now I want to evaluate the Fisher information that these $n$ measurements gave me about $p$. The formula on Wikipedia says that it is the average of $(\frac{d \ln L}{d p})^2$.

My question is: how should I take this average?

As far as I can understand the likelihood associatec with the given sample is a function of the parameter $p$. If $p$ is fixed, $L(p)$ is just a number with average equal to this number.
 A: If you write down (somewhat paradoxically) the likelihood derivative (or score function) $$\dfrac{\text{d} \ln L}{\text{d} p}(p)$$as
$$\dfrac{\text{d} \ln L}{\text{d} p}(p,m)=\dfrac{m}{p}-\dfrac{n-m}{1-p}$$
it becomes a function of both $p$ and of the random outcome $M=m$ of the coin throwing experiment, when $M\sim\mathcal{B}(n,q)$. And hence a random function as well. 

Note: There are two parameters in the above, the first one is $p$ the
  argument of the likelihood function, namely any probability in
  $(0,1)$. And the second one is $q$, the true value of the
  parameter for the coin experiment, which is unknown and only informed
  through the result $M=m$ of the experiment, but which is necessary when
  computing expectations in $M$.

Let us consider the special case when $p=q$. You can thus consider the expectation of this quantity
\begin{align}
\mathbb{E}_p\left[\dfrac{\text{d} \ln L}{\text{d} p}(p,M)\right]
&=\sum_{m=0}^n \dfrac{\text{d} \ln L}{\text{d} p}(p,m) \mathbb{P}_p(M=m)\\ 
&=\sum_{m=0}^n \dfrac{\text{d} \ln L}{\text{d} p}(p,m) {n \choose m} p^m (1-p)^{n-m}\\
&= \sum_{m=0}^n \left\{\dfrac{m}{p}-\dfrac{n-m}{1-p}\right\} {n \choose m} p^m (1-p)^{n-m}\\
&=\dfrac{1}{p} \sum_{m=0}^n m{ n \choose m} p^m (1-p)^{n-m} -\dfrac{1}{1-p} \sum_{m=0}^n (n-m){ n \choose m} p^m (1-p)^{n-m}\\
&=\dfrac{np}{p} - \dfrac{n(1-p)}{1-p}\\
&=0 
\end{align}
and its variance, which is
\begin{align}
\mathbb{E}_p\left[\left\{\dfrac{\text{d} \ln L}{\text{d} p}(p,M)\right\}^2\right]
&=\sum_{m=0}^n \left\{\dfrac{\text{d} \ln L}{\text{d} p}(p,m)\right\}^2 \mathbb{P}_p(M=m)\\ 
&= \sum_{m=0}^n \left\{\dfrac{m}{p}-\dfrac{n-m}{1-p}\right\}^2 {n \choose m} p^m (1-p)^{n-m}\\
&=\dfrac{1}{p^2(1-p)^2}\sum_{m=0}^n \left\{(1-p)m-p(n-m)\right\}^2 {n \choose m} p^m (1-p)^{n-m}\\
&=\dfrac{\text{Var}((1-p)M-p(n-M))}{p^2(1-p)^2}\\
&=\dfrac{\text{Var}((1-p+p)M-pn)}{p^2(1-p)^2}\\
&=\dfrac{\text{Var}(M)}{p^2(1-p)^2}\\
&=\dfrac{np(1-p)}{p^2(1-p)^2}\\
&=\dfrac{n}{p(1-p)}
\end{align}

Expectation of the second derivative gave me non-zero result. Strange 
  as wiki said this two definitions are all the same "under certain 
  regularity conditions". 

If you consider the second derivative,
\begin{align}
\mathbb{E}_p\left[\dfrac{\text{d}^2 \ln L}{\text{d} p^2}(p,M)\right]
&=\mathbb{E}_p\left[\dfrac{\text{d}}{\text{d} p}\left\{\dfrac{M}{p}-\dfrac{n-M}{1-p}\right\}\right]\\
&=\mathbb{E}_p\left[-\dfrac{M}{p^2}-\dfrac{n-M}{(1-p)^2}\right]\\
&=-\dfrac{\mathbb{E}\left[M\right]}{p^2}-\dfrac{\mathbb{E}\left[n-M\right]}{(1-p)^2}\\
&=-\dfrac{np}{p^2}-\dfrac{n(1-p)}{(1-p)^2}\\
&=-\dfrac{n(p+1-p)}{p(1-p)}=-\dfrac{n}{p(1-p)}
\end{align}
which leads to the same value (modulo the minus sign).
