In practice, the observed information matrix (Newton-Raphson) is usually replaced by its expectation, known as Fisher scoring.

Link: https://en.wikipedia.org/wiki/Scoring_algorithm#Fisher_scoring

What I don't understand is, why the expectation of the matrix is easier to compute than the matrix itself? Otherwise, there wouldn't be any point to use Fisher scoring.

  • 1
    $\begingroup$ Because you only need to take the first (not the second) derivatives, which can be expressed as the expectation. $\endgroup$
    – Randel
    Commented Oct 7, 2015 at 14:31
  • 1
    $\begingroup$ @Randel Can you elaborate a bit more? Other users might also find your answer helpful. Most textbooks don't say why Fisher scoring is better computationally. $\endgroup$
    – SmallChess
    Commented Oct 7, 2015 at 14:33
  • 1
    $\begingroup$ I would refer to Page 88, Section 2.11.2 Empirical FS algorithm of the book by Prof. Demidenko. This is the FS algorithm I usually see, and there are some approximation. $\endgroup$
    – Randel
    Commented Oct 7, 2015 at 23:53


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.