Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [fisher-scoring]

A form of Newton's method used in statistics to solve maximum likelihood equations numerically [Wikipedia]

4
votes
0answers
350 views

Fisher information matrix in logistic regression

I am self-studying the basics of logistic regression. I came across this sentence: In logistic regression expected and observed information matrixes are equal I am aware that the information ...
0
votes
0answers
90 views

Generalized Additive Models: How to fit models with the LMS method by Cole?

I have some problems to understand the univariate generalized models that were proposed by Cole (1992). My question is how the fitting procedure works described in the appendix. More specific, how to ...
17
votes
1answer
2k views

Why using Newton's method for logistic regression optimization is called iterative re-weighted least squares?

Why using Newton's method for logistic regression optimization is called iterative re-weighted least squares? It seems not clear to me because logistic loss and least squares loss are completely ...
0
votes
1answer
487 views

Variation in Fisher scoring compared with Newton-Raphson (logistic regression)

I have been trying to figure out the implementation in knime. The tool says it uses Fisher scoring (FS). I understand Newton Raphson-method from http://www.win-vector.com/blog/2011/09/the-simpler-...
1
vote
0answers
637 views

partial derivative of log likelihood function

I am trying to find $\partial/\partial \theta \log \ L $ where $L = \pi^y(1-\pi)^{1-y}, \pi = \frac{\exp(X \beta)}{1+\exp(X \beta)}, \ X$ is $ N(\mu,\sigma^2) \ \& \ y$ is binary. I first ...
6
votes
1answer
912 views

can you explicitly show me the first iteration of newton-raphson and fisher scoring?

I'm trying to understand the difference between the Newton-Raphson technique and the Fisher scoring technique by calculating the ...
7
votes
2answers
3k views

Fisher's score function has mean zero - what does that even mean?

I'm trying to follow the princeton review of likelihood theory. They define Fisher’s score function as The first derivative of the log-likelihood function, and they ...
2
votes
0answers
421 views

Fisher information matrix with a general covariance structure

For the linear model, general linear models which allow for a more general covariance structure $V(\theta)_{N\times N}=(I_{N}+\theta A_{N\times N})(I_{N}+\theta A_{N\times N})^{'}$ ,where $A_{N\times ...
5
votes
2answers
4k views

Implement Fisher Scoring for linear regression

I know there is an analytic solution to the following problem (OLS). Since I try to learn and understand the principles and basics of MLE, I implemented the fisher scoring algorithm for a simple ...
5
votes
0answers
845 views

Why Fisher Scoring is easier to compute?

In practice, the observed information matrix (Newton-Raphson) is usually replaced by its expectation, known as Fisher scoring. Link: https://en.wikipedia.org/wiki/Scoring_algorithm#Fisher_scoring ...
1
vote
1answer
4k views

Logistic regression: Fisher's scoring iterations do not match the selected iterations in glm

it happened to me that in a logistic regression in R with glm the Fisher scoring iterations in the output are less than the iterations selected with the argument <...
16
votes
2answers
855 views

Why do we make a big fuss about using Fisher scoring when we fit a GLM?

I'm curious about why we treat fitting GLMS as though they were some special optimization problem. Are they? It seems to me that they're just maximum likelihood, and that we write down the ...