Questions tagged [irls]
IRLS stands for Iteratively Re-weighted Least Squares. IRLS is a commonly used method to find maximum likelihood estimates when they cannot be found analytically.
13 questions with no upvoted or accepted answers
5
votes
0
answers
426
views
IRLS for truncated normal GLM
I have data for which responses fall in $y \in [0,\infty)$ for which, it seems, the standard GLMs based on, say, gamma or inverse-Gaussian fail since they don't allow responses with values equal to 0. ...
- 553
4
votes
0
answers
75
views
Would this modification accelerate convergence of generalized linear model, or break it?
This page describes the following iteratively reweighted linear least-squares (IRLS) method for solving a generalized linear model (GLM):
let $x_1=0$
for $j=1,2,...$ do
linear ...
- 395
2
votes
0
answers
2k
views
Weighted ridge regression in R
How can we do weighted ridge regression in R?
In MASS package in R, I can do weighted linear regression by passing a weight parameter to lm. It can be seen that ...
- 233
2
votes
0
answers
129
views
Intuitive explanation of failed convergence for GLM
Sometimes the software complains about some matrix not being positive definite. Is it the singularity that's causing issues (because the iterative algorithm involves inverting matrices)? Also, what ...
- 641
2
votes
1
answer
955
views
Initialization of weights in IRLS (Iteratively reweighted least squares)
How to initialize weights in IRLS?
On the wikipedia page about IRLS it is stated that:
$W^{(t)}$ is the diagonal matrix of weights, usually with all elements set initially to: $w^{(0)}_i = 1$
but ...
- 221
1
vote
1
answer
64
views
Fake distributed computation - secure summation on IRLS for binary logistic regression
I am attempting to perform an IRLS algorithm to estimate regression parameters for a logistic regression model.
This is the algorithm that I am following
Select initial values for the regression ...
- 93
1
vote
0
answers
2k
views
Iterative reweighted least squares versus MLE for heteroscedastic errors
Iterative reweighted least squares (IRLS) is used when errors are heteroscedastic. Let us assume that error comes from a distribution where its mean is zero and the variance is a function of the ...
1
vote
0
answers
127
views
Reconcile Different Statements of the IRWLS Algorithm
Different textbooks seem to have different definitions for the weight matrix W in the Iteratively Re-Weighted Least Squares (IRWLS) algorithm. They should be equivalent but I can't seem to piece it ...
- 876
1
vote
0
answers
434
views
Fitting a glm in practice
This question will be a little wordy - I'll try to summarize at the end.
I'm currently working on a machine learning library and I'm implementing GLMs. To fit my models I've been implementing an ...
- 111
1
vote
0
answers
429
views
What is wrong with this implementation of logistic regression (using iterative reweighted least squares)?
I am trying to implement logistic regression using the following algorithm:
fit a simple linear model $y \sim Xb_0$
calculate $W = \frac{e^{Xb_0}}{(1+e^{Xb_0})^2}$
calculate $z = Xb_0 + y \cdot \frac{...
- 2,828
1
vote
0
answers
265
views
Iteratively reweighted least squares : asymmetric weights
For robust m-estimation, all the convergence results I'm aware of assume symmetric weights (eg: Huber function) in their formulation of the iterative reweighted least squares algorithm. Does the ...
1
vote
0
answers
75
views
Is the symmetry of u function for the robust M estimation mandatory?
I have build a $\rho$ function which has the following definition:
\begin{equation}
\rho(x)=
\left\{
\begin{array}{ll}
4- \frac{8}{x^2} \text{if } x \lt-2\\
\frac{x^2}{2} \text{if } x \in [-2,3]...
- 91
0
votes
0
answers
717
views
Fitting heteroscedastic models using the `gls` function
Consider the following heteroscedastic model:
$$y_i = f(x_i, \beta) + g(x_i, \theta)\varepsilon_i, i = 1, \ldots, n, \tag{1}$$
where $f(\cdot, \beta)$ is the regression function and $g(\cdot, \theta)$
...
- 21.2k