Questions tagged [generalized-least-squares]
"Generalized least squares (GLS) is a technique for estimating the unknown parameters in a linear regression model. The GLS is applied when the variances of the observations are unequal (heteroscedasticity), or when there is a certain degree of correlation between the observations." [Wikipedia]
306
questions
26
votes
1
answer
2k
views
Bootstrapping Generalized Least Squares
Scenario:
Consider the use of bootstrapping to estimate the distribution of model parameters fitted per a linear or nonlinear generalized least squares model. In particular, assume there is a ...
- 13.4k
12
votes
2
answers
4k
views
Equivalent to Welch's t-test in GLS framework
How can Welch's t-test be expressed as a generalized least squares model?
A standard independent samples t-test (where it is assumed that the samples being compared are drawn from populations with ...
- 1,209
12
votes
1
answer
1k
views
Generalised least squares: from regression coefficients to correlation coefficients?
For least squares with one predictor:
$y = \beta x + \epsilon$
If $x$ and $y$ are standardised prior to fitting (i.e. $\sim N(0,1)$), then:
$\beta$ is the same as the Pearson correlation ...
- 460
11
votes
2
answers
4k
views
How to interpret these custom contrasts?
I am doing a one way ANOVA (per species) with custom contrasts.
...
- 3,748
11
votes
1
answer
2k
views
Difference between GLS and SUR
I've been reading some about Generalized Least Squares (GLS) and trying to tie it back to my basic econometric background. I recall in grad school using Seemingly Unrelated Regression (SUR) which ...
- 5,869
11
votes
2
answers
1k
views
Best method to create growth charts
I have to create charts (similar to growth charts) for children of ages 5 to 15 years (only 5,6,7 etc; there are no fractional values like 2.6 years) for a health variable which is non-negative, ...
- 10k
9
votes
3
answers
763
views
How to do regression with known correlations among the errors?
I was wondering if there is anything you can do when you have a regression problem:
$$\begin{cases}
Y_t = \beta_1x_t + \beta_0 + \varepsilon_t \\
\left(\varepsilon_1,\ldots,\varepsilon_n\right)\...
- 1,679
9
votes
2
answers
9k
views
9
votes
1
answer
6k
views
Why does gls model without random effects yield a similar fit to mixed effects model?
I am trying to answer a question from Pinhiero and Bates Mixed Effects Models in S and S-Plus, explaining how random effects fail to confer any benefit over a gls model that has mixed effects.
This ...
- 2,151
9
votes
1
answer
4k
views
In general, would you always prefer feasible GLS to OLS?
I know that GLS estimators only have exact distributions asymptotically, so the efficiency gains in finite samples are not all that clear. But otherwise, I'm struggling on how to attack this ...
- 303
9
votes
1
answer
555
views
Inference in linear model with conditional heteroskedasticity
Suppose I observe independent variable vectors $\vec{x}$ and $\vec{z}$ and dependent variable $y$. I would like to fit a model of the form:
$$y = \vec{x}^{\top}\vec{\beta_1} + \sigma g\left(\vec{z}^{\...
- 14.8k
8
votes
1
answer
40k
views
How to determine if GLS improves on OLS?
I have a multiple regression model, which I can estimate either with OLS or GLS. The weights for the GLS are estimated exogenously (the dataset for the weights is different from the dataset for the ...
- 539
8
votes
2
answers
15k
views
Lognormal Regression?
I'm trying to model a lognormal response variable. I want to take the log of the response variable and do a least-squares regression line over my predictive variable. However, I'm worried about this....
- 187
8
votes
2
answers
6k
views
Non-Correlated errors from Generalized Least Square model (GLS)
As a financial institution, we often run into analysis of time series data. A lot of times we end up doing regression using time series variables. As this happens, we often encounter residuals with ...
- 181
8
votes
1
answer
8k
views
Fitting a generalized least squares model with correlated data; use ML or REML?
Reading the Linear Mixed Model (LMM) literature I am aware that fitting a model using REML provides better estimates of variance parameters than fitting via ML. However, we should not compare nested ...
- 47.8k
8
votes
1
answer
5k
views
How to understand the vertical bar (pipe) in R formulas [closed]
I came upon this because I wanted to emulate Welch's t-test using gls. I found the answer here:
https://stats.stackexchange.com/a/144480/141304
and it says to add ...
- 1,108
8
votes
1
answer
3k
views
LARS - LASSO with weights
I am interested in solving the following problem
$$ \min_{\boldsymbol{\beta}} \left( \mathbf{y}-\mathbf{X}\boldsymbol{\beta} \right)^T W \left( \mathbf{y}-\mathbf{X}\boldsymbol{\beta} \right) + \...
- 391
7
votes
2
answers
1k
views
Least Squares removing first $k$ observations Woodbury formula?
Given the matrix $X_{n,p}$ from the least squares problem $$ \mathbf{X} \cdot \mathbf{\beta} = z $$
Where the normal equation is:
$$ \mathbf{\hat{\beta}} = \left(\mathbf{X}^T \mathbf{X}\right)^{-1} ...
- 205
7
votes
1
answer
1k
views
How to prove positive (semi-)definitness in matrix notation without numbers
I'd like to show that $$V[\hat\beta_{OLS}]-V[\hat\beta_{GLS}]=\sigma^2(X'X)^{-1}(X'\Omega X)(X'X)^{-1}-\sigma^2(X'\Omega X)^{-1}\geq 0$$ is positive (semi-) definite. $\Omega$ and $X$ are square-...
- 109
7
votes
1
answer
3k
views
Generalized Least Squares using Moore Penrose pseudo inverse
I'm using GLS to fit a model where some independent variables are strongly correlated. Therefore my covariance matrix is singular. I have found that Moore-Penrose pseudo inverse can be used to find an ...
- 71
6
votes
1
answer
740
views
Weighted least squares regression on random data, giving large t-statistics more often than "expected"
My question is about the distribution of the t-statistics in Weighted Least Squares regression.
I'm finding that for a fixed Y and random ...
- 63
6
votes
1
answer
5k
views
Are HAC estimators used for estimation of regression coefficients?
The references I can find on HAC procedures (like Newey-West) in regression focus on the standard error of the estimated regression coefficients and hypothesis testing involving the same. I cannot ...
- 14.8k
6
votes
2
answers
4k
views
ARIMA (with xreg) vs GLS
I am fitting both an arima model (with xreg variables) and a gls model to my data in R software. They both have the same ARMA structure and variables. The ARIMA model fits to the data better. Does ...
- 323
6
votes
1
answer
2k
views
FGLS and time fixed effects
Context: I am performing growth regressions on a panel data set in R, including individual- and time fixed effects. Estimating with OLS delivers results that seem to suffer form serial correlation. ...
6
votes
2
answers
4k
views
Autocorrelation and GLS
If autocorrelation in a model is detected by the Breusch-Godfrey test for r-th order autocorrelation, what is the GLS procedure for "fixing" the autocorrelation problem? And is Cochrane-Orcutt ...
- 108
6
votes
2
answers
3k
views
how to use GLS with correlation structure to compare two temperature time series?
I have 2 time series for temperature recorded with 2 sensors in 2 site in a bay. The sensors were recording with the same frequency and during the same period of time.
The 2 sites have different ...
- 349
6
votes
0
answers
204
views
Is the algorithm of calculating SEs of beta coefficients calculated by the nlme gls finally fixed?
I can read here:
https://www4.stat.ncsu.edu/~davidian/st732/examples/dental_pa.R
and here:
https://math.unm.edu/~luyan/stat57918/week14.pdf
that:
WARNING: There is a MISTAKE in gls(), and it DOES ...
- 163
5
votes
2
answers
4k
views
Prediction with GLS
Let's say I build a Generalized Least Squares model. I follow the standard procedure and first estimate a LM model. Then I create an error-response covariance matrix based on the residuals of this ...
- 3,091
5
votes
1
answer
2k
views
Estimating VAR by GLS versus OLS: efficiency
Suppose I have a VAR model with different regressors in different equations (this could be due to restricting some coefficients of a full VAR($p$) model to zero or having some different exogenous ...
- 67.6k
5
votes
1
answer
4k
views
How to specify in r spatial covariance structure similar to SAS sp(pow) in a marginal model?
I'm currently translating existing code from SAS to R. I'm working on longitudinal data (CD4 count over time). I have the following SAS code :
...
- 173
5
votes
1
answer
313
views
Is there a "generalized least norm" equivalent to generalized least squares?
In a standard regression problem
\begin{equation}
\mathbf{y} = \mathbf{X} \beta + \mathbf{e} \ ,
\end{equation}
the solution to $\beta$ when the system is overdetermined is $\hat{\beta}= \left(\...
- 859
5
votes
1
answer
102
views
Least Squares Solution involving regularizer and weighted sum
I have come across the following cost function:
$$
\text{min}_a\ \ (a^Tx^{(1)} -1)^2 + \sum_{j=1}^M \alpha_j (a^Tx_j^{(2)} +1)^2 + \frac{\lambda}{2}||a||^2
$$
This is a minimization over weight vector ...
- 217
5
votes
1
answer
6k
views
How does R compute r.squared for weighted least squares?
I used R for fitting a linear model using weighted least squares (due to heteroskedastic errors):
$\boldsymbol{y} = \boldsymbol{X}\boldsymbol{\beta} + \boldsymbol{\varepsilon}$, where $E(\boldsymbol{\...
- 131
5
votes
1
answer
716
views
Multivariate linear regression with dependant observations
the Multivariate linear regression model is given by
$\underset{n \times d}{\mathbf{Y}} = \underset{n \times k}{\mathbf{X}} \hspace{2mm}\underset{k \times d}{\mathbf{B}} + \underset{n \times d}{\...
- 152
5
votes
2
answers
441
views
Proving that the estimators of coefficients and variance in GLS model are independent
I have come across this question in a textbook: I have a linear model $Y=Xb+u$ with for instance autocorrelation, in order to introduce GLS $Y^*=X^*b+u^*$ (with $Z^* = \Omega^{-1/2}Z$).
Then an ...
- 1,197
5
votes
1
answer
263
views
Confusion with matrix algebra when deriving GLS
Consider the following linear model
$$y = X\beta + u$$
where $Var(u) = \mathbb{E}(uu') = \sigma^2\Omega$. I watched video on GLS derivation which proceeds as follows: since OLS is BLUE in case of ...
- 1,159
5
votes
1
answer
893
views
Efficiency of GLS over OLS when regressors are not fixed
Suppose we have a regression model
$$
y=X\beta+u,\quad E(u)=0,\quad E(uu')=\Sigma.
$$
Let $\hat\beta$ and $\bar\beta$ respectively denote the OLS and GLS estimator. Then, when $X$ is fixed (or when $X$...
- 205
5
votes
1
answer
3k
views
Least squares regression when data has error bars
Suppose I have some data $(x_i,y_i)$. If we perform ordinary least squares, we can get standard errors of the slope and intercept using estimates like $\hat{Var}(\hat{\beta}) = \hat{\sigma}^2 (X^\top ...
- 2,328
5
votes
0
answers
184
views
Generalized Least Square When Disturbance Covariance Matrix Is Rank Deficient
I cannot find any general results on the following Generalized Least Square (GLS) problem.
Let $Y = X\beta + E$, where $X$ is deterministic and of full column rank $k$, and $E$ is of zero mean, with a ...
- 491
5
votes
0
answers
1k
views
Prediction intervals for generalized least squares model with heteroscedastic errors
I wanted to model some data with heteroscedastic errors using a gls model of the form
...
- 3,210
4
votes
3
answers
2k
views
Do I get the heteroskedasticity-robust standard errors from my OLS or WLS regression?
I have a multiple regression linear model which I ran a simple OLS test on.
I then performed the White test and found that it was heteroskedastic.
Then I performed a Weighted Least Squares regression ...
- 273
4
votes
2
answers
2k
views
Is there a GLS estimator that has lower variance than OLS for sum of parameters in linear model under Gauss-Markov conditions?
I have a model $$Y=\beta_0 + \beta_1 x_1 + \beta_2x_2 +\epsilon$$
I would like the minimum variance unbiased estimate of $\gamma=\beta_1 + \beta_2$. Assuming the Gauss Markov conditions hold, but $...
- 958
4
votes
2
answers
654
views
Can the maximum-likelihood method be derived from something else?
I am an author of a paper, in which we show that the maximum-likelihood (ML) method can be derived a limiting case of an iterated weighted least-squares fit. https://arxiv.org/abs/1807.07911
We, the ...
- 296
4
votes
1
answer
5k
views
GLS versus Robust SE
I have been reading Mixed Effects Models and Extensions in Ecology with R by Zuur et al. where departures from iid errors (heteroscedacity and/or correlation) in linear regression (and glm) are dealt ...
- 8,670
4
votes
1
answer
404
views
Manually calculate the variance-covariance matrix for a fitted GLS model -- i.e., vcov(glsModel)
I've been working to better understand GLS by manually fitting the parameters in R. In the example below I fit the coefs of a GLS from the nlme package in R. ...
- 191
4
votes
1
answer
965
views
How to choose between linear mixed model and GLS
My study is a randomized clinical trial. How to choose between linear mixed model and GLS Linear Model Using Generalized Least Squares?
What are the advantages or disadvantages of both?
I have some ...
- 171
4
votes
1
answer
238
views
Possibility of solution in overdetermined system of moment conditions
Hayashi, in page 207-208 of his book Econometrics, ex.3 (see hint), discusses the possibility that when referring to the moment conditions that will determine the estimator formula, having an ...
- 5,692
4
votes
1
answer
7k
views
Generalized Least Squares vs Ordinary Least Squares under a special case
This question regards the problem of Generalized Least Squares. Vectors and matrices will be denoted in bold.
Premises. Let $N,K$ be given integers, with $K \gg N > 1$. The transpose of matrix $\...
- 518
4
votes
1
answer
1k
views
Why I got different variance-covariance matrices for different subjects from getVarCov function from R nlme package?
I fit a linear model using generalized least squares with gls {nlme} function in R. Then I use a ...
- 527
4
votes
2
answers
1k
views
Forecasting with a VAR estimated by GLS versus OLS
Suppose I have a VAR model with different regressors in different equations
(this could be due to restricting some coefficients of a full VAR($p$) model to zero or having some different exogenous ...
- 67.6k