Questions tagged [generalized-least-squares]
"Generalized least squares (GLS) is a technique for estimating the unknown parameters in a linear regression model. The GLS is applied when the variances of the observations are unequal (heteroscedasticity), or when there is a certain degree of correlation between the observations." [Wikipedia]
292
questions
0
votes
0
answers
19
views
How to compute the FGLS estimator for simulated data in Matlab (or any other language really)?
first time posting here so please let me know if I can improve my question.
As an exercise, I have to simulate 1000 iterations of a sample of 500 observations from a linear model:
$$ y_i = \beta_0 + \...
- 1
0
votes
1
answer
24
views
On the structure of Iterative Reweighted Least Squares(IRLS)
In Iterative Reweighted Least Squares (IRLS) algorithm, an optimization problem with the weight treated as known is solved in each iteration during solving the main optimization problem. For instance, ...
- 345
1
vote
1
answer
62
views
sample size calculation for generalized least squares regression
I want to calculate the sample size for a multiple linear regression with correlated residuals, specifically using generalized least squares. For OLS, it appears that there a various different rules ...
- 333
0
votes
0
answers
8
views
Can I compare if the outputs of different Generalized least-squares models are distinct in a statistically significant manner?
So I want to compare whether using human scoring or using modelling of avian vision for scoring significantly affects the correlation between rummage brightness and altitude. To do this, I am running ...
- 1
0
votes
0
answers
20
views
Maximum likelihood - First order conditions bivariate Normal
I am trying to estimate the first order conditions from the following:
p1,p2~N(mu1,mu2,sigma1,sigma2,sigma12)
In addition, another variable total_cost=p1x1+p2x2+noise
If I observe x1,x2, and tot_cost, ...
0
votes
0
answers
6
views
Estimating var-cov matrix after FGLS
If I generate a sample that includes p1 and p2 drawn from a bivariate normal distribution. Then I generate a sample that includes n1 and n2 drawn from a different bivariate normal distribution. Then I ...
0
votes
0
answers
42
views
Power analysis for correlated data points
I want to using simulation to determine the effect size that could be detected with my sample size. What makes the simulation a bit complicated is that my dataset is about species traits. It means ...
- 1
1
vote
1
answer
33
views
Comparing the goodness of fit for multiple GLS models applied to different data sets
I fitted simple linear models to several (6) data sets of different sizes, using GLS. All models have the same predictors (but the coefficients are different). I would like to compare the goodness of ...
- 111
0
votes
1
answer
66
views
When to remove interaction term in linear model
I have fitted my data to a linear model in R using generalized least squares accordingly:
mod<-gls(Y ~ A * B+Z,
data)
and used weights ...
- 45
1
vote
0
answers
31
views
Evaluating interaction effects with contrasts
I need help to interpret and understand contrasts of interaction effects for one of my models.
I have a gls model with an interaction effect as follows:
...
- 45
0
votes
0
answers
15
views
prediction interval from gls model with corAR1 or corARMA
In R using the nlme package, how can I compute a prediction interval for a gls model with a corARMA or corAR1 correlation structure?
For instance :
...
- 43
0
votes
0
answers
14
views
How is REML used in gls?
I am trying to understand gls and I have two questions:
(1) How are the standard errors of the coefficients calculated?
(2) what role REML plays in gls?
- 3
2
votes
1
answer
51
views
Unstructured model vs random slope model for repeated measures based on R functions lmer, lme and gls
Here is the dataset for repeated measures:
...
- 707
3
votes
1
answer
212
views
Manually calculate the variance-covariance matrix for a fitted GLS model -- i.e., vcov(glsModel)
I've been working to better understand GLS by manually fitting the parameters in R. In the example below I fit the coefs of a GLS from the nlme package in R. ...
- 85
1
vote
4
answers
226
views
Why the OLS underestimates the variances of the coefficients
CONSEQUENCES OF HETEROSCEDASTICITY
$\textbf{1}$. The presence of heteroscedasticity does not make the OLS estimates of coefficients biased, but it causes the variances of OLS estimates to increase.
$\...
- 310
1
vote
1
answer
43
views
Feasible GLS estimator
I'm approaching for the first time GLS estimators.
Suppose that $\operatorname{Var}(u|x)=\sigma^2 h(x)$, where $h(x)$ is some function of the explanatory variables that determines the ...
- 63
1
vote
1
answer
33
views
statistical method to analysis the trejectories (developmental study) but data is not longitudinal
For example, I am interested in specific brain regions' development over time.
I have data, but data is one-time point data. it is not longitudinal.
But in the data, I have participants aged from 0~17....
- 115
0
votes
0
answers
24
views
1
vote
2
answers
376
views
Multicollinearity and Interaction Effects
I know similar questions were asked before. However, none of the existing answers help me with my problem:
I have a gls model with y=B0+B1X1+B2X2+B3X1X2+e. The VIF value for X1 and the interaction ...
- 11
0
votes
0
answers
94
views
Variance in Generalized Ridge Regression/Weighted Least Squares
I'm following this collection of papers regarding ridge regression, https://arxiv.org/pdf/1509.09169.pdf , and I ran into this section on the mentioning of generalizing ridge regression.
And when ...
- 499
2
votes
0
answers
14
views
Can be Rubin's pooling method (multiple imputation) be combined with Kenward-Roger or Satterthwaite degrees of freedom?
I would like to use multiple imputation algorithm with a Generalized Least Square with Kenward-Roger or Satterthwaite degrees of freedom. Does the commonly implemented Rubin's method account for those ...
- 41
0
votes
0
answers
44
views
If we use the GLS estimation to handle paired data, why can't we use the Welch unpaired t-test as well?
Let's assume I have a paired data to analyse, for example from a longitudinal study.
A typical approach in biosciences is to use marginal (rather than conditional mixed) model, namely the linear model ...
0
votes
0
answers
27
views
In a single-group observational study with change-from-baseline, is it better to model change itself or calculate it later?
Let's assume there is a simple observational study of patients. They are assessed at baseline and 5 timepoints, t0, t1...t5
I am asked to calculate the model-based (to account for all data) difference ...
0
votes
1
answer
71
views
nlme gls different AR(1) correlations for two groups
I'm using the gls procedure of package nlme in R. Is it possible to specify that two different correlation matrices (Ar1 to start with but also, compound symmetry, Toeplitz and unstructured) should be ...
- 113
0
votes
0
answers
13
views
Who are the raters and which ICC is valid for a 2-arm longitudinal RCT?
I'm totally confused by the terminology, as I never used it.
There is a 2-arm randomized study with multiple timepoints: baseline (pre-treatment) and multiple post-treatment assessments.
I need to ...
- 1
1
vote
0
answers
57
views
When using both Sattertwaite degrees of freedom adjustment AND sandwich estimator of the covariance, how do they interact?
In longitudinal experimental trials, where typically (relatively) small sample number of participants are examined, two solutions are proposed to address two kinds of issues:
robust estimator of the ...
- 21
1
vote
1
answer
120
views
What is the matrix $(X'\Omega^{-1}X)$ in Generalized Least Square / Weighted Least Square? [closed]
What is the matrix $(X'\Omega^{-1}X)$ X) in Generalized Least Square / Weighted Least Square?
More precisely,
We know:
- 23
2
votes
0
answers
140
views
Efficiency of FGLS (Feasible GLS) estimator
I hava the following model
$$y=X\beta+u$$
where y and u are $(n\times 1)$ a matrix with full column rank and $\beta$ is an unknown $(k\times 1)$ vector of parameters.
$E(u)=0$ and $var(u)=\sigma^2 V$ ...
- 980
0
votes
1
answer
175
views
Error: "corSymm" objects must be a sequence of consecutive integers when running this code
The Applied Longitudinal Analysis, 2nd Edition book as an R code to replicate table in section 5.7. The data for the TLC trial (tlc.dta) can be downloaded here:https://content.sph.harvard.edu/fitzmaur/...
- 117
0
votes
0
answers
169
views
Weighted Least Squares worse than OLS
I want to compare OLS with WLS. Therefore I came up with a polynomial. I evaluated the polynomial at 2000 datapoints and added GWN with a certain, varying, variance $\sigma_i$. Hence, I have the model
...
- 91
1
vote
1
answer
88
views
Minimizing an objective function with input variables from a correlated error term
I've been reading into how to minimize objective functions and I am curious about the following,
I have a model $y=X \beta +\epsilon$ where $E[\epsilon|X]=0$ and $Var[\epsilon|X]=\Sigma$ where $\Sigma$...
- 499
0
votes
0
answers
48
views
Is it possible to apply the kernel trick to a "mahalnobis distance learner" such as GLS?
1.https://arxiv.org/pdf/0804.1441.pdf
2.https://www.sciencedirect.com/science/article/abs/pii/S0925231210001165
These papers describe kernelizing a mahalanobis distance learner.
I am interested in ...
user318514
1
vote
1
answer
88
views
$Var(\sqrt{n}\left(\hat{\beta}_{F G L S}-\beta\right))=s^2_{FGLS}(\frac{1}{N} \mathbf{X}^{\prime} \hat{\Omega}^{-1} \mathbf{X})^{-1}$?
If so, Wikipedia is wrong?
Wikipedia:
$\sqrt{n}\left(\hat{\beta}_{F G L S}-\beta\right) \stackrel{d}{\rightarrow} \mathcal{N}(0, \mathrm{p}-\lim \left(X^{\prime} \Omega^{-1} X / T\right))$
Powerll ...
user318514
0
votes
1
answer
155
views
Why is Ω unknown in GLS?
We run OLS and found the Homoscedasticity is violated and Hence, we go for GLS.
But from variance-covariance of OLS's error - we have already found the Ω.
Now, if we want to estimate β coefficients ...
- 153
1
vote
1
answer
51
views
high-dimensional GLS
I'm looking for a fast and stable method to compute high-dimensional GLS estimator.
Specifically, let $\mathbf{A}$ be a $p^2 \times m$ matrix with full column rank ($rank(\mathbf{A})=m$), $\mathbf{H}$ ...
- 345
0
votes
0
answers
38
views
nmle:gls with AR or MA terms
I am using nlme:gls to model trends. My understanding is that including AR or MA terms allows the model the correlation structure of the residuals rather than affecting the the regression estimates. ...
- 11
2
votes
1
answer
73
views
When I use the GLS estimation to analyse paired data, should I provide own degrees of freedom = number of pairs or leave the default?
Let's assume I have a repeated data study with 100 subjects.
Now let's assume I have just pairs.
I want to use the GLS estimation for it. Let's assume the compound symmetry residual structure, so we ...
- 61
1
vote
0
answers
60
views
For repeated data, why we don't use just OLS with sandwich SEs but rather GLS or mixed models?
If I have repeated observations and want to summarize the means at each time point, the OLS will give me the true, raw means, while the GLS will give me means dependent on the selected covariance ...
- 61
2
votes
1
answer
566
views
Why Generalized Least Squares?
So it is often advise to use Generalized Least Squares when we have a regression model with non-spherical(i.e. heteroskedastic or autocorrelated) errors. We do so by doing a weighted regression
$$
(y-...
- 321
1
vote
0
answers
76
views
If the normality assumption in the for the GLS estimation fails, would you switch to GEE?
I want a marginal model, ideally fit via GLS. But the normality of residuals doesn't hold. It isn't much skewed, I don't want any transformations. It's just non-normal in shape. Yet still reporting ...
- 41
2
votes
1
answer
185
views
Which mixed model is closest to GLS with unstructured covariance? Random intercept + slopes or random slopes only?
I would like to closely reproduce a model fit via generalized least square method with unstructured residual covariance by using a mixed model.
Which mixed model is closest to GLS with unstructured ...
- 51
1
vote
0
answers
134
views
Is GLS really a special case of the GEE?
I was told, that the GLS is a special case of the GEE, if the conditional distribution is gaussian and the link is identity.
How is that possible?
GLS is a two (or more - IWLS) stage procedure. It ...
- 71
0
votes
1
answer
219
views
What is the purpose to have the "independent" covariance structure in GEE or GLS?
The methods of estimation like GLS or GEE are especially helpful, when there are clusters of data, like repeated observations, many per cluster=subject. Such observations are naturally correlated in ...
- 71
1
vote
1
answer
115
views
Iterative generalized ridge regression
I am looking for some references. Assume I have a series of observable input/output pairs $(y_t, X_t)$ for which I assume the following relations to hold:
$$\beta_t\text{ are i.i.d. }\sim N(\bar{\beta}...
- 35
4
votes
1
answer
364
views
Controlling for phylogeny in linear models
I have run a linear model to test for a relationship between two continuous traits in a sample of 50 taxa. I'm using the packages phylolm and caper in R (just to check they tell me the same thing, ...
- 383
0
votes
1
answer
182
views
Problems finding GLS solution from estimated covariance matrix
I am using a mathematical function to estimate the covariance matrix for some process from the variances and then using this covariance matrix in a generalised least squares estimation of the slope ...
- 1
1
vote
0
answers
50
views
Comparability of variance estimation of residuals of WLS solutions
I have a linear regression model with $p$ parameters
$$ y = X \beta + \varepsilon $$
and multiple datasets for the same model $y_i \in \mathbb{R}^N$ with known weights for each data point $w_i \in \...
- 111
2
votes
1
answer
371
views
Generalized least squares: How are residual correlations estimated?
In a typical linear regression model, we assume that the errors/residuals w.r.t. predictions are i.i.d. and following a normal distribution with a given variance that is the same for all observations.
...
- 547
0
votes
1
answer
586
views
Residual standard error in GLS models
I am conducting a "residual analysis" in R (essentially an adapted Event Study), where I aim to use the RSE to construct a residual confidence interval to identify "outlier" ...
- 98
1
vote
0
answers
61
views
Proving efficiency of GLS over OLS [duplicate]
I'm trying to prove that GLS is more efficient than OLS.
I found out it is, I need to show that:
$\text{Var}^{-1}({\beta}_{GLS})-\text{Var}^{-1}(\beta_{OLS})$ is the positive semi-definite matrix, ...
- 19