Skip to main content

Questions tagged [generalized-least-squares]

"Generalized least squares (GLS) is a technique for estimating the unknown parameters in a linear regression model. The GLS is applied when the variances of the observations are unequal (heteroscedasticity), or when there is a certain degree of correlation between the observations." [Wikipedia]

Filter by
Sorted by
Tagged with
1 vote
0 answers
42 views

Why does accounting for autocorrelated residuals barely help parameter estimation in distributed lag models

This problem has been plaguing me for a long time. Basically, I have a distributed lag model $$y_t=\sum_{i=0}^{p} \beta_i x_{t-i} + u_t.$$ The regression problem is a bit misspecified, so I end up ...
Joe Janssen's user avatar
0 votes
0 answers
21 views

Which is the correct regression model for predicting the association of climate with Julian days nested within decades?

Below is a reproducible example: ...
Rahul's user avatar
  • 23
0 votes
0 answers
29 views

How to derive GEE from GLM?

I am now reading the lecture note from: https://dept.stat.lsa.umich.edu/~kshedden/Courses/Regression_Notes/gee.pdf Why do we have $V_{i}^{-1}(y_{i}-\mu_{i})$? I cannot link the last equation on page ...
doraemon's user avatar
  • 250
1 vote
1 answer
64 views

GLS combined with Random Effect

My data consists of repeated measurements (duration) per individual (ID). The fixed effect is habitat, the goal is to see if duration depends on habitat. However, the variance seems to differ quite a ...
user412691's user avatar
1 vote
0 answers
18 views

How to estimate a feasible Generalized Error-in-Variable Model (combine deming regression/TLS and f-generalized least squares)

I have observational data with spatial structure. A hypothetical dataset could be brain mass for 100 species of birds and body mass for those same species. The data has spatial structure because ...
A Friendly Fish's user avatar
1 vote
0 answers
15 views

Is it possible to control for autocorrelation within individuals and families using GLS corCAR1?

I have a sample of twins with repeated measures of BMI. I want to determine whether intake of a nutrient is associated with BMI trajectories. I have been using GLS in the ...
Gaby's user avatar
  • 11
2 votes
2 answers
79 views

Why does `systemfit` yield identical results for OLS and WLS?

I am estimating a system of seemingly unrelated regressions (SUR) using the systemfit package in R. Each of the equations has one unique regressor and one common ...
Richard Hardy's user avatar
0 votes
0 answers
50 views

Mixed models: equivalence between residual covariance structure and random effects?

Is there a way to specify the covariance structure of a within-subject repeated measures model (MMRM with no random effects) such that the model is mathematically equivalent to a mixed model with ...
Evan's user avatar
  • 225
1 vote
0 answers
17 views

Joint Distribution Formulation of a Spatial X, Spatial Y, and Spatial Error Model

Introductory Problem: I have $n$ points in 3-D space, where I know their X and Y coordinates (not Z), and therefore the distances between points in those 2 dimensions. Each of the three dimensions has ...
A Friendly Fish's user avatar
0 votes
0 answers
48 views

Correlation structure in model residuals

I ran some analysis, and I would feel much more confident with the feedback of the community. I do not provide a MWE (but I would be glad to do so if you feel the need) as I consider this query more ...
Keyvan's user avatar
  • 1
0 votes
0 answers
34 views

Calculating the fitted values from a gls() object in R

I have created a gls() object to create a linear model with AR(1) errors. By all indications this model is a good fit for the data and the resulting model appears ...
chris202's user avatar
1 vote
0 answers
42 views

Closed Form Solution for MLE parameter defining Linear Combination of two multivariate normal distributions

I have one set of $n$ observations which can be described as a single vector sampled from a multivariate normal distribution of the following form: $$ (1-\lambda)\mathbb{I}_n + \lambda \Sigma_{n} $$ ...
A Friendly Fish's user avatar
1 vote
0 answers
25 views

Interpreting GLS in the context of modeling a multivariate distribution

I have some paired observations $(x,y)$ which form some type of distribution. In an attempt to simplify things, I'm trying to fit a distribution of $Y=\beta_0 + \beta_1*X + Error$ (linear regression) ...
A Friendly Fish's user avatar
1 vote
1 answer
45 views

GLS Function - Fitting and Interpretation Issues

I am well aware, that this is a FAQ, but other questions could not provide me answers to my question. Also, I hope this will not be considered a double post, since I have posted this issue with a ...
Vik123's user avatar
  • 83
0 votes
0 answers
51 views

Normal or Linear relationship?

I'm generating simulated data from a multivariate normal distribution with a variance-covariance matrix and then fitting it by either A) finding the maximum likelihood parameter estimates for the ...
A Friendly Fish's user avatar
0 votes
0 answers
20 views

Inconsistency of GLS Estimator in the Presence of Predetermined Regressors and Serial Correlation

Let be the linear model: $$y_i = x_i'\beta + \varepsilon_i$$ Using its matrix form, consider strictly exogenous assumption and spherical assumption, respectivelly: $$E[\varepsilon | X]=0, \quad E[\...
user346624's user avatar
0 votes
0 answers
32 views

Advantages of GLS Estimator for OLS in the Presence of Violated Spherical Assumption

Let be the linear model given by: $$y_i = x_i'\beta + \varepsilon_i$$ Using its matrix form, consider strictly exogenous assumption and spherical assumption, respectivelly: $$E[\varepsilon | X]=0, \...
user346624's user avatar
1 vote
0 answers
33 views

Does GEE (Generalized Estimating Equations) need normality of residuals for inference in case of "approx. Gaussian" response?

A quick question. My intention is to analyze some numerical data across several categories (treat this as ANOVA, if you wish, but I'm going to focus on simple effects) that are "just numeric"...
FatimaShannn23's user avatar
0 votes
0 answers
91 views

Specifying covariance structure for unbalanced data in gls() and lme()

I'm wondering if we could use gls for unbalanced longitudinal data (in the sense that every group has different number of measurements and the measurement ...
George Lu's user avatar
2 votes
1 answer
383 views

R: nlme: can we use varIdent or varFixed to model known variances?

Can anyone familiar with nlme kindly explain how does the varIdent, with option fixed actually work? Documentation says: fixed.....
NeuroPanda's user avatar
0 votes
0 answers
30 views

Recover Variances From GLS Model (Phylogenetic Least Squares on Evolutionary Tree)

I don't know how to calculate the variance of a variable when all of its observations have an arbitrary correlation structure. I am simulating the evolution of animals as they branch apart into ...
A Friendly Fish's user avatar
2 votes
0 answers
73 views

GLS when error covariance matrix depends on regression coefficient

My data is a pair of points (x1, y1) & (x2, y2) [Just in case it's relevant, I explain how the data is created at the end]. I know how the data points are correlated. For a GLS (generalized ...
A Friendly Fish's user avatar
2 votes
1 answer
51 views

Variance at a datapoint?

I have been reading Boyd and Vandenberghe's book on optimisation, and wanted to understand more about weighted least squares in regression which they touch on a bit. This is used when the "...
user1729's user avatar
  • 123
0 votes
0 answers
44 views

How to compute the FGLS estimator for simulated data in Matlab (or any other language really)?

first time posting here so please let me know if I can improve my question. As an exercise, I have to simulate 1000 iterations of a sample of 500 observations from a linear model: $$ y_i = \beta_0 + \...
Bernie's user avatar
  • 1
0 votes
1 answer
183 views

On the structure of Iterative Reweighted Least Squares(IRLS)

In Iterative Reweighted Least Squares (IRLS) algorithm, an optimization problem with the weight treated as known is solved in each iteration during solving the main optimization problem. For instance, ...
user0131's user avatar
  • 357
1 vote
1 answer
169 views

sample size calculation for generalized least squares regression

I want to calculate the sample size for a multiple linear regression with correlated residuals, specifically using generalized least squares. For OLS, it appears that there a various different rules ...
ReadBeard's user avatar
  • 333
0 votes
0 answers
11 views

Can I compare if the outputs of different Generalized least-squares models are distinct in a statistically significant manner?

So I want to compare whether using human scoring or using modelling of avian vision for scoring significantly affects the correlation between rummage brightness and altitude. To do this, I am running ...
Birdman's user avatar
0 votes
0 answers
22 views

Maximum likelihood - First order conditions bivariate Normal

I am trying to estimate the first order conditions from the following: p1,p2~N(mu1,mu2,sigma1,sigma2,sigma12) In addition, another variable total_cost=p1x1+p2x2+noise If I observe x1,x2, and tot_cost, ...
Econometrician101's user avatar
0 votes
0 answers
11 views

Estimating var-cov matrix after FGLS

If I generate a sample that includes p1 and p2 drawn from a bivariate normal distribution. Then I generate a sample that includes n1 and n2 drawn from a different bivariate normal distribution. Then I ...
Econometrician101's user avatar
0 votes
0 answers
50 views

Power analysis for correlated data points

I want to using simulation to determine the effect size that could be detected with my sample size. What makes the simulation a bit complicated is that my dataset is about species traits. It means ...
Emma's user avatar
  • 1
1 vote
1 answer
194 views

Comparing the goodness of fit for multiple GLS models applied to different data sets

I fitted simple linear models to several (6) data sets of different sizes, using GLS. All models have the same predictors (but the coefficients are different). I would like to compare the goodness of ...
soungalo's user avatar
  • 111
0 votes
1 answer
181 views

When to remove interaction term in linear model

I have fitted my data to a linear model in R using generalized least squares accordingly: mod<-gls(Y ~ A * B+Z, data) and used weights ...
Blanca's user avatar
  • 45
1 vote
0 answers
85 views

Evaluating interaction effects with contrasts

I need help to interpret and understand contrasts of interaction effects for one of my models. I have a gls model with an interaction effect as follows: ...
Blanca's user avatar
  • 45
0 votes
0 answers
29 views

How is REML used in gls?

I am trying to understand gls and I have two questions: (1) How are the standard errors of the coefficients calculated? (2) what role REML plays in gls?
les2004's user avatar
2 votes
1 answer
150 views

Unstructured model vs random slope model for repeated measures based on R functions lmer, lme and gls

Here is the dataset for repeated measures: ...
Statisfun's user avatar
  • 737
4 votes
1 answer
502 views

Manually calculate the variance-covariance matrix for a fitted GLS model -- i.e., vcov(glsModel)

I've been working to better understand GLS by manually fitting the parameters in R. In the example below I fit the coefs of a GLS from the nlme package in R. ...
user111024's user avatar
2 votes
4 answers
626 views

Why the OLS underestimates the variances of the coefficients

CONSEQUENCES OF HETEROSCEDASTICITY $\textbf{1}$. The presence of heteroscedasticity does not make the OLS estimates of coefficients biased, but it causes the variances of OLS estimates to increase. $\...
Elisa's user avatar
  • 330
1 vote
1 answer
68 views

Feasible GLS estimator

I'm approaching for the first time GLS estimators. Suppose that $\operatorname{Var}(u|x)=\sigma^2 h(x)$, where $h(x)$ is some function of the explanatory variables that determines the ...
Dimitru's user avatar
  • 185
1 vote
1 answer
37 views

statistical method to analysis the trejectories (developmental study) but data is not longitudinal

For example, I am interested in specific brain regions' development over time. I have data, but data is one-time point data. it is not longitudinal. But in the data, I have participants aged from 0~17....
Tube's user avatar
  • 115
1 vote
2 answers
1k views

Multicollinearity and Interaction Effects

I know similar questions were asked before. However, none of the existing answers help me with my problem: I have a gls model with y=B0+B1X1+B2X2+B3X1X2+e. The VIF value for X1 and the interaction ...
Dave's user avatar
  • 11
0 votes
0 answers
139 views

Variance in Generalized Ridge Regression/Weighted Least Squares

I'm following this collection of papers regarding ridge regression, https://arxiv.org/pdf/1509.09169.pdf , and I ran into this section on the mentioning of generalizing ridge regression. And when ...
Warhawk1987's user avatar
2 votes
0 answers
17 views

Can be Rubin's pooling method (multiple imputation) be combined with Kenward-Roger or Satterthwaite degrees of freedom?

I would like to use multiple imputation algorithm with a Generalized Least Square with Kenward-Roger or Satterthwaite degrees of freedom. Does the commonly implemented Rubin's method account for those ...
Nikaraguien's user avatar
0 votes
0 answers
55 views

If we use the GLS estimation to handle paired data, why can't we use the Welch unpaired t-test as well?

Let's assume I have a paired data to analyse, for example from a longitudinal study. A typical approach in biosciences is to use marginal (rather than conditional mixed) model, namely the linear model ...
YoannakisTsiatis's user avatar
2 votes
1 answer
239 views

nlme gls different AR(1) correlations for two groups

I'm using the gls procedure of package nlme in R. Is it possible to specify that two different correlation matrices (Ar1 to start with but also, compound symmetry, Toeplitz and unstructured) should be ...
BenP's user avatar
  • 1,838
1 vote
0 answers
90 views

When using both Sattertwaite degrees of freedom adjustment AND sandwich estimator of the covariance, how do they interact?

In longitudinal experimental trials, where typically (relatively) small sample number of participants are examined, two solutions are proposed to address two kinds of issues: robust estimator of the ...
Kaloobin's user avatar
1 vote
1 answer
216 views

What is the matrix $(X'\Omega^{-1}X)$ in Generalized Least Square / Weighted Least Square? [closed]

What is the matrix $(X'\Omega^{-1}X)$ X) in Generalized Least Square / Weighted Least Square? More precisely, We know:
Dr. T's user avatar
  • 23
2 votes
0 answers
168 views

Efficiency of FGLS (Feasible GLS) estimator

I hava the following model $$y=X\beta+u$$ where y and u are $(n\times 1)$ a matrix with full column rank and $\beta$ is an unknown $(k\times 1)$ vector of parameters. $E(u)=0$ and $var(u)=\sigma^2 V$ ...
1190's user avatar
  • 1,140
1 vote
1 answer
314 views

Error: "corSymm" objects must be a sequence of consecutive integers when running this code

The Applied Longitudinal Analysis, 2nd Edition book as an R code to replicate table in section 5.7. The data for the TLC trial (tlc.dta) can be downloaded here:https://content.sph.harvard.edu/fitzmaur/...
user1916067's user avatar
0 votes
0 answers
335 views

Weighted Least Squares worse than OLS

I want to compare OLS with WLS. Therefore I came up with a polynomial. I evaluated the polynomial at 2000 datapoints and added GWN with a certain, varying, variance $\sigma_i$. Hence, I have the model ...
Steradiant's user avatar
1 vote
1 answer
113 views

Minimizing an objective function with input variables from a correlated error term

I've been reading into how to minimize objective functions and I am curious about the following, I have a model $y=X \beta +\epsilon$ where $E[\epsilon|X]=0$ and $Var[\epsilon|X]=\Sigma$ where $\Sigma$...
Warhawk1987's user avatar

1
2 3 4 5
7