Questions tagged [generalized-least-squares]

"Generalized least squares (GLS) is a technique for estimating the unknown parameters in a linear regression model. The GLS is applied when the variances of the observations are unequal (heteroscedasticity), or when there is a certain degree of correlation between the observations." [Wikipedia]

Filter by
Sorted by
Tagged with
0 votes
0 answers
15 views

nmle:gls with AR or MA terms

I am using nlme:gls to model trends. My understanding is that including AR or MA terms allows the model the correlation structure of the residuals rather than affecting the the regression estimates. ...
user avatar
2 votes
1 answer
24 views

When I use the GLS estimation to analyse paired data, should I provide own degrees of freedom = number of pairs or leave the default?

Let's assume I have a repeated data study with 100 subjects. Now let's assume I have just pairs. I want to use the GLS estimation for it. Let's assume the compound symmetry residual structure, so we ...
user avatar
1 vote
0 answers
30 views

For repeated data, why we don't use just OLS with sandwich SEs but rather GLS or mixed models?

If I have repeated observations and want to summarize the means at each time point, the OLS will give me the true, raw means, while the GLS will give me means dependent on the selected covariance ...
user avatar
0 votes
0 answers
13 views

What means to report in longitudinal (repeated data) studies without covariates? Raw meand or model-based (GLS) means? They differ!

I have a study with a few repeated observations per subject, recorded at t0...t3. I want to assess the change from baseline at each time point and apply the Dunnett adjustment to get the corrected p-...
user avatar
0 votes
0 answers
23 views

GLS vs. OLS with sandwich (or GEE with sandwich?) - repeated data (no covariates!) [duplicate]

Let's assume I have a series of repeated observations at time t0..t3. I want to test the difference at each time point vs. t0. When doing a set of independent comparisons "vs. control" (...
user avatar
0 votes
0 answers
12 views

Empirical risk minimization for relu/max loss function

Classical risk minimization (RM) minimizes the expected loss over the training distribution $p_{\mathrm{train}}(x)$, $$\theta^*_{RM} = \arg \min_\theta E_{p_{\text{train}}}[\ell(x, \theta)].$$ As the ...
user avatar
  • 101
1 vote
1 answer
76 views

Why Generalized Least Squares?

So it is often advise to use Generalized Least Squares when we have a regression model with non-spherical(i.e. heteroskedastic or autocorrelated) errors. We do so by doing a weighted regression $$ (y-...
user avatar
  • 111
1 vote
0 answers
19 views

If the normality assumption in the for the GLS estimation fails, would you switch to GEE?

I want a marginal model, ideally fit via GLS. But the normality of residuals doesn't hold. It isn't much skewed, I don't want any transformations. It's just non-normal in shape. Yet still reporting ...
user avatar
0 votes
0 answers
11 views

Is the Generalized Estimating Equtions method a good non-parametric replacement for the Generalized Least Squares?

I want to use GLS on my longitudinal data, but it turns out that residuals are non-normal and is a non-easy way. Not just "transformable" skewness, no known relatioship between mean and ...
user avatar
1 vote
1 answer
43 views

Which mixed model is closest to GLS with unstructured covariance? Random intercept + slopes or random slopes only?

I would like to closely reproduce a model fit via generalized least square method with unstructured residual covariance by using a mixed model. Which mixed model is closest to GLS with unstructured ...
user avatar
  • 31
1 vote
0 answers
18 views

Is GLS really a special case of the GEE?

I was told, that the GLS is a special case of the GEE, if the conditional distribution is gaussian and the link is identity. How is that possible? GLS is a two (or more - IWLS) stage procedure. It ...
user avatar
  • 41
0 votes
1 answer
35 views

What is the purpose to have the "independent" covariance structure in GEE or GLS?

The methods of estimation like GLS or GEE are especially helpful, when there are clusters of data, like repeated observations, many per cluster=subject. Such observations are naturally correlated in ...
user avatar
  • 41
1 vote
1 answer
89 views

Iterative generalized ridge regression

I am looking for some references. Assume I have a series of observable input/output pairs $(y_t, X_t)$ for which I assume the following relations to hold: $$\beta_t\text{ are i.i.d. }\sim N(\bar{\beta}...
user avatar
  • 25
3 votes
1 answer
69 views

Controlling for phylogeny in linear models

I have run a linear model to test for a relationship between two continuous traits in a sample of 50 taxa. I'm using the packages phylolm and caper in R (just to check they tell me the same thing, ...
user avatar
  • 143
0 votes
0 answers
107 views

Problems finding GLS solution from estimated covariance matrix

I am using a mathematical function to estimate the covariance matrix for some process from the variances and then using this covariance matrix in a generalised least squares estimation of the slope ...
user avatar
  • 1
1 vote
0 answers
15 views

Comparability of variance estimation of residuals of WLS solutions

I have a linear regression model with $p$ parameters $$ y = X \beta + \varepsilon $$ and multiple datasets for the same model $y_i \in \mathbb{R}^N$ with known weights for each data point $w_i \in \...
user avatar
  • 111
2 votes
1 answer
62 views

Generalized least squares: How are residual correlations estimated?

In a typical linear regression model, we assume that the errors/residuals w.r.t. predictions are i.i.d. and following a normal distribution with a given variance that is the same for all observations. ...
user avatar
0 votes
1 answer
100 views

Residual standard error in GLS models

I am conducting a "residual analysis" in R (essentially an adapted Event Study), where I aim to use the RSE to construct a residual confidence interval to identify "outlier" ...
user avatar
1 vote
0 answers
41 views

Proving efficiency of GLS over OLS [duplicate]

I'm trying to prove that GLS is more efficient than OLS. I found out it is, I need to show that: $\text{Var}^{-1}({\beta}_{GLS})-\text{Var}^{-1}(\beta_{OLS})$ is the positive semi-definite matrix, ...
user avatar
  • 19
0 votes
0 answers
71 views

Levenberg-Marquardt made scale invariant with diagonal matrix?

From the paper Improvements to the Levenberg-Marquardt algorithm for nonlinear least-squares minimization We now describe how to choose an effective damping matrix $D^TD$. Levenberg originally ...
user avatar
0 votes
1 answer
63 views

How to choose between linear mixed model and GLS

My study is a randomized clinical trial. How to choose between linear mixed model and GLS Linear Model Using Generalized Least Squares? What are the advantages or disadvantages of both? I have some ...
user avatar
  • 41
0 votes
0 answers
16 views

Covariance of the least squares error equal to var*I?

In what case is the covariance of the least squares error equal to $cov(e)= \sigma^2 I $?
user avatar
0 votes
1 answer
30 views

Mean of the variables in GLS estimation

I have a time series whose regression is as: $$ Y_t^* = \beta_1X_t^* + e_t $$ where $Y_t^* = Y_t - Y_{t-1}$ and $X_t^* =X_t-X_{t-1}$. So $\hat\beta_1 = $$ \sum(X_t^*- \bar{X}^*)(Y_t^*-\bar{Y}^*)\over ...
user avatar
0 votes
0 answers
25 views

Estimators for range and nugget parameters extremely correlated

When fitting a generalised least squares (GLS) model to a spatial dataset, I was surprised to find how correlated the estimators for the range and nugget are. I was assuming a covariance matrix $\...
user avatar
  • 1,534
0 votes
0 answers
74 views

What is the relationship between GLS and REML?

I know, that there are several parameter estimation methods for models: OLS, GLS, TLS and maybe others. There is also the MLE - maximum likelihood estimation and the REML for models with non-constant ...
user avatar
1 vote
0 answers
31 views

Propagating uncertainty in least squares regression

I am currently running a nonlinear regression on $y=f(x,\theta)$, the variable $x$ being a known input and $y$ being a measurement result. I would like to estimate the parameters $\theta$ as well as ...
user avatar
1 vote
0 answers
36 views

Power analysis for GLS / OLS Generalized Least Squares

I would like to perform a GLS model to test the effect of several physiological predictors (as covariates) on another physiological trait, in primates. Because the time window between 2 consecutive ...
user avatar
  • 31
0 votes
0 answers
29 views

Generalised least squares regression when variance and regression is heteroscedastic

I am trying to model a continuous response variable using a categorical grouping variable and a continuous covariate. The data looks like this, and there are the same number of data points in each ...
user avatar
0 votes
0 answers
21 views

Calculating Generalised Least Squares Manually

I would like to know how to estimate the error-covariance matrix in order to manually calculate a Generalised Least Squares model. Based on Wikipedia and this paper: https://www.uh.edu/~bsorense/...
user avatar
  • 1
0 votes
0 answers
17 views

How to estimate this model using GLS

Source of Equation: https://www.tandfonline.com/doi/abs/10.1080/096031001300313983 I read the paper and I saw that the following equation was estimated using GLS. I wonder how they can have the ...
user avatar
  • 1
0 votes
1 answer
144 views

emmeans ignoring `adjust`

I am running a gls on a repeated measures design. I am trying to check if my contrasts are being adjusted but I am failing to get any adjustment using either ...
user avatar
1 vote
0 answers
179 views

How to get corrected residuals from nlme::gls model with AR1 structure in R

I am new to time series analysis. I am trying to use nlme::gls to add an AR1 structure to a regression model. I'd like to look at the autocorrelation of the residuals with and without this structure ...
user avatar
0 votes
1 answer
133 views

Mixed Model Repeated Measures (MMRM)

I should specify a repeated measures model where I have two visits for each patient (number of patients =100). I am asking: 1) is this model below correctly specified as a repeated measures model? ...
user avatar
  • 23
5 votes
0 answers
104 views

Generalized Least Square When Disturbance Covariance Matrix Is Rank Deficient

I cannot find any general results on the following Generalized Least Square (GLS) problem. Let $Y = X\beta + E$, where $X$ is deterministic and of full column rank $k$, and $E$ is of zero mean, with a ...
user avatar
0 votes
0 answers
499 views

Prove that the variance of the Generalized Least squares estimator is less than the variance of the OLS estimator

Suppose, we consider the following regression model, $$Y = X\beta + \varepsilon$$ where $\varepsilon$ ~ $N(0, \sigma^2V)$ and V is a known $n\times n$ non-singular, positive definite square matrix. ...
user avatar
  • 3
5 votes
1 answer
83 views

Least Squares Solution involving regularizer and weighted sum

I have come across the following cost function: $$ \text{min}_a\ \ (a^Tx^{(1)} -1)^2 + \sum_{j=1}^M \alpha_j (a^Tx_j^{(2)} +1)^2 + \frac{\lambda}{2}||a||^2 $$ This is a minimization over weight vector ...
user avatar
  • 217
0 votes
1 answer
29 views

How to derive variance from a regression model with only μ as the estimator (i.e. no coefficients)?

I am having trouble deriving the variance of this regression model given below: So far, I have managed to derive the OLS estimate but I just cannot wrap my head around how the variance is derived here:...
user avatar
1 vote
0 answers
12 views

Alternative to ssing linear least squares for solving a matrix equation [closed]

Assume we have the following matrix equation: $$ \Delta Y = \frac{\partial Y}{\partial X}\Delta X $$ where, $Y$ is n-dimensional vector, $\frac{\partial Y}{\partial X}$ is (n,m) dimensional, and $\...
user avatar
  • 53
1 vote
1 answer
129 views

The difference between statistical models and estimation methods, for example GLS and GLM

I was checking out Genarlised Least Squares on Wikipedia and wondered if it had anything to do with Generalised Linear Models, which I have been using for quite a while. The Wikipedia showed me that ...
user avatar
  • 239
1 vote
1 answer
73 views

Unbiased estimation for constrained input-output data?

I am trying to understand the following. I have a series of measured ground true data $Y = (y_1,y_2,\ldots,y_m)$ and a series of estimated data $\hat Y = (\hat y_1, \hat y_2,\ldots,\hat y_m)$. Then, ...
user avatar
  • 81
1 vote
1 answer
39 views

This representation of the precision matrix (Inverse of the covariance matrix) confuses me

I am currently reading the book titled "Generalized Least Squares" by Takaeki Kariya and Hiroshi Kurata. In one section, a General linear regression model of the form \begin{equation} y=X\...
user avatar
  • 1,030
1 vote
0 answers
126 views

GLS estimator - derivation

I'm stuck with the following question: Given the model $$Y_t=\alpha+\beta X_t+u_t\,,$$ where the standard assumptions hold but $Eu_t^2=\sigma^2 X_t^2$, derive the GLS estimator. Basically, all Gauss ...
user avatar
  • 31
0 votes
1 answer
44 views

How to show least square estimator is not BLUE when residual is dependent of each other?

Suppose we have $y=X\beta+\varepsilon$, where $\varepsilon \sim (0,\sigma^2V)$, $\sigma^2$ unknown but $V$ known (we can assume a valid $V$ for this model). Then, by general least square, we can find $...
user avatar
  • 31
1 vote
0 answers
83 views

Unconditional and conditional models

I don't know if the question is worded weirdly, but I'm having difficulties understanding its logic. I have the solution, but if possible, can someone explain the reason behind it? We have two models (...
user avatar
3 votes
2 answers
136 views

What's wrong with this derivation of the GLS of a linear model?

Question: Find the GLS estimator $\hat{\beta}$ for the linear model $$Y=\beta X+\epsilon$$ where $\mathbb{E}\epsilon=0$ and $\text{Cov}(\epsilon)=\sigma^2\Sigma$, where $\Sigma$ is positive definite. ...
user avatar
  • 133
1 vote
0 answers
20 views

OLS and GLS distribution for $b|X$ [closed]

Does anyone now the distribution for $b|X$ for OLS and GLS? \begin{aligned} b^{OLS} &= (X'X)^{-1}X'y \\ b^{GLS} &= (X'\Omega^{-1}X)^{-1}X'\Omega^{-1}y \end{aligned}
user avatar
  • 11
2 votes
0 answers
90 views

Quasi-differencing of irregular AR(1) process

Suppose $\delta_{jt} = x'_{jt}\beta + \xi_{jt}$ where $\xi_{jt}$ is an AR(1) process, i.e. $\xi_{jt} = \rho \xi_{jt-1} + \eta_{jt}$. I am assuming that $\text{Cov}(x_{jt}, \eta_{jt}) = 0$. I know $\...
user avatar
  • 121
0 votes
1 answer
110 views

Errors only in variables model, and polynomial fitting

I have a bunch of data points $(x, y)$, and I know that they fit well to a model of the form $y = a + bx + c x^2$, with $a \approx 0.01, \ b \approx 1\ \textrm{and}\ c \lesssim 0.1$. I'd like to fit ...
user avatar
  • 101
1 vote
1 answer
57 views

GLS based on standard deviation condition

I want to perform a generalized least square model (GLS) with the gas data provided by the NIST/SEMATECH e-Handbook of Statistical Methods [1], where I want to predict pressure, based on temperature. ...
user avatar
  • 13
1 vote
0 answers
94 views

What does it mean to have a model fit via GLS with REML? Aren't GLS and REML two different methods of estimation?

As in the title. I am confused. We often read that a regression model was fit using the OLS, GLS, TLS or ML. But recently I found a text about the analysis of repeated data, where it was modelled ...
user avatar

1
2 3 4 5 6