Questions tagged [weighted-regression]

Weighted least squares regression is a generalization of OLS regression used when different data points have different importance, or "weights". See also [weighted-data].

Filter by
Sorted by
Tagged with
2 votes
1 answer
33 views

Is it possible to estimate IPTW in two different subgroups to evaluate interaction? (Subgroup Balancing Propensity Score?)

This questions needs a toy example to be explained. I apologize if the question is not clear. Suppose we have an observational study in which we want to evalute the association between exposure to ...
user avatar
2 votes
2 answers
53 views

IPTW in Cox Regression model using the WeightIt package - Question on ATT vs. ATE interpretation

I am currently trying to perform some IPTW adjustment in the context of Cox Regression models. I was interested in expanding my understanding of the differences between ATE vs. ATT estimation. I've ...
user avatar
0 votes
0 answers
19 views

Weighting the R-squared as a measure of goodness-of-fit in Linear Regression [duplicate]

I have two observed time series $x_i$ and $y_i$ and I want to test if $x_i$ is a good predictor of of $y_i$. So I run a simple linear regression Y ~ X and use $R^2$ as a measure of goodness of fit. ...
user avatar
  • 1
0 votes
1 answer
47 views

Why Generalized Least Squares?

So it is often advise to use Generalized Least Squares when we have a regression model with non-spherical(i.e. heteroskedastic or autocorrelated) errors. We do so by doing a weighted regression $$ (y-...
user avatar
  • 101
0 votes
0 answers
13 views

How to do weighted Total Least Squares (TLS, aka Orthogonal) Regression faster in R?

I am working on a problem where I am using orthogonal regression (aka Total Least Squares (TLS)). I originally used the odregress() function in R from the pracma package. However, it doesn't allow ...
user avatar
1 vote
1 answer
21 views

Parallel with Weighted Least Squared in Bayesian Regression

I have a dataset with a column of ratios $Y = z_1 / z_2$, which will be my depending variable, and a set of columns that explain $Y$. Here $z_1$ means "imports" and $z_2$ means "exports&...
user avatar
0 votes
0 answers
17 views

Weighted estimation

The following equation was derived from an optimization problem. I have data on the parameters $ b, v, n $ and $ r $ and I would like to estimate the parameter $ a $ that fits the data most closely. I ...
user avatar
  • 21
0 votes
1 answer
31 views

Using WLS vs making the residuals homoscedastic to fit OLS regression

I'm convinced that homoscedasticity (of errors) is not an assumption (at least not explicit) for OLS regression. Also, even though WLS is advocated for heteroscedastic errors, OLS is not particularly ...
user avatar
0 votes
0 answers
17 views

How to assign weights to a set of ordered objects?

I have been trying to assign weights to set of objects. The problem that I have is as following. I have a set of objects ordered either ascending or decending. . Each object shall be provided a weight ...
user avatar
  • 103
0 votes
0 answers
22 views

R package for spatial regression

I want to model counts of a beetle (B) as a function of three continuous dependent variables (PC1, PC2, RL), weighted by a fourth variable (W) using either poisson or quasipoisson regression with ...
user avatar
1 vote
0 answers
14 views

Comparability of variance estimation of residuals of WLS solutions

I have a linear regression model with $p$ parameters $$ y = X \beta + \varepsilon $$ and multiple datasets for the same model $y_i \in \mathbb{R}^N$ with known weights for each data point $w_i \in \...
user avatar
  • 101
0 votes
0 answers
17 views

Determine feature importance of nested regression model

For remote sensing purposes, I'm trying to combine data from two different satellites. To do so, one of the two has a much higher weight than the other, but this is area specific. My goal is to use a ...
user avatar
0 votes
1 answer
119 views

How to get glmnet to work for proportions as response variable

I am trying to run a penalized logistic regression in R. My response variables are proportions (they are winning percentages for a sports team), and I have the number of total games played by each ...
user avatar
0 votes
0 answers
30 views

Regularization: How do you penalize weights of some exact value?

Assume we have a loss function of the form: L(f(X; theta),T) where X is the input dataset and T is the target dataset? Then you would update the the paramters by doing p = p - p.grad where p.grad is ...
user avatar
1 vote
0 answers
40 views

Can I use GLM weights to model whether an outcome *ever* occurs, regardless of frequency?

I realized my original post was really two questions, so I'm splitting it up. My other post describes the same problem, and asks whether it can solved using a mixed effects model. My data include a ...
user avatar
  • 143
0 votes
0 answers
11 views

Weighting for modelling probability of selection

I want to use inverse probability weighting in some regressions and to estimate some weighted means from a non-representative sample. I plan to estimate a probit model for probability of selection ...
user avatar
  • 113
0 votes
1 answer
13 views

Calculating variance / standard errors for a Weighted Repeat Sales model

I'm writing an implementation of the Case-Shiller Real Estate Index, which is based on a variation of the weighted least squares, except for the introduction of a dummy matrix Z. I've calculated the ...
user avatar
0 votes
0 answers
29 views

Deriving initial weights for IRLS in DESeq2's GLM model

DeSEQ2 is a frequently-used R package for researchers studying differential gene expression via changes in molecular markers such as Poly(A) RNA. Understanding how ...
user avatar
  • 3,341
0 votes
0 answers
15 views

Weighted clustered standard errors

I am thinking about a weighted clustered standard error with heteroskedasticity. The estimate can be calculated as follows: \begin{equation} \begin{split} \hat{\beta} &= \left(\sum_g X_g'W_gX_g\...
user avatar
  • 33
3 votes
1 answer
64 views

standard error in Weighted Least Squares in R

I am thinking of using a weighted least square method and checking the meaning of this approach. In particular, I am checking how the method in lm(., weights = w) ...
user avatar
  • 33
0 votes
0 answers
13 views

Domain weights as hyperparameters

Suppose I have data from two different sources, $(X_1, y_1)$ and $(X_2, y_2)$, where $X_1$ and $X_2$ have the same variables but different distributions, and similarly for $y_1$ and $y_2$. I want to ...
user avatar
  • 796
0 votes
0 answers
83 views

How to include standard error of input values in weighted linear regression?

In first step I have made linear regression for every array of 10 points for 25 devices as in the plot I get resistance R = 1/slope for every device and SE(standard error). Every 5 out of 25 devices ...
user avatar
  • 1
0 votes
0 answers
19 views

Binomial GLM with Logit Link on Continuous Data given Frequency Weights [duplicate]

I'm wondering what R is doing in the background when given rate/proportion data and frequency weights. The Binomial GLM should only fit {0,1} data but the results still seem fairly accurate. Does it ...
user avatar
1 vote
1 answer
29 views

Show condition so $\beta_{m+1}$ is the solution of weighted least squares problem

In my exercise we assume that $Y_i|X_i$ has distribution with density $f_i(y_i,\eta_i) $ for $i=1,...,n$ where $\eta_i=X_i^T$ is the linear predictor. The generalized linear model with an exponential ...
user avatar
  • 253
0 votes
1 answer
70 views

Weighted linear regression coefficient without intercept's closed-form solution differs from R's lm(.., weights=)

I am working through Harold Larson's Introduction to Probability Theory and Statistical Inference. In Chapter 9, Example 9.1.4 he discusses the simple linear regression model but where we assume the ...
user avatar
1 vote
0 answers
36 views

Weighted least square regression - different ways of estimating weights

Newbie here. I have a question regarding WLS regression. Specifically, I've come across different ways of estimating weights in WLS regression, the most frequent ones being: ...
user avatar
  • 21
0 votes
0 answers
31 views

Literature on residual analysis

I am looking for literature (papers, books, etc) that could have information about residuals analysis on non-linear regression. I have done a residual analysis in python following the MATLAB Guides ...
user avatar
2 votes
1 answer
88 views

Degrees of freedom for calculating parameter covariance in weighted least squares

Wikipedia says the standard way to estimate the parameter covariance $M^\beta$ for weighted least squares (WLS) is $$ M^\beta = \chi_\nu^2 (X^TWX)^{-1} $$ with $$ \chi_\nu^2=\frac{r^TWr}{\nu} $$ and ...
user avatar
  • 954
1 vote
0 answers
57 views

Differences between least-squares estimate and weighted least-squares estimate when I focus on only one coefficient in a multiple linear regression?

Suppose we have $n$ samples from the following linear model: $$y=x_1 \beta_1+x_2 \beta_2 + e,$$ where $e_i$ i.i.d. comes from $N(0,\sigma^2)$; $x_1$, $x_2$ are centralized with mean $0$. I am only ...
user avatar
  • 95
0 votes
1 answer
57 views

Weights to use for a Weighted Least Squares model regarding tests/deaths

I'm currently doing an imaginary project where I'm investigating a possible correlation between the number of tests done for a specific disease and the number of deaths caused by said disease. I'm ...
user avatar
0 votes
0 answers
42 views

Bayesian Weighted Least Squares Regression - Conjugate Prior with known correlation structure

I found this video (https://www.youtube.com/watch?v=LL3Dx79DIRw) which discusses a particular formulation of Weighted Least Squares Regression in a Baysian perspective. The model is: $$ y \sim Normal(...
user avatar
1 vote
0 answers
62 views

Logistic regression for case-control studies

If I have designed a study where participants from 3 disease groups of fixed size were being sampled and suppose the three groups A, B and C are of sizes n_A=50, n_B=50 and n_C=100. Group A is a ...
user avatar
  • 311
1 vote
1 answer
81 views

Can one compare Hazard Ratio and Average Hazard Ratio?

I would like to compare three predictors of survival by comparing three cox regression models. One model violates the PH assumption which brought me to weighted cox regression as implemented by ...
user avatar
4 votes
1 answer
80 views

Least Squares Solution involving regularizer and weighted sum

I have come across the following cost function: $$ \text{min}_a\ \ (a^Tx^{(1)} -1)^2 + \sum_{j=1}^M \alpha_j (a^Tx_j^{(2)} +1)^2 + \frac{\lambda}{2}||a||^2 $$ This is a minimization over weight vector ...
user avatar
  • 177
1 vote
1 answer
98 views

How to address twang package warnings in mnps function [closed]

I am using twang package as I have an analysis where I want to compare 3 categories and want to compute propensity scores to include as weights in the regression ...
user avatar
0 votes
0 answers
30 views

Least Squares Regression with Custom Weights

I have a dataset in which I have an underlying variable x and for each x I have some forecasts of x. I run a regression in which the forecasts of x are my dependent variable and some news about x is ...
user avatar
  • 35
1 vote
1 answer
45 views

Macro version of RMSE

I want to calculate the error of a prediction on an imbalanced dataset, i.e.: ...
user avatar
  • 133
0 votes
1 answer
76 views

Weighted cox regression: test for goodness of fit?

I want to do a weighted Cox regression analysis (coxphw command in R) since there is violation of proportional hazards assumption in standard Cox model shown by Schoenfeld residuals. What would be the ...
user avatar
  • 11
5 votes
1 answer
716 views

Iteratively Reweighted Least Squares - Weights Confusion

In performing Iteratively Reweighted Least Squares (IRLS) to derive $\hat{\beta}$ estimates for logistic regression, all the resource I've read online say to use weights inversely proportional to the ...
user avatar
  • 625
1 vote
1 answer
507 views

Weights from inverse probability treatment weighting in regression model in R?

I obtained the weights for the data from inverse probability weighting for exposure variable. Can I use these weights in rms::lrm model or ...
user avatar
  • 59
3 votes
1 answer
53 views

Weighted least-squares when weights are not known

I have two questions while I am doing a weighted non-linear least-squares fit. I vaguely remember from some class that the weights should be 1/abs(residuals), or 1/(residuals^2). I am not sure what I ...
user avatar
  • 41
1 vote
0 answers
77 views

Rewriting integral/summation as weighting estimator

I recently read a biostats paper which featured the following identity: $$ \sum_{y, l, m} y P(y, l, m \mid c, a) \frac{P(l \mid a, c) P\left(m \mid a^{*}, c\right)}{P(l, m \mid c, a)}=E\left(Y \frac{P\...
user avatar
  • 11
2 votes
1 answer
246 views

GLM weights vs. identical observations

I was recently working on a homework assignment on binary GLMs and the following question came up when comparing solutions with a classmate. The data for the problem was given as a contingency table, ...
user avatar
1 vote
0 answers
53 views

Estimate Variance-Covariance matrix via Error Propagation of Weighted Least Squares Equation

Given a linear system $b_{obs} = Ax$, how can I derive the covariance of $x$ (i.e. $C$) from the weighted least square solution equation: $$x = (A^TS^{−1}A)^{-1}A^TS^{−1}b_{obs}$$ With $C$ being the ...
user avatar
3 votes
1 answer
171 views

What are the consequences of “duplicating” a subset of data for OLS?

Suppose I have a sample $\{X_i,Y_i\}_{i=1}^n$. Then the OLS estimator of the slope coefficient is given by $$\hat{\beta}=\frac{Cov(X,Y)}{Var(X)}$$ Now suppose I take my data set and replicate a subset ...
user avatar
1 vote
0 answers
13 views

I want to compare two least-squares algorithms. One provides lower CI than the other, but not always. How do you compare?

I am trying to compare two weighing schemes of a least-squares fit for several datasets (about 100). I can calculate the confidence intervals and the sigma values of the parameters (a,b,c). I compare ...
user avatar
  • 41
0 votes
1 answer
84 views

AIC of a weighted cox regression model (coxphw)

I have to compare three cox-regression models. However, one of these violates the PH assumption. Stratfying the variable did not work because it is already categorical. Thus, weighted cox regression ...
user avatar
1 vote
1 answer
894 views

How does the probability weight, called a pweight in Stata, work?

I am using inverse weights in a panel data analysis (fixed effects) in Stata, to see if my regression coefficients are the same after I reweight the analysis to better represent respondents most ...
user avatar
  • 85
1 vote
0 answers
215 views

Why Weighted Ridge Regression gives same results as weighted least squares only when solved iteratively?

I was experimenting with weighted ridge regression for a linear system, where the closed-form solution is given by: $$ b =(X^T WX + \lambda I)^{-1}X^T W y $$ and also weighted least squares whose ...
user avatar
  • 11
0 votes
0 answers
78 views

Why do we need iterative approach for Weighted Least Squares

While I understand why we need to iterate in the case where we are solving non-linear weighted least squares or iteratively reweighted least square. But I do not understand why there is a need to ...
user avatar
  • 11

1
2 3 4 5
7