Questions tagged [weighted-regression]

Weighted least squares regression is a generalization of OLS regression used when different data points have different importance, or "weights". See also [weighted-data].

Filter by
Sorted by
Tagged with
2
votes
1answer
29 views

Weighted least-squares when weights are not known

I have two questions while I am doing a weighted non-linear least-squares fit. I vaguely remember from some class that the weights should be 1/abs(residuals), or 1/(residuals^2). I am not sure what I ...
1
vote
0answers
71 views

Rewriting integral/summation as weighting estimator

I recently read a biostats paper which featured the following identity: $$ \sum_{y, l, m} y P(y, l, m \mid c, a) \frac{P(l \mid a, c) P\left(m \mid a^{*}, c\right)}{P(l, m \mid c, a)}=E\left(Y \frac{P\...
2
votes
1answer
46 views

GLM weights vs. identical observations

I was recently working on a homework assignment on binary GLMs and the following question came up when comparing solutions with a classmate. The data for the problem was given as a contingency table, ...
0
votes
0answers
12 views

Interpretation of WLS estimates

Why are we able to interpret GLS/WLS estimates as though they were regular OLS estimates? For example, I have been studying the use of Feasible Generalised Least Squares to correct heteroskedasticity. ...
1
vote
0answers
8 views

Estimate Variance-Covariance matrix via Error Propagation of Weighted Least Squares Equation

Given a linear system $b_{obs} = Ax$, how can I derive the covariance of $x$ (i.e. $C$) from the weighted least square solution equation: $$x = (A^TS^{−1}A)^{-1}A^TS^{−1}b_{obs}$$ With $C$ being the ...
2
votes
1answer
43 views

What are the consequences of “duplicating” a subset of data for OLS?

Suppose I have a sample $\{X_i,Y_i\}_{i=1}^n$. Then the OLS estimator of the slope coefficient is given by $$\hat{\beta}=\frac{Cov(X,Y)}{Var(X)}$$ Now suppose I take my data set and replicate a subset ...
1
vote
0answers
13 views

I want to compare two least-squares algorithms. One provides lower CI than the other, but not always. How do you compare?

I am trying to compare two weighing schemes of a least-squares fit for several datasets (about 100). I can calculate the confidence intervals and the sigma values of the parameters (a,b,c). I compare ...
0
votes
0answers
11 views

AIC of a weighted cox regression model (coxphw)

I have to compare three cox-regression models. However, one of these violates the PH assumption. Stratfying the variable did not work because it is already categorical. Thus, weighted cox regression ...
1
vote
1answer
50 views

How does the probability weight, called a pweight in Stata, work?

I am using inverse weights in a panel data analysis (fixed effects) in Stata, to see if my regression coefficients are the same after I reweight the analysis to better represent respondents most ...
1
vote
0answers
54 views

Why Weighted Ridge Regression gives same results as weighted least squares only when solved iteratively?

I was experimenting with weighted ridge regression for a linear system, where the closed-form solution is given by: $$ b =(X^T WX + \lambda I)^{-1}X^T W y $$ and also weighted least squares whose ...
0
votes
0answers
31 views

Why do we need iterative approach for Weighted Least Squares

While I understand why we need to iterate in the case where we are solving non-linear weighted least squares or iteratively reweighted least square. But I do not understand why there is a need to ...
0
votes
0answers
5 views

What happens if the weights are plot?

Binary Regression for Images I'm not very good at coding, so couldn't really test much this hypothesis. But if weights are a sort of "measure of importance" of a particular pixel, then if ...
0
votes
1answer
38 views

Including a weighting variable in a linear regression

I'm looking at how temperature affects length. My length variable is the mean length calculated for every year, it is derived from ~10,000 data points. Not every year had the same sampling effort (e.g....
2
votes
0answers
25 views

How to determine sample weights, when the prevalence in the population is based on percentages

The central question is: How to do I determine the weights, when prevalence in the population is expressed as percentages? I have posted this related question on Stack Overflow: According to the ...
1
vote
1answer
51 views

Should you always weight observations by exposure in a Poisson/Rate GLM

There are many great posts here on the importance of using an offset in a rate regression. For example, if you are modeling the the propensity of murder in towns, using population of town $n_i$ and ...
0
votes
0answers
66 views

Given the following heteroscedastic linear model, why we can assum E(W) = 1 without loss of generality

Consider a heteroscedastic linear model with data consisting of $n$ independent copies of $(Y,X,W)$, where $ Y\in \mathbb{R}^1, X \in \mathbb{R}^p, W >0$ is a weight with expectation one: $$ Y = u +...
1
vote
0answers
16 views

A custom loss function which weights the loss depending on the age of data used

I was wondering if it is possible to weight the loss so that old data are evaluated less than new data. Lets say I have a product which has a trend in value spanning across decades, I want the model ...
1
vote
1answer
35 views

Least squares with variable errors

I am trying to understand the classical linear model $$ Y=X\beta +\varepsilon$$ where instead we have that $\epsilon_k\sim N(0,a_k\sigma^2)$ are independent where $a_k>0$ are given. I want to ...
1
vote
0answers
15 views

How to numerically solve for a variant of the weighted least squares

I encountered the following problem, and I don't have a good idea on where to start. Suppose that we have the following weighted least square problem: $$ \hat{\beta}_{WLS} = \arg\min_\beta(y - X\beta)^...
0
votes
0answers
30 views

BART for aggregated data

I have experimental data with more than a million units in treatment and control and a (numeric) outcome variable "y". I want to detect heterogenous treatment effects along 5 numeric ...
2
votes
0answers
11 views

Using the residual standard deviation of a weighted standard regression for effect size computation

I'm facing a trivial problem but I want to be sure that the approach is appropriate. I have a situation in which n subjects perform a task with k trials for each subject. k is not a fixed number and ...
0
votes
0answers
26 views

Bayesian regression learning (RVM) on weighted data (data with different importance/exposure)

I am working with an extension of the relevance vector machine (RVM) by Tipping (2001), and want to model some data which requires handling of an exposure column (different importance). Is someone ...
3
votes
0answers
50 views

Seeking authoritative references on weighted ANOVA

I would be grateful for any reference, such as a textbook or peer-reviewed journal article (of any vintage), that contains explicit formulas for conducting a weighted ANOVA with arbitrary weights. ...
0
votes
0answers
11 views

Statsmodels heteroskedasticity test: consider weight of data next to difference in variance

For a current project, I have generated a number of data points that show a clear triangular pattern: When trying to quantify this pattern through a Statsmodels Breusch-Pagan test, the data however ...
2
votes
2answers
204 views

Improve Adaboost that using weighted logistic regression instead of decision trees

I implemented Adaboost that using weighted logistic regression instead of decision trees and I managed to get to 0.5% error, I'm trying to improve it for days with no success and I know it possible to ...
1
vote
0answers
31 views

t-test to compare weighted means

I have an unbalanced panel of paired differences. That is, my panel consists of 100 firm time series of paired differences with varying length. I have tested wether the unweighted mean is different ...
2
votes
1answer
89 views

Why is importance-weighted empirical risk minimization finite-sample biased?

Classical risk minimization (RM) minimizes the expected loss over the training distribution $p_{\mathrm{train}}(x)$, $$\theta^*_{RM} = \arg \min_\theta E[\ell(x, \theta)]_{p_{\text{train}}}.$$ As the ...
1
vote
1answer
46 views

Estimation of measurement reliability

I have the following statistical setting: I have two unknowns $a$ and $b$ an I have two measurements of their values corrupted by noise distibuted uniformly in $[-e,e]$ and I have exact value of $e$ ...
0
votes
0answers
43 views

Conditional logit (fixed effects logit) regression for propensity score estimation

Is using clogit problematic in estimating propensity scores (e.g., for weighted OLS regressions)? I have an individual-year panel data and a treatment that occurs every year for some of the ...
3
votes
0answers
51 views

Extending external validation diagnostics to experiments with continuous treatment

Does the external validation diagnostic methods discussed in Stuart et al. (2011) (i.e., inverse propensity score weighted regressions) also apply to the experimental setting in which the treatment is ...
1
vote
0answers
36 views

Compound Independent Probability of Random Events

I'm trying to write a solver for random data and I hit a bit of a statistics brick wall, I wonder if someone might be able to shed some light. If we consider a bag of 10 balls, each one is unique and ...
1
vote
1answer
50 views

In a weighted least squares regression, can we use the weight as a control variable?

I have found Weighted Least Squares with Endogenous Weights but the answers primarily tackle the question of when $w_i$ correlates with $\epsilon_i$. I would like to ask if we use $w_i$ as a control ...
1
vote
1answer
29 views

Algorithm to estimate aggregated outcome variable

I am looking for an algorithm that can estimate weights for an aggregated weighted average. The difficulty is that my outcome variable is an aggregated group variable. I have the following data that ...
0
votes
0answers
64 views

Weighted Least Squares vs Monte Carlo comparison

This is a copy of a question originally posted on stackoverflow I have an experimental dataset of the following values (y, x1, x2, w), where y is the measured quantity, x1 and x2 are the two ...
0
votes
0answers
13 views

Weighted least squares using poisson regression link-log example in R

I'm studying about “weighted least squares” using poisson regression example. And I could got it for poisson link “identity” as following. it would be well because of the same one to minimize (or ...
3
votes
1answer
155 views

Goodness of Fit ot Least Squares with known measurement uncertainty

We want to estimate $\beta$ for $$ y = x\beta + \epsilon $$ where $y$ and $x$ are $n\times 1$ vector and $\epsilon$ is not i.i.d, but $\epsilon \sim N(0, \sigma^2\Omega)$, where $\Omega$ and $W$ are $...
0
votes
0answers
15 views

Calculate Variance Inflation Factors for WLS (using python package?)

Is there an easy extension of VIFs (variance inflation factors) to WLS regression, hopefully easily available in Python? I have an application where I am optimizing the operation a system, modeled as ...
0
votes
0answers
301 views

Interpreting regression coefficients in weighted linear regression

I have a dataset where I'm doing Multiple Linear Regression. Examining the residuals vs fitted plot, it was seen to exhibits heteroskedasticity. As an antidote, I am doing Weighed Least Squares ...
0
votes
0answers
31 views

Weighted zero inflated negative binomial regression

Before encountering the ZINB model, my choice of model was the standard NBR model with the R function glm.nb. My dependent variable is non-negative integer in ...
2
votes
0answers
99 views

Prove variance of locally weighted regression increases with degree

I am interested in proving the following fact for locally weighted polynomial regression from The Elements of Statistical Learning by Hastie et. al. It can be shown that $||l(x_0)||$ increases with ...
1
vote
1answer
52 views

Ratio Estimator as WLS

Apologies in advance for whatever rules this post breaks. I'm looking at a problem where we're currently using a ratio estimator for a certain survey. $$r = \dfrac{\sum_i{y_i}}{\sum_i{x_i}}$$ This ...
2
votes
2answers
370 views

Combining estimates from multivariate multiple regression using MICE in R?

A similar question was posted last year, but it didn't get any love, so!: I'm trying to calculate a pooled estimate after fitting a multivariate regression in multiply imputed data (having used MICE ...
0
votes
0answers
29 views

Best method for rescaling weights [duplicate]

I'm investigating the effect of overhead cover (tree canopy) on the proportion of birds (in relation to mammals) scavenging on carcasses left out in nature. For this study I placed 35 carcasses in ...
2
votes
0answers
91 views

Dynamic Linear Model (DLM) vs Weighted Linear Regression

In Applied Bayesian Forecasting and Time Series Analysis (1994, page 14), they state: The passage of time erodes the value of information - sales figures from six months ago are potentially less ...
2
votes
0answers
19 views

imbalanced regression problem + lower bound prediction + custom error weighting

I'm looking for a simple approach (e.g. defining a new target label / sample weights and then using some off-the-shelf regressor with some standard objective) for the following problem: I want to ...
1
vote
1answer
218 views

appropriate use of prior weight in glm

I am modelling the effects of several variables on the number of scallops caught. The variables are: Stratum (categorical, 23 levels), Vessel (categorical, 3 levels) , Density of scallops (continuous)...
1
vote
1answer
119 views

Comparing mixed models with weighted variance

I'm performing some linear mixed models for a psychological experiment. I'm not a statistician so my knowledge is limited. The basic idea is that: I have an experiment in which I model my response ...
1
vote
1answer
194 views
4
votes
1answer
80 views

Non-Linear regression and variance misspecification

Given a non-linear regression model for cross-section data $$y_i = f(x_i,\theta_0) + \epsilon_i,$$ where it is assumed that $\mathbb E[y_i\lvert x_i] = f(x_i,\theta_0)$, I understand that it is a ...
2
votes
0answers
123 views

Weights formula for Weighted Linear Regression [closed]

I am working with Weighted Linear Regression. I have read that I need to calculate the weights $W_i$ to perform this regression. My question is; Which formula should be used for calculating the ...

1
2 3 4 5
7