Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

"Weights" may refer to: (1) observation weights that come from sample surveys -- consider tagging "survey-sampling"; (2) Monte Carlo sample weights that arise when sampling from intractable distributions -- consider tagging "weighted-sampling"; (3) variable weights in statistical or machine learning ...

1
vote
0answers
14 views

Effective sample size of a weighted sample

I am trying to understand the meaning of the effective sample size when we have weights for the sample. I have an intuitive understanding of why effective sample size can be less than the raw sample ...
0
votes
0answers
13 views

R - Replicate Weights

Currently I'm interested in learning how to obtain information from the American Community Survey PUMS files. I have read some of the the ACS documentation and found that to replicate weights I must ...
3
votes
2answers
56 views

Is ratio of ranks a well-studied statistic?

I have two lists of recommendations: that is, two different algorithms have assigned ranks to the same list of objects. I would like to know if they're similar. I specifically care more about ...
0
votes
0answers
9 views

How to test for difference in weighted proportion difference? (and average weighted proportion differences)

Suppose I have a survey where respondents rate different questions as positive (1), negative (-1), or neutral (0), and their responses are given a different weight based on which job they have and ...
1
vote
0answers
9 views

Benefit of scaling survey design weights

In the european social survey the design weights are scaled to the sample size (see quote and source): As with the design weights also the post-stratification weights are scaled to the sample size, ...
0
votes
0answers
7 views

Assigning weights for binary endpoint analysis, using survival analysis

This problem concerns the evaluation of predictors which could potentially be used to decide on which patients to perform colonoscopy. Study design: Analysis of administrative data - patients ...
3
votes
0answers
24 views

Using multinomial logistic regression to make predictions [closed]

I have run a multinomial logistic model in SAS with 5 independant variables and I need to use the results from this model to make forecasts of use of care. I have used the predicted probabilities from ...
1
vote
0answers
8 views

Weighting data with uneven groups

I've seen that you can account for uneven amounts of respondents within different groups, but every example has the same amount starting. How do you handle if you have an uneven starting number per ...
1
vote
0answers
20 views

Shouldn't we consider larger standard errors for effect measures or outcomes that are converted in meta-analysis?

There are methods to convert effect measures in meta-analysis (pdf). There are also methods to convert outcomes; at least, I am aware of the conversion described in Furukawa et al. (2005) from ...
0
votes
0answers
18 views

images in the paper “Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs”

I have some confusion about this image in the paper "Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs" I am wondering what kind of technique they use to lower the dimension of the ...
0
votes
0answers
30 views

Simulating by hand the predictions of a complex mixed model with beta distribution from glmmTMB for diagnostic

I am conducting a GLMM for a meta-analysis using the beta distribution with the package glmmTMB. My response variable is a vector of correlations (No exact 0 or 1), but Fisher’s transformation fails ...
5
votes
2answers
188 views

Regression as a way to determine variable importance

At my work, we employ a nearest neighbor algorithm to classify records. Part of this process, of course, includes determining which features to use as auxiliary information in the algorithm. Also, ...
2
votes
1answer
50 views

Mixing of liquids where concentration follows normal distribution

We have several liquids where the concentration of a certain element follows a normal distribution, and we take a weighted combination of the elements. The concentrations are: $$C_i \sim \text{IID N}...
4
votes
2answers
40 views

How do sample weights work in classification models?

What does it mean to provide weights to each sample in a classification algorithm? How does a classification algorithm (eg. Logistic regression, SVM) use weights to give more emphasis to certain ...
1
vote
1answer
23 views

Potential bias of effect sizes due to ceiling (or floor) effects when weighting effect sizes by their variance for meta-analysis

Regarding meta-analysis, I am wondering whether there is a potential bias due to the ceiling effects (or floor effects) when weighting effect sizes by their sampling error variances. In case of ...
2
votes
2answers
49 views

How to find the weight of the weighted least squares regression analysis?

As the title, I am having trouble to the find weight at the weighted least squares estimation. I found that some people use weights like wts <- 1/fitted(lm(abs(residuals(regmodel.1)) ~ x))^2 or ...
0
votes
0answers
16 views

Weights of ratio of weighted data

Suppose I have the two datasets $x_i$ and $y_i$ with weights $w_{x,i}$ and $w_{y,i}$, respectively ($1 \le i \le N$). The weighted means are for example: $$\bar x = \frac{\sum_i x_i w_{x,i}}{\sum_i ...
2
votes
2answers
92 views

Which weighting to use for regression analyses at different levels of aggregation

I run a study with subjects in 400 groups of heterogeneous sizes ranging from 2 to 20 individuals. I have outcome data at the group level and at the individual level. Treatment was randomly assigned ...
1
vote
2answers
50 views

Reweighting features in PAM/K-means clustering

As stated in Hennig et al. 2016 Handbook of cluster analysis: If for subject matter reasons some variables are more important than others regardless of the within-variable variation, one could ...
1
vote
0answers
18 views

Formulating the Netwon Raphson

If this is the dataset under consideration and ...
0
votes
0answers
33 views

Regression with varying weights within fixed effect units

I'm running a regression where within the fixed effect groups, the weights are not constant. However, since there are many fixed effects, I would like to use an estimation method as implemented for ...
1
vote
0answers
59 views

Adding weights to forest plot with subgroups

I am creating a forest plot with subgroups (roughly following this example). Now I want each subgroups to be weighted as if I would plot them individually in a separate forest plot, and the weights ...
0
votes
0answers
30 views

Interpreting weighted MLE, MoM estimates with stochastic weights

I deal with the weighted probability density $h(x)=w*f(x, \theta)$, where x is the stochastic variable, f is the unweighted probability density, w is stochastic weight depending on x and also measured ...
0
votes
0answers
60 views

lmer in R: How and when apply weights to factor with 3 levels and unbalanced case numbers?

I want to run a mixed effects regression with lmer (package lme4) in R, predicting reaction ...
3
votes
2answers
168 views

Are frequency weights and sampling weights, in practice, the same thing?

Frequency weights indicate how many cases in the population a given observation represents. Sampling weights indicate the probability (sometimes the inverse of the probability) of an observation ...
0
votes
0answers
42 views

Weights vs offsets in logistic tree models

I'm interested in the differences in interpretation and functionality regarding weights and offsets in logistic regression trees. In my case, I am using XGBoost trees for logistic regression where the ...
0
votes
0answers
10 views

make weighted adjustment for sample selection bias

I have a survey results data which is know as biased. Each row record the age, education and ethnicity. I also I have census summary for the national population. So I know the basic results of ...
2
votes
1answer
30 views

How to compute adjustment weights to match marginal sums of a matrix?

I have a feeling this should be very simple, but I somehow got stuck thinking about it. I have $\it X$, which is a 15 x 18 matrix containing non-negative real values. I smoothed this matrix using a ...
2
votes
1answer
36 views

Can I use combination of eigenvectors as a single vector to explain most of variance?

I have a problem trying to find a combination (or weighted average) of variables (statistics) that best explains the sample statistics. A – n x p matrix (n: observations p: variables, here are ...
1
vote
0answers
46 views

Inverse weighted average

Is there something like an "inverse weighted average"? In my case, I have daily prices and daily units sold of a bunch of products and I would like to give prices with more units higher weights than ...
2
votes
1answer
31 views

Standard deviation of a ratio and calculation of weight

I have several "skilled" and "unskilled" wage observations for a number of countries, and would like to construct a single skilled-unskilled wage premium by country as the ratio of the simple average ...
0
votes
0answers
42 views

Can I compare coefficient values from different models?

I'm modeling conversions with a logistic regression. Each conversion can have up to 3 events (page visit, fill questionnaire, and call). I have 3 domains let's call them A,B, and C. Rather than using ...
0
votes
0answers
17 views

Final model equation for logistic regression with weights

I have built a logistic regression model with weights using glm function in R, as the event was rare. I used all 1's and 10% of 0's, then assigned a weight of 10 to each of the 0- rows and no weight (...
1
vote
0answers
10 views

What weight and how to use it for a 3 country analysis using the EWCS in Stata?

I'm fairly new to weights because the databases I used previously were not samples but actual populations I wanted to study, so apologies if all of this sounds very noobish. The way I understand it, ...
2
votes
0answers
127 views

Diagnostic residuals for Beta GLMM weighted by sample size (Meta-analysis) using glmmTMB

I am conducting a GLMM for a meta-analysis using the beta distribution with the package glmmTMB. My response variable is a vector of correlations (No exact 0 or 1), but Fisher’s transformation fails ...
0
votes
1answer
30 views

How to interpret the sampling weight of each observation in the dataset? [closed]

I have a data set containing a sampling weight for each observation. Here are the top observations from the data set : My question is what does this sampling (survey) weight means? How to interpret ...
1
vote
0answers
30 views

Can analytic weights be converted into sampling weights?

I am running a linear probability model, so I know I need to correct for heteroskedasticity. Since I know the variance of the error (from theory) to be $y(\pmb x)(1-y(\pmb x))$ where $y$ is the ...
0
votes
0answers
93 views

Weight minority classes using XGBoost (Multiclass)

I'm dealing with a multiclass problem in R using XGBoost. The dataset has 3 Classes representing the following proportion: 20% - 75% - 5%. Given the description above, it would be awesome some tips ...
0
votes
1answer
244 views

How to change a weight/bias with gradient

After watching 3Blue1Brown's tutorial series, and an array of others, I'm attempting to make my own neural network from scratch. So far, I'm able to calculate the gradient for each of the weights and ...
1
vote
0answers
10 views

Find a cost-sensitive multiclass algorithm

I am working on decision trees which can directly work in a multiclass context. My aim is reduce the misclassification's errors of a decision tree by improving its ability to tackle imbalanced ...
0
votes
1answer
102 views

How to use Tukey's Biweight Function to appropriately weight outliers to generate a normal distribution

I am working with a distribution that has outliers beyond 1.5*3rd Qu.. I'm using Shankar, et al. Recommendations for the validation of immunoassays used for ...
2
votes
0answers
17 views

What is the effect of more than one chance of selection when sampling?

I am designing a clustered sample. Within each cluster there are 3 groups that will be sampled from - two at a higher rate than the other. The two subgroups that will be sampled at a higher rate are ...
0
votes
1answer
38 views

Free lunch Autoencoder? Data dimensionality reduction

I came across Autoencoders, and saw one example were no activation is used - it's simply a linear transform to lower dimension and then back up $$ B(Ax+a)+b=x$$ with $x\in \mathbb{R}^d$ and $A\in \...
0
votes
0answers
28 views

Weighting covariates in a regression

Sometimes I hear about people talk about how this or that covariate or feature is "weighted" in a predictive model, but I can't find an easy mechanism to do that in ...
1
vote
0answers
38 views

How to weigh samples (representative of population) when comparing group means

I want to clarify my understanding of using sample weights when comparing means of sub-groups of a sample. Here is an artificial case with data that can be replicated in r (see code below): Let’s ...
0
votes
0answers
67 views

Weighted least squares: how do I find weights

I have a dataset of the cars and their corresponding speed and distance and the R output. I am asked to find weights for the first 2 cars. First I find the residual for each car: $e_i=Y_i-\hat{Y}_i$ ...
5
votes
1answer
1k views

A simple Neural Network, finding weights to achieve 100% accuracy

So I've been watching lectures and doing the problem sheets from the class CS 229 2017, taught at Stanford by Andrew Ng. In problem sheet 3 he puts forward the following question: $\textbf{So this is ...
0
votes
0answers
19 views

Finding weights and y-values in multiple linear regression?

Suppose I have observations from 15 hospitals that show its expenses, number of services offered, number of employees and the reciprocal of its average rank. I want to use these 4 independent ...
2
votes
1answer
151 views

Is a weighted average unambiguously superior to an unweighted average?

I want to describe a statistic on my data. For example, I have data on firms (companies), and I have a measure of $x$ for each of them. Now, firms differ in size, which could be understood in terms of ...
2
votes
0answers
67 views

Equivalence of svyglm and glm for simple random surveys

I have been exploring the use of the svyglm function in R's survey package to analyse surveys with both equal and unequal sampling probabilites. For an unequal ...