# Questions tagged [weighted-data]

Datasets where different pieces of data can have different "weights", i.e. different importance.

189 questions
Filter by
Sorted by
Tagged with
9 views

### Rescaling the ormalized weight of a variable greater than 1 [closed]

In my data set, I'm trying to calculate the normalized weight of my cumulative frequency variable. To do this, I divide the cumulative frequency (subset by different unit for every week 't') by the ...
19 views

### Smoothing time series with Adjusted R2-weighted averages

I have two parameters (a,b) resulting from an exponential estimation of a curve. I have estimated this curve every hour for one month. In other words, I have a total of 720 parameters a and b, and I ...
1 vote
14 views

### weighted median result is integer

I am computing the weighted median for a dataset using the R reldist package. The input data are integers but my result is an integer number too. Should't my result ...
• 284
20 views

### How to bias-correct proportions for a prop.test in R?

Let's say I ask 200 participants in a study what experience brought the most joy to their lives: A, B, C, or D. ...
1 vote
25 views

### How reliable is conclusion drawn out from a known data?

Most of time conducting experiment is expensive. Suppose I and my collaborator decided to use a known databank(NHANE, NCI's data) to do some digging with some untested hypothesis given. Most of time ...
• 839
18 views

### How to assign weights to a set of ordered objects?

I have been trying to assign weights to set of objects. The problem that I have is as following. I have a set of objects ordered either ascending or decending. . Each object shall be provided a weight ...
• 103
14 views

### Optimizing weights of elements based on their correlation to the same metric

I'm in my freshman year in college and working on a data science project the goal of which is to measure a country's prosperity. We've grouped similar indicators or elements into one pillar. We have 5 ...
30 views

### Weighting the calculation of sample quantiles

Is there a general method for determining what the weights are when calculating sample quantiles? Specifically, how do you do this in the case of repeat measurements -- can (should) you take the ...
• 121
31 views

### How can I "weight" an average to favor strong contributions from each addend?

tl;dr: I'm assigning "scores" to groups. Groups contain objects with values. I want (50 + 50 / 2) to be way higher than (0 + 100/2) because for my purposes, low scores from one object in a ...
36 views

### Can I use oversampling to increase sample weights?

I am working on a balanced binary classification problem. I don't need to adjust the proportion of either of the classes, as it's already 50/50 . However, some of the samples are more valuable than ...
• 1,082
25 views

### Age adjustment or Age standardization state of the art?

I want to run some comparisons of a quantity between different populations. Such quantity comes from a model that includes, among other variables, the variable age. So, one of the variables affecting ...
31 views

### How do I calculate a finite-population correction for a weighted sample?

I have a weighted sample from a population. These are probability weights (as samples are taken with unequal probability). What's the variance in the estimated total? All samples are exchangeable (...
17 views

### How to weight my data in SAS - categorical and numerical variables

I've collected data from 1000 animals (categorical and numerical variables) from 2 different farms. My problems are: one farm corresponds to 68% of my data, in some of my variables I've observed more ...
13 views

### Domain weights as hyperparameters

Suppose I have data from two different sources, $(X_1, y_1)$ and $(X_2, y_2)$, where $X_1$ and $X_2$ have the same variables but different distributions, and similarly for $y_1$ and $y_2$. I want to ...
• 796
10 views

### Proper statistical test for NGS dataset with two samples containing a library of 12,000 sequences each sorted into 4 bins with counts in each bin

Description of the experiment: A library containing 12,000 unique nucleotide sequences is integrated into two samples, a WT cell line and a knockout cell line. Each cell contains only one reporter ...
18 views

### Applying ranking weights to quantitative mixed-method data?

I have a dataset where some participants were timed doing a task 5 different times with 6 different devices. I want to run a mixed methods ANOVA to find out the mean time differences between the 6 ...
24 views

### Weigh the percentages based on the number of inhabitants

I am analyzing the diffusion of a service in some regions. Is there a legitimate criterion for weighing percentages by the number of inhabitants? I would like to "improve" the results of the ...
• 121
1 vote
70 views

### How do we calculate statistics on data weighted by relative frequencies?

I have the following data with some values and their corresponding weights. groups values weights a 1 0.25 a 3 0.75 b 8 1.00 The weights represent relative frequencies of each value in the group ...
• 113
18 views

### Similarity measure of weighted rankings

I have two algorithms which are going to rank the same set of data, by associating to each data element a real number. I'd like to compare the similarity of the outputs, but taking into account the ...
• 101
9 views

### Determine section weights for Norm Development in IRT calibrated Item bank

I have an item bank containing 60 binary items (fitted to IRT 2PL model), divided into 3 sections (easy, medium and difficult) based on b parameter values (less than -.8 is easy, between -.8 and 0.8 ...
18 views

### weight variable based on opposite direction variable

I have two variables. From these two variables, I want to produce one weight variable. The first (A) is a percentage, 0.0 - 1.0. The second variable (B) is a count, 0 to infinity. I want to weight the ...
• 215
60 views

### Q: Aggregating audience and critic movie ratings, and creating a composite rating of the two

I’m building a movie review aggregation site that combines user ratings and critical reviews for a given movie. The objective is to create a list of the “best of the best” movies which were rated ...
9 views

### Use a calculated weight in an variable

I have a dataset that has 3 variable: Age, Income and Weight In this dataset ...
• 143
1 vote
40 views

### Variance in variance-weighted variance estimate?

Apologies for the confusing title, but I couldn't resist. Much can and has been said about computing the unbiased variance using a sample of points, weighting by the variances of each point (for ...
37 views

### Grouping by similarity of variables as weights (2 variables into one index) and locations (using python): Weighted location clustering with an index

I'm currently in a problem. My task is to group n subzones into larger m zones (n>m, but for this case is not very important to tell the numbers, just looking for an adequate approach). Also, for ...
349 views

### What does the "weights" argument do when fitting Kaplan-Meier curveswith the survival package?

In R's survival package, there is an optional weights argument you can supply when you fit a Kaplan-Meier curve. I can't find any documentation about what this does or what exactly a weighted Kaplan ...
• 2,012
43 views

### Stratified Sampling from a population according to given weights

I have a population and i want to take a sample from it in order to examine a mean of a characteristic (say $p$ the probability of having an infringment).My population is 13996 and are divided into 7 ...
37 views

### Ranking subjective datasets

I hope this is the right forum for this question. Please point me to the right one if it isnt. Suppose there is a factory where there are workers who assemble widgets that come down a manufacturing ...
• 101
11 views

### How would I go about implementing continual learning in a supervised multi-output regression problem?

I want to implement a regressor (such as a neural network) with multiple outputs, based on a vector of inputs. It would be trained on a big initial dataset, before being used by some human user. This ...
33 views

### Success rate in BetaBinomial given weighted samples

I need advice on modeling my use case. Lets assume you have a population of X items and you have N trials. Each item $i$ from the overall population is selected/sampled to be evaluated with ...
14 views

### Compare the average for groups with different cohort sizes

I would like to compare the performance of 3 sales rep. Each sales rep has a before vs after total sales revenue, and a before vs after number of clients. Some big clients may pay a lot, small ...
501 views

### Best way to construct a QQ-plot

I want to assess the normality of a dataset (which is log-normally distributed data transformed back to normal) using a Q-Q plot. I stumbled on the fact that there are many ways to build such a plot, ...
• 183
13 views

### Can you perform time series analysis on weight average values?

My data are markouts for different currency pairs, and how profitable it is in time intervals of 0,5,10, etc, (what i mean in terms of profitability, is that it takes the weighted average of the ...
16 views

### Calculate effort impact score by weighting user complexity ratings

For every project we have a number of "Assets" associated with it. For one of our teams this drives all of their work. There are two type of Assets: Customised and Templatised. With ...
• 101
43 views

### Weighting the loss function based on previous seen true positive rates

Similiar to class imbalance there is always something I would call "learnability imbalance" in multi-class classification. What I mean by that: Even when the classes are evenly distributed ...
• 59
68 views

### Calculating Weighted Quantiles causing NA error by coercion in R

I'm new to R and I'm trying through two different packages and formulas to calculate quantiles from a weighted dataset, as you can see below: ...
2k views

### Can I (justifiably) train a second model only on the observations that a previous model predicted poorly?

Say I commit the following sins while building a predictive model: I take my dataset and split it into four subsets: Three for training (Train_A, Train_B, and Train_C) and one for validation. I ...
• 155
87 views

### NHANES Weighting

I am in the process of analyzing some data derived from 8 cycles of NHANES data. Due to the nature of my research, my sample size is rather small (n=98). Since NHANES surveys utilize a complex ...
• 21
1 vote
94 views

### How can I weight ordinal observations and reduce them to one statistic? [closed]

It will be a complicated question and I try to briefly explain. I am studying on educational inequalities. The survey I use for analysis incudes ordinal variables and the education degree which ...
• 39
1 vote
110 views

### Are there time series models for weighted data?

Let's say I have the following time series: Weekly data spanning a few years; The dependent variable is the proportion of items that pass x. On average, that proportion is around 80%. The number of ...
• 822
1 vote
199 views

### How to compute weights using logistic regression

I'm not interested in creating a logistic regression model which is weighted, I'm interested in using a logistic regression model to compute the weights of a survey. It seems that, given the weights ...
• 858
40 views

### How to properly calculate weighted average when the data is in form of ratio

I had a question related to calculating weighted average, on data example below given definition : Column C = Product Supply Column D = Product Demand Column E = Supply / Demand Column F = Demand ...
844 views

### Meaning of the weight argument in glmer and lmer

I have been looking into how to use the weight argument of glmer/lmer to represent "frequency" weights. I was ...
• 2,741
339 views

### EWMA covariance matrix number of lags

When calculating an exponentially weighted covariance matrix for t observations, formula 10.2 here: https://www.oreilly.com/library/view/analysis-of-financial/9781118017098/c10_level1_1.xhtml Uses ...
1 vote
31 views

### Assign more importance to recent records during training

I have a large dataset with information from the last year. I have to build a classification model in order to predict if a customer will buy a product or not (binary classification). Since in the ...
• 193
1 vote
28 views

### Equation for weighted mean of percentages [closed]

I am trying to get the equation for a weighted average of percentages to be normalized between 0 and 1. Am I doing it correctly? Thank you so much!
• 31
614 views

### Equation for weighted average with normalization(?)

I am trying to understand how I should denote the equation for rescaling the following 7 categories into one value. The final weighted average is 0.57, which is possible by subtracting one and ...
• 31
119 views

### Euclidean distance from zero

I am trying to create my own weights for relative work task importance, or weight. For every task, I have a value of importance, ...
• 31
1 vote
331 views

### Winsorizing propensity scores

Is it kosher? Inverse propensity weights (IPW) has been shown to perform poorly when selection probabilities are small (Kang and Schafer, 2007). Are there any standard solutions to this issue?
• 43
1 vote