# Questions tagged [weighted-data]

Datasets where different pieces of data can have different "weights", i.e. different importance.

189 questions
Filter by
Sorted by
Tagged with
9 views

### Rescaling the ormalized weight of a variable greater than 1 [closed]

In my data set, I'm trying to calculate the normalized weight of my cumulative frequency variable. To do this, I divide the cumulative frequency (subset by different unit for every week 't') by the ...
19 views

### Smoothing time series with Adjusted R2-weighted averages

I have two parameters (a,b) resulting from an exponential estimation of a curve. I have estimated this curve every hour for one month. In other words, I have a total of 720 parameters a and b, and I ...
1 vote
14 views

### weighted median result is integer

I am computing the weighted median for a dataset using the R reldist package. The input data are integers but my result is an integer number too. Should't my result ...
20 views

### How to bias-correct proportions for a prop.test in R?

Let's say I ask 200 participants in a study what experience brought the most joy to their lives: A, B, C, or D. ...
1 vote
25 views

### How reliable is conclusion drawn out from a known data?

Most of time conducting experiment is expensive. Suppose I and my collaborator decided to use a known databank(NHANE, NCI's data) to do some digging with some untested hypothesis given. Most of time ...
18 views

### How to assign weights to a set of ordered objects?

I have been trying to assign weights to set of objects. The problem that I have is as following. I have a set of objects ordered either ascending or decending. . Each object shall be provided a weight ...
14 views

### Optimizing weights of elements based on their correlation to the same metric

I'm in my freshman year in college and working on a data science project the goal of which is to measure a country's prosperity. We've grouped similar indicators or elements into one pillar. We have 5 ...
30 views

### Weighting the calculation of sample quantiles

Is there a general method for determining what the weights are when calculating sample quantiles? Specifically, how do you do this in the case of repeat measurements -- can (should) you take the ...
31 views

### How can I "weight" an average to favor strong contributions from each addend?

tl;dr: I'm assigning "scores" to groups. Groups contain objects with values. I want (50 + 50 / 2) to be way higher than (0 + 100/2) because for my purposes, low scores from one object in a ...
36 views

### Can I use oversampling to increase sample weights?

I am working on a balanced binary classification problem. I don't need to adjust the proportion of either of the classes, as it's already 50/50 . However, some of the samples are more valuable than ...
25 views

### Age adjustment or Age standardization state of the art?

I want to run some comparisons of a quantity between different populations. Such quantity comes from a model that includes, among other variables, the variable age. So, one of the variables affecting ...
31 views

### How do I calculate a finite-population correction for a weighted sample?

I have a weighted sample from a population. These are probability weights (as samples are taken with unequal probability). What's the variance in the estimated total? All samples are exchangeable (...
17 views

### How to weight my data in SAS - categorical and numerical variables

I've collected data from 1000 animals (categorical and numerical variables) from 2 different farms. My problems are: one farm corresponds to 68% of my data, in some of my variables I've observed more ...
13 views

### Domain weights as hyperparameters

Suppose I have data from two different sources, $(X_1, y_1)$ and $(X_2, y_2)$, where $X_1$ and $X_2$ have the same variables but different distributions, and similarly for $y_1$ and $y_2$. I want to ...
10 views

### Proper statistical test for NGS dataset with two samples containing a library of 12,000 sequences each sorted into 4 bins with counts in each bin

Description of the experiment: A library containing 12,000 unique nucleotide sequences is integrated into two samples, a WT cell line and a knockout cell line. Each cell contains only one reporter ...
18 views

### Applying ranking weights to quantitative mixed-method data?

I have a dataset where some participants were timed doing a task 5 different times with 6 different devices. I want to run a mixed methods ANOVA to find out the mean time differences between the 6 ...
24 views

### Weigh the percentages based on the number of inhabitants

I am analyzing the diffusion of a service in some regions. Is there a legitimate criterion for weighing percentages by the number of inhabitants? I would like to "improve" the results of the ...
1 vote
70 views

### How do we calculate statistics on data weighted by relative frequencies?

I have the following data with some values and their corresponding weights. groups values weights a 1 0.25 a 3 0.75 b 8 1.00 The weights represent relative frequencies of each value in the group ...
18 views

### Similarity measure of weighted rankings

I have two algorithms which are going to rank the same set of data, by associating to each data element a real number. I'd like to compare the similarity of the outputs, but taking into account the ...
9 views

### Determine section weights for Norm Development in IRT calibrated Item bank

I have an item bank containing 60 binary items (fitted to IRT 2PL model), divided into 3 sections (easy, medium and difficult) based on b parameter values (less than -.8 is easy, between -.8 and 0.8 ...
18 views

### weight variable based on opposite direction variable

I have two variables. From these two variables, I want to produce one weight variable. The first (A) is a percentage, 0.0 - 1.0. The second variable (B) is a count, 0 to infinity. I want to weight the ...
60 views

### Q: Aggregating audience and critic movie ratings, and creating a composite rating of the two

I’m building a movie review aggregation site that combines user ratings and critical reviews for a given movie. The objective is to create a list of the “best of the best” movies which were rated ...
9 views

### Use a calculated weight in an variable

I have a dataset that has 3 variable: Age, Income and Weight In this dataset ...
1 vote
40 views

### Variance in variance-weighted variance estimate?

Apologies for the confusing title, but I couldn't resist. Much can and has been said about computing the unbiased variance using a sample of points, weighting by the variances of each point (for ...
37 views

### Grouping by similarity of variables as weights (2 variables into one index) and locations (using python): Weighted location clustering with an index

I'm currently in a problem. My task is to group n subzones into larger m zones (n>m, but for this case is not very important to tell the numbers, just looking for an adequate approach). Also, for ...
349 views

### What does the "weights" argument do when fitting Kaplan-Meier curveswith the survival package?

In R's survival package, there is an optional weights argument you can supply when you fit a Kaplan-Meier curve. I can't find any documentation about what this does or what exactly a weighted Kaplan ...
43 views

### Stratified Sampling from a population according to given weights

I have a population and i want to take a sample from it in order to examine a mean of a characteristic (say $p$ the probability of having an infringment).My population is 13996 and are divided into 7 ... 37 views

### Ranking subjective datasets

I hope this is the right forum for this question. Please point me to the right one if it isnt. Suppose there is a factory where there are workers who assemble widgets that come down a manufacturing ...
11 views

### How would I go about implementing continual learning in a supervised multi-output regression problem?

I want to implement a regressor (such as a neural network) with multiple outputs, based on a vector of inputs. It would be trained on a big initial dataset, before being used by some human user. This ...
33 views

### Success rate in BetaBinomial given weighted samples

I need advice on modeling my use case. Lets assume you have a population of X items and you have N trials. Each item $i$ from the overall population is selected/sampled to be evaluated with ...
14 views

### Compare the average for groups with different cohort sizes

I would like to compare the performance of 3 sales rep. Each sales rep has a before vs after total sales revenue, and a before vs after number of clients. Some big clients may pay a lot, small ...
501 views

### Best way to construct a QQ-plot

I want to assess the normality of a dataset (which is log-normally distributed data transformed back to normal) using a Q-Q plot. I stumbled on the fact that there are many ways to build such a plot, ...
13 views

### Can you perform time series analysis on weight average values?

My data are markouts for different currency pairs, and how profitable it is in time intervals of 0,5,10, etc, (what i mean in terms of profitability, is that it takes the weighted average of the ...
16 views

### Calculate effort impact score by weighting user complexity ratings

For every project we have a number of "Assets" associated with it. For one of our teams this drives all of their work. There are two type of Assets: Customised and Templatised. With ...
43 views

### Weighting the loss function based on previous seen true positive rates

Similiar to class imbalance there is always something I would call "learnability imbalance" in multi-class classification. What I mean by that: Even when the classes are evenly distributed ...
68 views

### Calculating Weighted Quantiles causing NA error by coercion in R

I'm new to R and I'm trying through two different packages and formulas to calculate quantiles from a weighted dataset, as you can see below: ...
2k views

### Can I (justifiably) train a second model only on the observations that a previous model predicted poorly?

Say I commit the following sins while building a predictive model: I take my dataset and split it into four subsets: Three for training (Train_A, Train_B, and Train_C) and one for validation. I ...
87 views

### NHANES Weighting

I am in the process of analyzing some data derived from 8 cycles of NHANES data. Due to the nature of my research, my sample size is rather small (n=98). Since NHANES surveys utilize a complex ...
1 vote
94 views

### How can I weight ordinal observations and reduce them to one statistic? [closed]

It will be a complicated question and I try to briefly explain. I am studying on educational inequalities. The survey I use for analysis incudes ordinal variables and the education degree which ...
1 vote
110 views

### Are there time series models for weighted data?

Let's say I have the following time series: Weekly data spanning a few years; The dependent variable is the proportion of items that pass x. On average, that proportion is around 80%. The number of ...
1 vote
199 views

### How to compute weights using logistic regression

I'm not interested in creating a logistic regression model which is weighted, I'm interested in using a logistic regression model to compute the weights of a survey. It seems that, given the weights ...
40 views

### How to properly calculate weighted average when the data is in form of ratio

I had a question related to calculating weighted average, on data example below given definition : Column C = Product Supply Column D = Product Demand Column E = Supply / Demand Column F = Demand ...
844 views

### Meaning of the weight argument in glmer and lmer

I have been looking into how to use the weight argument of glmer/lmer to represent "frequency" weights. I was ...
339 views

### EWMA covariance matrix number of lags

When calculating an exponentially weighted covariance matrix for t observations, formula 10.2 here: https://www.oreilly.com/library/view/analysis-of-financial/9781118017098/c10_level1_1.xhtml Uses ...
1 vote
31 views

### Assign more importance to recent records during training

I have a large dataset with information from the last year. I have to build a classification model in order to predict if a customer will buy a product or not (binary classification). Since in the ...
1 vote
28 views

### Equation for weighted mean of percentages [closed]

I am trying to get the equation for a weighted average of percentages to be normalized between 0 and 1. Am I doing it correctly? Thank you so much!
614 views

### Equation for weighted average with normalization(?)

I am trying to understand how I should denote the equation for rescaling the following 7 categories into one value. The final weighted average is 0.57, which is possible by subtracting one and ...
119 views

### Euclidean distance from zero

I am trying to create my own weights for relative work task importance, or weight. For every task, I have a value of importance, ...
1 vote