Questions tagged [weighted-data]

Datasets where different pieces of data can have different "weights", i.e. different importance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
9 views

Rescaling the ormalized weight of a variable greater than 1 [closed]

In my data set, I'm trying to calculate the normalized weight of my cumulative frequency variable. To do this, I divide the cumulative frequency (subset by different unit for every week 't') by the ...
user avatar
0 votes
0 answers
19 views

Smoothing time series with Adjusted R2-weighted averages

I have two parameters (a,b) resulting from an exponential estimation of a curve. I have estimated this curve every hour for one month. In other words, I have a total of 720 parameters a and b, and I ...
user avatar
1 vote
1 answer
14 views

weighted median result is integer

I am computing the weighted median for a dataset using the R reldist package. The input data are integers but my result is an integer number too. Should't my result ...
user avatar
  • 284
0 votes
0 answers
20 views

How to bias-correct proportions for a prop.test in R?

Let's say I ask 200 participants in a study what experience brought the most joy to their lives: A, B, C, or D. ...
user avatar
1 vote
0 answers
25 views

How reliable is conclusion drawn out from a known data?

Most of time conducting experiment is expensive. Suppose I and my collaborator decided to use a known databank(NHANE, NCI's data) to do some digging with some untested hypothesis given. Most of time ...
user avatar
  • 839
0 votes
0 answers
18 views

How to assign weights to a set of ordered objects?

I have been trying to assign weights to set of objects. The problem that I have is as following. I have a set of objects ordered either ascending or decending. . Each object shall be provided a weight ...
user avatar
  • 103
0 votes
0 answers
14 views

Optimizing weights of elements based on their correlation to the same metric

I'm in my freshman year in college and working on a data science project the goal of which is to measure a country's prosperity. We've grouped similar indicators or elements into one pillar. We have 5 ...
user avatar
  • 1
0 votes
0 answers
30 views

Weighting the calculation of sample quantiles

Is there a general method for determining what the weights are when calculating sample quantiles? Specifically, how do you do this in the case of repeat measurements -- can (should) you take the ...
user avatar
0 votes
0 answers
31 views

How can I "weight" an average to favor strong contributions from each addend?

tl;dr: I'm assigning "scores" to groups. Groups contain objects with values. I want (50 + 50 / 2) to be way higher than (0 + 100/2) because for my purposes, low scores from one object in a ...
user avatar
  • 1
2 votes
1 answer
36 views

Can I use oversampling to increase sample weights?

I am working on a balanced binary classification problem. I don't need to adjust the proportion of either of the classes, as it's already 50/50 . However, some of the samples are more valuable than ...
user avatar
0 votes
1 answer
25 views

Age adjustment or Age standardization state of the art?

I want to run some comparisons of a quantity between different populations. Such quantity comes from a model that includes, among other variables, the variable age. So, one of the variables affecting ...
user avatar
2 votes
0 answers
31 views

How do I calculate a finite-population correction for a weighted sample?

I have a weighted sample from a population. These are probability weights (as samples are taken with unequal probability). What's the variance in the estimated total? All samples are exchangeable (...
user avatar
0 votes
0 answers
17 views

How to weight my data in SAS - categorical and numerical variables

I've collected data from 1000 animals (categorical and numerical variables) from 2 different farms. My problems are: one farm corresponds to 68% of my data, in some of my variables I've observed more ...
user avatar
0 votes
0 answers
13 views

Domain weights as hyperparameters

Suppose I have data from two different sources, $(X_1, y_1)$ and $(X_2, y_2)$, where $X_1$ and $X_2$ have the same variables but different distributions, and similarly for $y_1$ and $y_2$. I want to ...
user avatar
  • 796
0 votes
0 answers
10 views

Proper statistical test for NGS dataset with two samples containing a library of 12,000 sequences each sorted into 4 bins with counts in each bin

Description of the experiment: A library containing 12,000 unique nucleotide sequences is integrated into two samples, a WT cell line and a knockout cell line. Each cell contains only one reporter ...
user avatar
  • 1
0 votes
0 answers
18 views

Applying ranking weights to quantitative mixed-method data?

I have a dataset where some participants were timed doing a task 5 different times with 6 different devices. I want to run a mixed methods ANOVA to find out the mean time differences between the 6 ...
user avatar
  • 1
0 votes
0 answers
24 views

Weigh the percentages based on the number of inhabitants

I am analyzing the diffusion of a service in some regions. Is there a legitimate criterion for weighing percentages by the number of inhabitants? I would like to "improve" the results of the ...
user avatar
  • 121
1 vote
0 answers
70 views

How do we calculate statistics on data weighted by relative frequencies?

I have the following data with some values and their corresponding weights. groups values weights a 1 0.25 a 3 0.75 b 8 1.00 The weights represent relative frequencies of each value in the group ...
user avatar
  • 113
0 votes
0 answers
18 views

Similarity measure of weighted rankings

I have two algorithms which are going to rank the same set of data, by associating to each data element a real number. I'd like to compare the similarity of the outputs, but taking into account the ...
user avatar
  • 101
0 votes
0 answers
9 views

Determine section weights for Norm Development in IRT calibrated Item bank

I have an item bank containing 60 binary items (fitted to IRT 2PL model), divided into 3 sections (easy, medium and difficult) based on b parameter values (less than -.8 is easy, between -.8 and 0.8 ...
user avatar
  • 1
0 votes
0 answers
18 views

weight variable based on opposite direction variable

I have two variables. From these two variables, I want to produce one weight variable. The first (A) is a percentage, 0.0 - 1.0. The second variable (B) is a count, 0 to infinity. I want to weight the ...
user avatar
  • 215
0 votes
1 answer
60 views

Q: Aggregating audience and critic movie ratings, and creating a composite rating of the two

I’m building a movie review aggregation site that combines user ratings and critical reviews for a given movie. The objective is to create a list of the “best of the best” movies which were rated ...
user avatar
0 votes
0 answers
9 views

Use a calculated weight in an variable

I have a dataset that has 3 variable: Age, Income and Weight In this dataset ...
user avatar
  • 143
1 vote
0 answers
40 views

Variance in variance-weighted variance estimate?

Apologies for the confusing title, but I couldn't resist. Much can and has been said about computing the unbiased variance using a sample of points, weighting by the variances of each point (for ...
user avatar
0 votes
0 answers
37 views

Grouping by similarity of variables as weights (2 variables into one index) and locations (using python): Weighted location clustering with an index

I'm currently in a problem. My task is to group n subzones into larger m zones (n>m, but for this case is not very important to tell the numbers, just looking for an adequate approach). Also, for ...
user avatar
3 votes
2 answers
349 views

What does the "weights" argument do when fitting Kaplan-Meier curveswith the survival package?

In R's survival package, there is an optional weights argument you can supply when you fit a Kaplan-Meier curve. I can't find any documentation about what this does or what exactly a weighted Kaplan ...
user avatar
  • 2,012
0 votes
0 answers
43 views

Stratified Sampling from a population according to given weights

I have a population and i want to take a sample from it in order to examine a mean of a characteristic (say $p$ the probability of having an infringment).My population is 13996 and are divided into 7 ...
user avatar
0 votes
0 answers
37 views

Ranking subjective datasets

I hope this is the right forum for this question. Please point me to the right one if it isnt. Suppose there is a factory where there are workers who assemble widgets that come down a manufacturing ...
user avatar
0 votes
0 answers
11 views

How would I go about implementing continual learning in a supervised multi-output regression problem?

I want to implement a regressor (such as a neural network) with multiple outputs, based on a vector of inputs. It would be trained on a big initial dataset, before being used by some human user. This ...
user avatar
0 votes
0 answers
33 views

Success rate in BetaBinomial given weighted samples

I need advice on modeling my use case. Lets assume you have a population of X items and you have N trials. Each item $i$ from the overall population is selected/sampled to be evaluated with ...
user avatar
  • 1
0 votes
0 answers
14 views

Compare the average for groups with different cohort sizes

I would like to compare the performance of 3 sales rep. Each sales rep has a before vs after total sales revenue, and a before vs after number of clients. Some big clients may pay a lot, small ...
user avatar
6 votes
1 answer
501 views

Best way to construct a QQ-plot

I want to assess the normality of a dataset (which is log-normally distributed data transformed back to normal) using a Q-Q plot. I stumbled on the fact that there are many ways to build such a plot, ...
user avatar
  • 183
0 votes
0 answers
13 views

Can you perform time series analysis on weight average values?

My data are markouts for different currency pairs, and how profitable it is in time intervals of 0,5,10, etc, (what i mean in terms of profitability, is that it takes the weighted average of the ...
user avatar
0 votes
0 answers
16 views

Calculate effort impact score by weighting user complexity ratings

For every project we have a number of "Assets" associated with it. For one of our teams this drives all of their work. There are two type of Assets: Customised and Templatised. With ...
user avatar
  • 101
0 votes
0 answers
43 views

Weighting the loss function based on previous seen true positive rates

Similiar to class imbalance there is always something I would call "learnability imbalance" in multi-class classification. What I mean by that: Even when the classes are evenly distributed ...
user avatar
  • 59
0 votes
1 answer
68 views

Calculating Weighted Quantiles causing NA error by coercion in R

I'm new to R and I'm trying through two different packages and formulas to calculate quantiles from a weighted dataset, as you can see below: ...
user avatar
15 votes
2 answers
2k views

Can I (justifiably) train a second model only on the observations that a previous model predicted poorly?

Say I commit the following sins while building a predictive model: I take my dataset and split it into four subsets: Three for training (Train_A, Train_B, and Train_C) and one for validation. I ...
user avatar
  • 155
0 votes
1 answer
87 views

NHANES Weighting

I am in the process of analyzing some data derived from 8 cycles of NHANES data. Due to the nature of my research, my sample size is rather small (n=98). Since NHANES surveys utilize a complex ...
user avatar
  • 21
1 vote
0 answers
94 views

How can I weight ordinal observations and reduce them to one statistic? [closed]

It will be a complicated question and I try to briefly explain. I am studying on educational inequalities. The survey I use for analysis incudes ordinal variables and the education degree which ...
user avatar
1 vote
0 answers
110 views

Are there time series models for weighted data?

Let's say I have the following time series: Weekly data spanning a few years; The dependent variable is the proportion of items that pass x. On average, that proportion is around 80%. The number of ...
user avatar
1 vote
0 answers
199 views

How to compute weights using logistic regression

I'm not interested in creating a logistic regression model which is weighted, I'm interested in using a logistic regression model to compute the weights of a survey. It seems that, given the weights ...
user avatar
  • 858
0 votes
0 answers
40 views

How to properly calculate weighted average when the data is in form of ratio

I had a question related to calculating weighted average, on data example below given definition : Column C = Product Supply Column D = Product Demand Column E = Supply / Demand Column F = Demand ...
user avatar
  • 1
3 votes
0 answers
844 views

Meaning of the weight argument in glmer and lmer

I have been looking into how to use the weight argument of glmer/lmer to represent "frequency" weights. I was ...
user avatar
  • 2,741
0 votes
1 answer
339 views

EWMA covariance matrix number of lags

When calculating an exponentially weighted covariance matrix for t observations, formula 10.2 here: https://www.oreilly.com/library/view/analysis-of-financial/9781118017098/c10_level1_1.xhtml Uses ...
user avatar
1 vote
0 answers
31 views

Assign more importance to recent records during training

I have a large dataset with information from the last year. I have to build a classification model in order to predict if a customer will buy a product or not (binary classification). Since in the ...
user avatar
  • 193
1 vote
0 answers
28 views

Equation for weighted mean of percentages [closed]

I am trying to get the equation for a weighted average of percentages to be normalized between 0 and 1. Am I doing it correctly? Thank you so much!
user avatar
  • 31
2 votes
0 answers
614 views

Equation for weighted average with normalization(?)

I am trying to understand how I should denote the equation for rescaling the following 7 categories into one value. The final weighted average is 0.57, which is possible by subtracting one and ...
user avatar
  • 31
0 votes
0 answers
119 views

Euclidean distance from zero

I am trying to create my own weights for relative work task importance, or weight. For every task, I have a value of importance, ...
user avatar
  • 31
1 vote
1 answer
331 views

Winsorizing propensity scores

Is it kosher? Inverse propensity weights (IPW) has been shown to perform poorly when selection probabilities are small (Kang and Schafer, 2007). Are there any standard solutions to this issue?
user avatar
1 vote
1 answer
161 views

How to use weight variables within aggregated results, for group proportions

If I have the following data, where each row represents and individual response, how would I go about reporting the proportions of different groups? (here the groups are ...
user avatar
  • 858