Questions tagged [weighted-data]

Datasets where different pieces of data can have different "weights", i.e. different importance.

Filter by
Sorted by
Tagged with
0
votes
1answer
21 views

Q: Aggregating audience and critic movie ratings, and creating a composite rating of the two

I’m building a movie review aggregation site that combines user ratings and critical reviews for a given movie. The objective is to create a list of the “best of the best” movies which were rated ...
0
votes
0answers
7 views

Use a calculated weight in an variable

I have a dataset that has 3 variable: Age, Income and Weight In this dataset ...
1
vote
0answers
25 views

Variance in variance-weighted variance estimate?

Apologies for the confusing title, but I couldn't resist. Much can and has been said about computing the unbiased variance using a sample of points, weighting by the variances of each point (for ...
0
votes
0answers
18 views

Grouping by similarity of variables as weights (2 variables into one index) and locations (using python): Weighted location clustering with an index

I'm currently in a problem. My task is to group n subzones into larger m zones (n>m, but for this case is not very important to tell the numbers, just looking for an adequate approach). Also, for ...
1
vote
2answers
39 views

What does the "weights" argument do when fitting Kaplan-Meier curveswith the survival package?

In R's survival package, there is an optional weights argument you can supply when you fit a Kaplan-Meier curve. I can't find any documentation about what this does or what exactly a weighted Kaplan ...
0
votes
0answers
22 views

Stratified Sampling from a population according to given weights

I have a population and i want to take a sample from it in order to examine a mean of a characteristic (say $p$ the probability of having an infringment).My population is 13996 and are divided into 7 ...
0
votes
0answers
36 views

Ranking subjective datasets

I hope this is the right forum for this question. Please point me to the right one if it isnt. Suppose there is a factory where there are workers who assemble widgets that come down a manufacturing ...
0
votes
0answers
6 views

How would I go about implementing continual learning in a supervised multi-output regression problem?

I want to implement a regressor (such as a neural network) with multiple outputs, based on a vector of inputs. It would be trained on a big initial dataset, before being used by some human user. This ...
0
votes
0answers
25 views

Success rate in BetaBinomial given weighted samples

I need advice on modeling my use case. Lets assume you have a population of X items and you have N trials. Each item $i$ from the overall population is selected/sampled to be evaluated with ...
0
votes
0answers
9 views

Compare the average for groups with different cohort sizes

I would like to compare the performance of 3 sales rep. Each sales rep has a before vs after total sales revenue, and a before vs after number of clients. Some big clients may pay a lot, small ...
5
votes
1answer
199 views

Best way to construct a QQ-plot

I want to assess the normality of a dataset (which is log-normally distributed data transformed back to normal) using a Q-Q plot. I stumbled on the fact that there are many ways to build such a plot, ...
0
votes
0answers
12 views

Can you perform time series analysis on weight average values?

My data are markouts for different currency pairs, and how profitable it is in time intervals of 0,5,10, etc, (what i mean in terms of profitability, is that it takes the weighted average of the ...
0
votes
0answers
8 views

Calculate effort impact score by weighting user complexity ratings

For every project we have a number of "Assets" associated with it. For one of our teams this drives all of their work. There are two type of Assets: Customised and Templatised. With ...
0
votes
0answers
40 views

Weighting the loss function based on previous seen true positive rates

Similiar to class imbalance there is always something I would call "learnability imbalance" in multi-class classification. What I mean by that: Even when the classes are evenly distributed ...
0
votes
1answer
24 views

Calculating Weighted Quantiles causing NA error by coercion in R

I'm new to R and I'm trying through two different packages and formulas to calculate quantiles from a weighted dataset, as you can see below: ...
15
votes
2answers
2k views

Can I (justifiably) train a second model only on the observations that a previous model predicted poorly?

Say I commit the following sins while building a predictive model: I take my dataset and split it into four subsets: Three for training (Train_A, Train_B, and Train_C) and one for validation. I ...
0
votes
1answer
34 views

NHANES Weighting

I am in the process of analyzing some data derived from 8 cycles of NHANES data. Due to the nature of my research, my sample size is rather small (n=98). Since NHANES surveys utilize a complex ...
0
votes
0answers
14 views

Unusual p-values after weighting

I'm still new to R and most probably this is a rookie question, but maybe some of you could help me understand what is happening. I'm analyzing some results of an experiment in which I have three ...
0
votes
0answers
8 views

Case weighted classification in Python

I'm trying to find a good method for showing whether or not some variables are statistically significant explainers of a binary (0, 1) response in Python. Catch is, I'd like to weight the data points ...
0
votes
0answers
14 views

Weighted similarity coefficient for binary data?

I would like to evaluate the accuracy of land degradation maps I have estimated using remote sensing compared against observed field land degradation maps. The maps are classified in a binary way and ...
1
vote
0answers
23 views

How can I weight ordinal observations and reduce them to one statistic? [closed]

It will be a complicated question and I try to briefly explain. I am studying on educational inequalities. The survey I use for analysis incudes ordinal variables and the education degree which ...
0
votes
0answers
32 views

Variables' weights using PCA

I have recently read a paper where the authors applied PCA to determine the weights of the variables used to calculate a composite index. In the methodology, they mentioned that for a set of $N$ ...
0
votes
0answers
21 views

K-means++ for weighted clustering

I have implemented k-means for weighted points; that is, the final clusters take into account the fact that each input point is weighted. I wanted to initialize the clusters using k-means++, and I was ...
1
vote
0answers
46 views

Are there time series models for weighted data?

Let's say I have the following time series: Weekly data spanning a few years; The dependent variable is the proportion of items that pass x. On average, that proportion is around 80%. The number of ...
0
votes
0answers
26 views

Bayesian regression learning (RVM) on weighted data (data with different importance/exposure)

I am working with an extension of the relevance vector machine (RVM) by Tipping (2001), and want to model some data which requires handling of an exposure column (different importance). Is someone ...
1
vote
0answers
75 views

How to compute weights using logistic regression

I'm not interested in creating a logistic regression model which is weighted, I'm interested in using a logistic regression model to compute the weights of a survey. It seems that, given the weights ...
0
votes
0answers
12 views

Assigning weights to a list of averages when all I have is the mean, standard deviation, and number of samples

I have some data which, essentially, gives the change in value of a particular stock option according to 1 movement in price of the underlying stock value. For example, say stock 'ABC' has 4 different ...
0
votes
0answers
34 views

How to properly calculate weighted average when the data is in form of ratio

I had a question related to calculating weighted average, on data example below given definition : Column C = Product Supply Column D = Product Demand Column E = Supply / Demand Column F = Demand ...
2
votes
0answers
407 views

Meaning of the weight argument in glmer and lmer

I have been looking into how to use the weight argument of glmer/lmer to represent "frequency" weights. I was ...
0
votes
1answer
161 views

EWMA covariance matrix number of lags

When calculating an exponentially weighted covariance matrix for t observations, formula 10.2 here: https://www.oreilly.com/library/view/analysis-of-financial/9781118017098/c10_level1_1.xhtml Uses ...
1
vote
0answers
23 views

Assign more importance to recent records during training

I have a large dataset with information from the last year. I have to build a classification model in order to predict if a customer will buy a product or not (binary classification). Since in the ...
1
vote
0answers
25 views

Equation for weighted mean of percentages [closed]

I am trying to get the equation for a weighted average of percentages to be normalized between 0 and 1. Am I doing it correctly? Thank you so much!
2
votes
0answers
349 views

Equation for weighted average with normalization(?)

I am trying to understand how I should denote the equation for rescaling the following 7 categories into one value. The final weighted average is 0.57, which is possible by subtracting one and ...
0
votes
0answers
74 views

Euclidean distance from zero

I am trying to create my own weights for relative work task importance, or weight. For every task, I have a value of importance, ...
1
vote
1answer
186 views

Winsorizing propensity scores

Is it kosher? Inverse propensity weights (IPW) has been shown to perform poorly when selection probabilities are small (Kang and Schafer, 2007). Are there any standard solutions to this issue?
0
votes
1answer
66 views

How to use weight variables within aggregated results, for group proportions

If I have the following data, where each row represents and individual response, how would I go about reporting the proportions of different groups? (here the groups are ...
1
vote
0answers
26 views

Are probability weights "relative", i.e. can they be rescaled and work in the same way?

I am using survey data from different countries, and I pool them together for part of the analysis. Now, weights puzzle me for two reasons: 1) some of them are below 0 (i.e. the probability of being ...
0
votes
1answer
169 views

Bayesian Statistics - How to weight a Poisson distributed response

I have a Bayesian GLM where the response that I'm interested in is count data. I want to weight the the response by the variance to account for uncertainty in the measurement. If the response was ...
0
votes
1answer
888 views

How to calculate the confidence interval with weighted data?

I've done some search for similar questions, but they're not the same as what I'm trying to get. Assume that there's a server that handles requests $r$ and returns a set of items $I_{r}$ of random ...
0
votes
0answers
19 views

Help with weighting sample according to population

I am a beginner with basic knowledge of statistics - just learning. I have a doubt regarding weighting survey sample distribution to population distribution. I have to create a weighting variable that ...
0
votes
1answer
37 views

How important are the survey weights?

I run logit model with a cross-sectional dataset of Indian individuals. I am using descriptive statistics of the same dataset to justify and interpret the estimations of logit model. However, I must ...
1
vote
0answers
98 views

sklearn f1_score=weighted not matching sample_weight specification

I am trying to figure out exactly what this is doing: sklearn.metrics.f1_score(y_pred, y_test, sample_weight=[...]) Numerically it simply does not seem to be ...
1
vote
0answers
22 views

How can I aggregate the estimates for the rate of decay from different experimental runs?

I have data for a device that dispenses material and I want to use an exponential decay model in python to relate the flow rate to the mass left inside the device, in particular $flow = \beta_0 + \...
2
votes
0answers
133 views

OOB error prediction in RF if case weights are used

I have a dataset for which grossing-up factors are given. I am using these factors as case weights for a random forest (R package ranger). Until now I was using the OOB prediction error for tuning, ...
2
votes
1answer
3k views

Using instance weights in XGBoost

I want to understand whether giving weights to instances across a dataset in XGBoost using the below method makes sense. I switched to this method after trying out a few approaches that didn't fare ...
4
votes
1answer
51 views

Should I weight points in a mixed model to account for groups having different numbers of points?

My research investigates the carriage of Salmonella by raccoons captured on multiple occasions. I am interested in modelling the relationship between sampling interval (number of days between two ...
2
votes
1answer
3k views

Weighted Categorical Variable

Consider for example I have a retrospective data that contains one categorical variable Race and one other variable Weight <...
0
votes
1answer
514 views

Why does my N change when weighting cases in SPSS?

I'm stuck here while analyzing data in SPSS and I could really use some help. I'm weighting cases in my dataset, but when I run any analysis my N has changed from 189 to 176. Is this normal with ...
0
votes
1answer
1k views

On feature scaling and weighting for clustering

The issue of feature scaling and weighting for cluster formation has been widely discussed in several books and papers as well as several questions (e.g. here ). To my understanting, variable range is ...
2
votes
0answers
109 views

Combining multiple observation weights for classification

Let's say you have multiple sources of observation weights for a dataset. For example, you have a $[0,1]$ weight coming from the label's certainty ($w_c$) and another one coming from its recency ($w_t$...