Questions tagged [weighted-data]

Datasets where different pieces of data can have different "weights", i.e. different importance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
36 views

Estimate weights given a weighted geometric median

Problem 1 Given a set of points and their weighted geometric median, find the weights associated with each point. For instance, given the points ...
0 votes
0 answers
52 views

Combining standard errors of two separate variables

I have a dependent variable associated with a SE and an independent variable associated with a SE. I would like to weight my regression such that both of these SEs are taken into account. What is the ...
  • 41
0 votes
0 answers
16 views

Outputting the weighted version of a variable

Is it possible to apply weights to a variable (eg, the independent variable in a regression analysis) and then output the weighted version of this variable? I would like to weight both my dependent ...
  • 41
2 votes
1 answer
35 views

Conducting multiple weighted t.tests to count over what fraction of a population a model performs significantly better or worse than another

I have two different model families $A_i$ and $B_i$ each of which predicts a continuous value for an item $i$ at various points in time throughout a given day. For a given day, and item $i$, I can ...
0 votes
0 answers
33 views

How to get weighted mean with two weighting sources

I have some data on a cyclists distance traveled and hours it took. ...
  • 23
2 votes
0 answers
7k views

Calculate probability of dying in school shooting using spatial statistics

Some public commentators say that the chance of dying in a school shooting is very low, so they conclude that Americans over-react to them. https://www.washingtonpost.com/outlook/school-shootings-are-...
0 votes
0 answers
25 views

Smoothing time series with Adjusted R2-weighted averages

I have two parameters (a,b) resulting from an exponential estimation of a curve. I have estimated this curve every hour for one month. In other words, I have a total of 720 parameters a and b, and I ...
1 vote
1 answer
25 views

weighted median result is integer

I am computing the weighted median for a dataset using the R reldist package. The input data are integers but my result is an integer number too. Should't my result ...
  • 284
0 votes
0 answers
54 views

How to bias-correct proportions for a prop.test in R?

Let's say I ask 200 participants in a study what experience brought the most joy to their lives: A, B, C, or D. ...
1 vote
0 answers
27 views

How reliable is conclusion drawn out from a known data?

Most of time conducting experiment is expensive. Suppose I and my collaborator decided to use a known databank(NHANE, NCI's data) to do some digging with some untested hypothesis given. Most of time ...
  • 1,295
0 votes
0 answers
51 views

How to assign weights to a set of ordered objects?

I have been trying to assign weights to set of objects. The problem that I have is as following. I have a set of objects ordered either ascending or decending. . Each object shall be provided a weight ...
  • 103
0 votes
0 answers
16 views

Optimizing weights of elements based on their correlation to the same metric

I'm in my freshman year in college and working on a data science project the goal of which is to measure a country's prosperity. We've grouped similar indicators or elements into one pillar. We have 5 ...
  • 1
0 votes
0 answers
50 views

Weighting the calculation of sample quantiles

Is there a general method for determining what the weights are when calculating sample quantiles? Specifically, how do you do this in the case of repeat measurements -- can (should) you take the ...
0 votes
0 answers
33 views

How can I "weight" an average to favor strong contributions from each addend?

tl;dr: I'm assigning "scores" to groups. Groups contain objects with values. I want (50 + 50 / 2) to be way higher than (0 + 100/2) because for my purposes, low scores from one object in a ...
  • 1
2 votes
1 answer
87 views

Can I use oversampling to increase sample weights?

I am working on a balanced binary classification problem. I don't need to adjust the proportion of either of the classes, as it's already 50/50 . However, some of the samples are more valuable than ...
0 votes
1 answer
29 views

Age adjustment or Age standardization state of the art?

I want to run some comparisons of a quantity between different populations. Such quantity comes from a model that includes, among other variables, the variable age. So, one of the variables affecting ...
2 votes
0 answers
55 views

How do I calculate a finite-population correction for a weighted sample?

I have a weighted sample from a population. These are probability weights (as samples are taken with unequal probability). What's the variance in the estimated total? All samples are exchangeable (...
0 votes
0 answers
28 views

How to weight my data in SAS - categorical and numerical variables

I've collected data from 1000 animals (categorical and numerical variables) from 2 different farms. My problems are: one farm corresponds to 68% of my data, in some of my variables I've observed more ...
0 votes
0 answers
19 views

Domain weights as hyperparameters

Suppose I have data from two different sources, $(X_1, y_1)$ and $(X_2, y_2)$, where $X_1$ and $X_2$ have the same variables but different distributions, and similarly for $y_1$ and $y_2$. I want to ...
  • 866
0 votes
0 answers
14 views

Proper statistical test for NGS dataset with two samples containing a library of 12,000 sequences each sorted into 4 bins with counts in each bin

Description of the experiment: A library containing 12,000 unique nucleotide sequences is integrated into two samples, a WT cell line and a knockout cell line. Each cell contains only one reporter ...
  • 1
0 votes
0 answers
28 views

Weigh the percentages based on the number of inhabitants

I am analyzing the diffusion of a service in some regions. Is there a legitimate criterion for weighing percentages by the number of inhabitants? I would like to "improve" the results of the ...
  • 131
1 vote
0 answers
246 views

How do we calculate statistics on data weighted by relative frequencies?

I have the following data with some values and their corresponding weights. groups values weights a 1 0.25 a 3 0.75 b 8 1.00 The weights represent relative frequencies of each value in the group ...
  • 113
0 votes
0 answers
22 views

weight variable based on opposite direction variable

I have two variables. From these two variables, I want to produce one weight variable. The first (A) is a percentage, 0.0 - 1.0. The second variable (B) is a count, 0 to infinity. I want to weight the ...
  • 235
0 votes
1 answer
75 views

Q: Aggregating audience and critic movie ratings, and creating a composite rating of the two

I’m building a movie review aggregation site that combines user ratings and critical reviews for a given movie. The objective is to create a list of the “best of the best” movies which were rated ...
1 vote
0 answers
81 views

Variance in variance-weighted variance estimate?

Apologies for the confusing title, but I couldn't resist. Much can and has been said about computing the unbiased variance using a sample of points, weighting by the variances of each point (for ...
3 votes
2 answers
1k views

What does the "weights" argument do when fitting Kaplan-Meier curveswith the survival package?

In R's survival package, there is an optional weights argument you can supply when you fit a Kaplan-Meier curve. I can't find any documentation about what this does or what exactly a weighted Kaplan ...
  • 2,414
6 votes
1 answer
758 views

Best way to construct a QQ-plot

I want to assess the normality of a dataset (which is log-normally distributed data transformed back to normal) using a Q-Q plot. I stumbled on the fact that there are many ways to build such a plot, ...
  • 183
0 votes
0 answers
44 views

Weighting the loss function based on previous seen true positive rates

Similiar to class imbalance there is always something I would call "learnability imbalance" in multi-class classification. What I mean by that: Even when the classes are evenly distributed ...
  • 59
0 votes
1 answer
136 views

Calculating Weighted Quantiles causing NA error by coercion in R

I'm new to R and I'm trying through two different packages and formulas to calculate quantiles from a weighted dataset, as you can see below: ...
15 votes
2 answers
2k views

Can I (justifiably) train a second model only on the observations that a previous model predicted poorly?

Say I commit the following sins while building a predictive model: I take my dataset and split it into four subsets: Three for training (Train_A, Train_B, and Train_C) and one for validation. I ...
  • 155
0 votes
1 answer
136 views

NHANES Weighting

I am in the process of analyzing some data derived from 8 cycles of NHANES data. Due to the nature of my research, my sample size is rather small (n=98). Since NHANES surveys utilize a complex ...
  • 21
1 vote
0 answers
157 views

How can I weight ordinal observations and reduce them to one statistic? [closed]

It will be a complicated question and I try to briefly explain. I am studying on educational inequalities. The survey I use for analysis incudes ordinal variables and the education degree which ...
1 vote
0 answers
234 views

Are there time series models for weighted data?

Let's say I have the following time series: Weekly data spanning a few years; The dependent variable is the proportion of items that pass x. On average, that proportion is around 80%. The number of ...
1 vote
0 answers
428 views

How to compute weights using logistic regression

I'm not interested in creating a logistic regression model which is weighted, I'm interested in using a logistic regression model to compute the weights of a survey. It seems that, given the weights ...
  • 888
0 votes
0 answers
117 views

How to properly calculate weighted average when the data is in form of ratio

I had a question related to calculating weighted average, on data example below given definition : Column C = Product Supply Column D = Product Demand Column E = Supply / Demand Column F = Demand ...
  • 1
4 votes
0 answers
1k views

Meaning of the weight argument in glmer and lmer

I have been looking into how to use the weight argument of glmer/lmer to represent "frequency" weights. I was ...
  • 2,893
0 votes
1 answer
512 views

EWMA covariance matrix number of lags

When calculating an exponentially weighted covariance matrix for t observations, formula 10.2 here: https://www.oreilly.com/library/view/analysis-of-financial/9781118017098/c10_level1_1.xhtml Uses ...
1 vote
0 answers
33 views

Assign more importance to recent records during training

I have a large dataset with information from the last year. I have to build a classification model in order to predict if a customer will buy a product or not (binary classification). Since in the ...
  • 193
1 vote
0 answers
39 views

Equation for weighted mean of percentages [closed]

I am trying to get the equation for a weighted average of percentages to be normalized between 0 and 1. Am I doing it correctly? Thank you so much!
  • 31
2 votes
0 answers
935 views

Equation for weighted average with normalization(?)

I am trying to understand how I should denote the equation for rescaling the following 7 categories into one value. The final weighted average is 0.57, which is possible by subtracting one and ...
  • 31
0 votes
0 answers
179 views

Euclidean distance from zero

I am trying to create my own weights for relative work task importance, or weight. For every task, I have a value of importance, ...
  • 31
1 vote
1 answer
461 views

Winsorizing propensity scores

Is it kosher? Inverse propensity weights (IPW) has been shown to perform poorly when selection probabilities are small (Kang and Schafer, 2007). Are there any standard solutions to this issue?
1 vote
1 answer
317 views

How to use weight variables within aggregated results, for group proportions [closed]

If I have the following data, where each row represents and individual response, how would I go about reporting the proportions of different groups? (here the groups are ...
  • 888
1 vote
0 answers
32 views

Are probability weights "relative", i.e. can they be rescaled and work in the same way?

I am using survey data from different countries, and I pool them together for part of the analysis. Now, weights puzzle me for two reasons: 1) some of them are below 0 (i.e. the probability of being ...
  • 11
0 votes
1 answer
361 views

Bayesian Statistics - How to weight a Poisson distributed response

I have a Bayesian GLM where the response that I'm interested in is count data. I want to weight the the response by the variance to account for uncertainty in the measurement. If the response was ...
  • 11
1 vote
1 answer
2k views

How to calculate the confidence interval with weighted data?

I've done some search for similar questions, but they're not the same as what I'm trying to get. Assume that there's a server that handles requests $r$ and returns a set of items $I_{r}$ of random ...
  • 11
0 votes
0 answers
128 views

Help with weighting sample according to population

I am a beginner with basic knowledge of statistics - just learning. I have a doubt regarding weighting survey sample distribution to population distribution. I have to create a weighting variable that ...
  • 1
0 votes
1 answer
112 views

How important are the survey weights?

I run logit model with a cross-sectional dataset of Indian individuals. I am using descriptive statistics of the same dataset to justify and interpret the estimations of logit model. However, I must ...
1 vote
0 answers
208 views

sklearn f1_score=weighted not matching sample_weight specification

I am trying to figure out exactly what this is doing: sklearn.metrics.f1_score(y_pred, y_test, sample_weight=[...]) Numerically it simply does not seem to be ...
1 vote
0 answers
30 views

How can I aggregate the estimates for the rate of decay from different experimental runs?

I have data for a device that dispenses material and I want to use an exponential decay model in python to relate the flow rate to the mass left inside the device, in particular $flow = \beta_0 + \...
  • 33