Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [weighted-data]

Datasets where different pieces of data can have different "weights", i.e. different importance.

1
vote
1answer
15 views

Weighted Categorical Variable

Consider for example I have a retrospective data that contains one categorical variable Race and one other variable Weight <...
0
votes
0answers
9 views

Can a ratio of performance be used to measure dependence and assign weights in a weighted average?

With only a modest grasp of inferential stats and maths in general, I've been wondering how one might derive the weights for a weighted average from a variable that isn't as straightforward as a ...
0
votes
1answer
21 views

Why does my N change when weighting cases in SPSS?

I'm stuck here while analyzing data in SPSS and I could really use some help. I'm weighting cases in my dataset, but when I run any analysis my N has changed from 189 to 176. Is this normal with ...
0
votes
1answer
53 views

On feature scaling and weighting for clustering

The issue of feature scaling and weighting for cluster formation has been widely discussed in several books and papers as well as several questions (e.g. here ). To my understanting, variable range is ...
0
votes
0answers
13 views

How to assign class weights & also mis-classification cost in multi class classification of Vowpal Wabit?

I have a problem of Ordinal Classification consisting of labels 1 < 2 < 3 < 4 < 5 < 6. Since the data and cardinality of features are extremely large I have but one choice to use Vowpal ...
2
votes
0answers
36 views

Combining multiple observation weights for classification

Let's say you have multiple sources of observation weights for a dataset. For example, you have a $[0,1]$ weight coming from the label's certainty ($w_c$) and another one coming from its recency ($w_t$...
0
votes
1answer
30 views

R Calculate sample weights based on two socio-demographic variables and aggregate results

After conducting a short survey, I have collected the results in the form of a dataframe. This is a reproducible version of what the actual data frame looks like. ...
1
vote
0answers
36 views

Weighted covariance with different reliability weights?

Suppose observables $X$ and $Y$ possess different reliability weights $w_{x,i}$ and $w_{y,i}$ for the possible elements $x_i\in X$ and $y_i\in Y$ respectively. Considering a sequence of consecutive ...
1
vote
1answer
40 views

Duplicated Rows in Mixed Data Type Clustering

I have a dataset which has ~200k rows and looks like the following - ...
0
votes
1answer
12 views

how to estimate prognostic core for a continuous, multinomial, and binary treatment, respectively

Quoting David Hajage and Finbarr Leacy, respectively: "Introduced by Hansen in 2008, the prognostic score (PGS) has been presented as ‘the prognostic analogue of the propensity score’ (PPS). ...
0
votes
1answer
26 views

Comparison of means from different sample sizes

I have a data set which contains avg profit and the number of samples for each data point, no more information. I'd like to compare the data points and decide which one to focus on more. However, I'm ...
0
votes
0answers
36 views

Do I replicate observations to weight by population? For regression

Note: This question may well be a duplicate, but I don't know the language to use to find it. I have searched the site and Google but haven't seen anything that I am looking for – maybe dasymetric ...
1
vote
0answers
28 views

Stochastic gradient descent (SGD) on data with weights

Mostly deep learning model training is on data with a unit weight. In this case, every mini-batch of a fixed size, say, 32, contains exactly the same total weight (32) for each update. This is the ...
0
votes
0answers
40 views

How to test for difference in weighted proportion difference? (and average weighted proportion differences)

Suppose I have a survey where respondents rate different questions as positive (1), negative (-1), or neutral (0), and their responses are given a different weight based on which job they have and ...
1
vote
0answers
47 views

Logistic Regression: Using weights?

I am running a logistic regression model with the following data and variables: Independent variables (characteristics of a person coded as dummy) Test score (continuous variable between 0 and 1) ...
0
votes
0answers
27 views

Weights of ratio of weighted data

Suppose I have the two datasets $x_i$ and $y_i$ with weights $w_{x,i}$ and $w_{y,i}$, respectively ($1 \le i \le N$). The weighted means are for example: $$\bar x = \frac{\sum_i x_i w_{x,i}}{\sum_i ...
0
votes
2answers
804 views

How to set class weights for multi-class image segmentation?

I am trying to set class weights for a neural network with an imbalanced dataset. Let's say I have the following values: I have 8000 images of class A, 1100 images of class B, 400 images of class C, ...
0
votes
0answers
77 views

How to determine the weights for weighted CrossEntropyLoss?

I want to do image segmentation with 2 classes(everything else and the object of interest). Therefore I got a FCN8 net structure implemented. In the first try the net learned to make the whole image ...
1
vote
1answer
244 views

what is instance weight in boosting?

Hi, I am reading something about Boosting, and I had hard time understanding one of the steps in boosting - assign greater weights to those instances. What does the sentence - assign greater weights ...
1
vote
0answers
37 views

Using weighted or unweighted wPCA scores in Logistic regression model-SAS

Im working on data that I should weight the data using a weight variable that I have and I want to evaluate the association of dietary pattern and obesity. So first I run Proc factor and use the "...
0
votes
1answer
17 views

How to calculate the weighted value to give numbers? [closed]

Big picture question: How do I calculate the weight to give certain categories? Specifics: I have data for 70 counties, population in each county, and then number of sales in each county. How do I ...
2
votes
0answers
18 views

What is the effect of more than one chance of selection when sampling?

I am designing a clustered sample. Within each cluster there are 3 groups that will be sampled from - two at a higher rate than the other. The two subgroups that will be sampled at a higher rate are ...
2
votes
0answers
49 views

Dealing with Numerical Issues from Computing Weighted Sample Covariance Matrix

I have vector-valued samples $\mathbf{x}_1, \mathbf{x}_2, \ldots, \mathbf{x}_N$ with normalized weights $w_1, \ldots, w_N$. I'm trying to compute a weighted sample covariance matrix: $$ \sum_i w_i (\...
0
votes
2answers
67 views

Learning weights for variables based on cluster assignments

I need an approach to assign weights to variables based on the cluster. I am not trying to learn the clusters, as I already have them, but I would just like to now understand how "important" each ...
0
votes
0answers
97 views

How to calculate the standard deviation to be used for a confidence interval on a weighted data set?

Although there were some similar questions on the board, none seemed to quite be the same as what I'm asking. I have a set of survey values (satisfaction ratings) that have been weighted for non-...
1
vote
0answers
302 views

Test difference of population proportion and weighted sample proportion

I want to test for differences of the known proportion of the population vs. the proportion of a sample. The sample is weighted by sampling weights. The sum of weights is equal to the size of the ...
1
vote
0answers
379 views

Calculating a confidence interval for a weighted sample

In a nutshell, I'd like to compute a confidence interval for some weighted sample day where the final value I'm seeking is a sum of different weighted samples. Illustrated here with a toy example set ...
1
vote
0answers
79 views

Does there exist a variant of ARIMA allowing for weighted samples?

I have a univariate time series exhibiting strong periodicity that I want to forecast, and I plan to use ARIMA. However due to specifics of the prediction task that I'm interested in performing, some ...
0
votes
0answers
61 views

relative frequency

I have a number of data such as the following. Each represents a trial where two options (V1 and V2) were given to a participant ("judge") and they were asked to pick one over the other. V1 and V2 can ...
2
votes
1answer
51 views

Application of Bayesian Averaging for Ranking

I have a sample with two metrics and one ratio per attribute. I am trying to rank the attributes based on the ratio and variable amounts and from my research I have found that most people find ...
3
votes
2answers
1k views

Observations weight-age in a Machine Learning model

I want to know is there any way in R/Python to specify to the model to emphasize its learning more on specific subset of data , while it considers the whole data. For example - i have sales behavior ...
0
votes
1answer
35 views

Weighting prior proposals based on distance function in approximate Bayesian computation

The typical approach in approximate Bayesian computation (ABC) is to propose parameters from the prior, simulate data $\chi'_\text{sim}$ and then accept data that minimises the data misfit $\lambda$ ...
1
vote
0answers
56 views

How does one choose which factors to weight on?

How does one choose which factors to weight on? The context is the following: I am running a satisfaction survey. The key outcome metric is satisfaction. The survey has moderate to significant under-...
2
votes
0answers
18 views

Community Detection for Complete Weighted Network [duplicate]

I am wondering if someone who has experience in trying to perform community detection algorithm with complete graph with weighted network. As weighted networks represents the similarity of each node ...
0
votes
0answers
82 views

estimate Gamma from weighted data

I have weighted data (Dist = "Distance") that I expect to be in a Gamma distribution. There are about 28,000 observations that expand to a population of about 730,...
2
votes
1answer
317 views

Shannon's entropy negative indicator

I have a composite indicators whose indicators I want to weight, for which I am calculating the Shannon's entropy. Problem is some indicators measure growth and thus can be negative. Standardizing ...
1
vote
0answers
111 views

One sample proportion test for weighted data

I have a sample of observations (n) representing individual evaluations (positive/negative) of pieces of text (n texts). Each individual can evaluate more texts (thus there are m individuals with m &...
0
votes
1answer
67 views

Computing growth rates with drastically changing sample sizes

I'm computing growth rate for some companies annually for 3 years using this formula, for each year: ((# customers year 2 - # customers year 1) / # customers year 1) * 100 The problem I'm running ...
0
votes
1answer
55 views

Weighting hospital visit (independent variable)

I am interested in profiling patients that has high number of hospital visits in a year. However, I noticed that not all hospital visits are equal. That is, an inpatient visit is more expensive than ...
5
votes
1answer
3k views

What does this sampling weight mean?

The data comes from agricultural market research on farming. The sample was derived based on stratification of farming industries (sheep, beef, grains, etc.) and random sampling within each stratum. ...
1
vote
0answers
264 views

Should we adjusted the weight according to sample size in propensity score with IPTW method?

I'm a new learner in biostatistics, recently doing a project that might need to use IPTW method for balancing covariates between treatment groups. When I read one of the sas references [Ref 1] for ...
5
votes
1answer
890 views

How to compute a weighted AUC?

Say that I have a set of training examples with binary 0-1 labels, and corresponding regression labels ranging from 0 to 1. Normally, to compute AUC I've used the ROCR package in R, which also allows ...
2
votes
1answer
6k views

Scikit-Learn - Adding Weights to Features

I'm pretty new to machine learning and I have a question regarding weighting features. I was able to get code running with scikit-learn and random forest. Now I would like to weight certain features ...
1
vote
0answers
32 views

Distribution of Sums Confidence Interval

How do I find a confidence interval for a series of values that are log-normally distributed weighted by their values? The distribution would look like: $\sum_{k=1}^n 2^{mU_k}$ Where U is a random ...
0
votes
0answers
283 views

Why isn't it possible to use fractional weights for Kruskal-Wallis test?

When I try to run a Kruskal-Wallis test on a sample with fractional weights, SPSS throws the following warning: ...
0
votes
1answer
63 views

Weighing groups when there is a significant difference in sample sizes

Let's say I am trying to compare average scores between three different groups. Group A contains 20 cases, group B contains 40 cases, and group C contains 300 cases. What I originally wanted to do is ...
1
vote
0answers
37 views

Distance metric between 2 integers and between 2 sets of integers with weights

I have several positive integer numbers representing the count of features, for example delay, successful_attempts, ...
0
votes
1answer
58 views

Can someone point me towards research works relevant to Importance or Weighting Datapoints like SAW(Stepwise adaptation of weights) technique?

I am working on Fitness case importance for Symbolic Regression and found a Paper "Step-wise Adaptation of Weights for Symbolic Regression with Genetic Programming" which talks about weights of ...
4
votes
1answer
2k views

How is a Poisson rate regression equal to a Poisson regression with corresponding offset term?

I do not understand the role of weights in "weighted Poisson regression". What exactly is being weighted? Is it the contribution of the observation to the log-likelihood of the model, or something ...
1
vote
0answers
88 views

Weighting vs. regression to take into account different auxiliary characteristics of groups

I'm interested in comparing different hospitals by their performance in a number of metrics (for example waiting times). Unfortunately the patient characteristics vary between hospitals (hospitals in ...