Questions tagged [weighted-data]

Datasets where different pieces of data can have different "weights", i.e. different importance.

Filter by
Sorted by
Tagged with
0
votes
0answers
8 views

Calculating a Weighted Standard Error of the Fit for Nonlinear Regression

I have a data set of $N$ points to which I have fit an equation of $n$ parameters $\theta_{1..n}$ such that $y_i \sim f(x_i; \theta_{1..n})$. These data $(x_{1..N},y_{1..N})$ have been provided with ...
1
vote
0answers
14 views

Are probability weights “relative”, i.e. can they be rescaled and work in the same way?

I am using survey data from different countries, and I pool them together for part of the analysis. Now, weights puzzle me for two reasons: 1) some of them are below 0 (i.e. the probability of being ...
0
votes
1answer
31 views

Bayesian Statistics - How to weight a Poisson distributed response

I have a Bayesian GLM where the response that I'm interested in is count data. I want to weight the the response by the variance to account for uncertainty in the measurement. If the response was ...
0
votes
0answers
11 views

Can I adjust sample_weight in Scikit-learn for each feature in each sample separately?

My understanding is that when using sample_weight when training a classification model in Scikit-learn, I can supply different weights for each sample. Is it also possible to give a different weight ...
0
votes
0answers
11 views

How do we actually give weights to the decision trees in Weighted Random Forest

I have built a Random Forest algorithm in C# myself using alot of documentation. Now I have come accross the below: Refined weighted Random Forest (assigning different weights to different decision ...
0
votes
1answer
27 views

How to calculate the confidence interval with weighted data?

I've done some search for similar questions, but they're not the same as what I'm trying to get. Assume that there's a server that handles requests $r$ and returns a set of items $I_{r}$ of random ...
0
votes
0answers
10 views

What is the best way to visualize IE matrix data?

I have this table and I wondering which is the best way to visualize these data? Using the R script below I have managed to produce a simple scatter plot to visualize the ...
0
votes
0answers
7 views

Comparing 2 Sets of Changed Weighted data

Assume the following simplified data set. I have 5 products of % share of each (Weight). With a value of minutes for each. Take product 2: Original - 20% share at 20 Minutes After 1 - Now has 10% ...
0
votes
0answers
10 views

Help with weighting sample according to population

I am a beginner with basic knowledge of statistics - just learning. I have a doubt regarding weighting survey sample distribution to population distribution. I have to create a weighting variable that ...
0
votes
1answer
11 views

How important are the survey weights?

I run logit model with a cross-sectional dataset of Indian individuals. I am using descriptive statistics of the same dataset to justify and interpret the estimations of logit model. However, I must ...
1
vote
0answers
20 views

sklearn f1_score=weighted not matching sample_weight specification

I am trying to figure out exactly what this is doing: sklearn.metrics.f1_score(y_pred, y_test, sample_weight=[...]) Numerically it simply does not seem to be ...
0
votes
0answers
16 views

Weightings using exponents

I've got this piece of work to do where we assign weightings to different variable to achieve a score: var A = 60% var B = 40% var C = 20% var D = 5% The score was calculated as: $(A^{0.6} x B^{...
0
votes
0answers
18 views

How to add weights to some values in my dataset? [duplicate]

I am creating a model where I predict runs scored by a team against another team. Name of both teams are given but runs scored is also dependent on the year factor. Suppose Team A was really good ...
0
votes
0answers
24 views

Stochastic/batch gradient descent (type SGD/ADAM) with weighted mean square error loss

Assume I assign uneven weights to losses of different examples, i.e. I set my SGD/ADAM to train a universal approximator f (e.g. a neural net) by minimization of a weighted mean square error: $ L = \...
0
votes
0answers
10 views

Compensating for bias in a location-based weighted average

I have a data set where each point comes from a specific geographic location. I want my users to see a weighted average of the data, so that the average is biased toward their location. The problem ...
1
vote
0answers
21 views

How can I aggregate the estimates for the rate of decay from different experimental runs?

I have data for a device that dispenses material and I want to use an exponential decay model in python to relate the flow rate to the mass left inside the device, in particular $flow = \beta_0 + \...
0
votes
0answers
43 views

Weigh Kullback Leibler Divergence with P entropy?

I'm wondering if it makes sense to weight Kullback-Leibler Divergences to highlight divergences on highly distinguishing features. I am however not very well versed in Information Theory, and would ...
0
votes
0answers
18 views

How to combine PCA scores? [duplicate]

I was wondering if anyone knew how to combine the different PCA scores. In my dataset I have 3 PCs that explain more than 10% of the variation of shape of a bone and each one explains different ...
2
votes
0answers
76 views

OOB error prediction in RF if case weights are used

I have a dataset for which grossing-up factors are given. I am using these factors as case weights for a random forest (R package ranger). Until now I was using the OOB prediction error for tuning, ...
0
votes
0answers
30 views

calculate the average visits and running days

What do I've got? I have received a huge data from a real estate portal. It contains more than 50 million rows and data for over 10 years. In addition to the rental prices and other parameters (...
1
vote
1answer
629 views

Using instance weights in XGBoost

I want to understand whether giving weights to instances across a dataset in XGBoost using the below method makes sense. I switched to this method after trying out a few approaches that didn't fare ...
4
votes
1answer
45 views

Should I weight points in a mixed model to account for groups having different numbers of points?

My research investigates the carriage of Salmonella by raccoons captured on multiple occasions. I am interested in modelling the relationship between sampling interval (number of days between two ...
2
votes
1answer
914 views

Weighted Categorical Variable

Consider for example I have a retrospective data that contains one categorical variable Race and one other variable Weight <...
0
votes
1answer
90 views

Why does my N change when weighting cases in SPSS?

I'm stuck here while analyzing data in SPSS and I could really use some help. I'm weighting cases in my dataset, but when I run any analysis my N has changed from 189 to 176. Is this normal with ...
0
votes
1answer
405 views

On feature scaling and weighting for clustering

The issue of feature scaling and weighting for cluster formation has been widely discussed in several books and papers as well as several questions (e.g. here ). To my understanting, variable range is ...
2
votes
0answers
57 views

Combining multiple observation weights for classification

Let's say you have multiple sources of observation weights for a dataset. For example, you have a $[0,1]$ weight coming from the label's certainty ($w_c$) and another one coming from its recency ($w_t$...
0
votes
1answer
104 views

R Calculate sample weights based on two socio-demographic variables and aggregate results

After conducting a short survey, I have collected the results in the form of a dataframe. This is a reproducible version of what the actual data frame looks like. ...
2
votes
0answers
68 views

Weighted covariance with different reliability weights?

Suppose observables $X$ and $Y$ possess different reliability weights $w_{x,i}$ and $w_{y,i}$ for the possible elements $x_i\in X$ and $y_i\in Y$ respectively. Considering a sequence of consecutive ...
1
vote
1answer
118 views

Duplicated Rows in Mixed Data Type Clustering

I have a dataset which has ~200k rows and looks like the following - ...
0
votes
1answer
14 views

how to estimate prognostic core for a continuous, multinomial, and binary treatment, respectively

Quoting David Hajage and Finbarr Leacy, respectively: "Introduced by Hansen in 2008, the prognostic score (PGS) has been presented as ‘the prognostic analogue of the propensity score’ (PPS). ...
0
votes
1answer
35 views

Comparison of means from different sample sizes

I have a data set which contains avg profit and the number of samples for each data point, no more information. I'd like to compare the data points and decide which one to focus on more. However, I'm ...
0
votes
0answers
67 views

Do I replicate observations to weight by population? For regression

Note: This question may well be a duplicate, but I don't know the language to use to find it. I have searched the site and Google but haven't seen anything that I am looking for – maybe dasymetric ...
2
votes
0answers
70 views

Stochastic gradient descent (SGD) on data with weights

Mostly deep learning model training is on data with a unit weight. In this case, every mini-batch of a fixed size, say, 32, contains exactly the same total weight (32) for each update. This is the ...
1
vote
0answers
146 views

Logistic Regression: Using weights?

I am running a logistic regression model with the following data and variables: Independent variables (characteristics of a person coded as dummy) Test score (continuous variable between 0 and 1) ...
1
vote
2answers
2k views

How to set class weights for multi-class image segmentation?

I am trying to set class weights for a neural network with an imbalanced dataset. Let's say I have the following values: I have 8000 images of class A, 1100 images of class B, 400 images of class C, ...
1
vote
1answer
583 views

what is instance weight in boosting?

Hi, I am reading something about Boosting, and I had hard time understanding one of the steps in boosting - assign greater weights to those instances. What does the sentence - assign greater weights ...
1
vote
0answers
81 views

Using weighted or unweighted wPCA scores in Logistic regression model-SAS

Im working on data that I should weight the data using a weight variable that I have and I want to evaluate the association of dietary pattern and obesity. So first I run Proc factor and use the "...
0
votes
1answer
49 views

How to calculate the weighted value to give numbers? [closed]

Big picture question: How do I calculate the weight to give certain categories? Specifics: I have data for 70 counties, population in each county, and then number of sales in each county. How do I ...
2
votes
0answers
18 views

What is the effect of more than one chance of selection when sampling?

I am designing a clustered sample. Within each cluster there are 3 groups that will be sampled from - two at a higher rate than the other. The two subgroups that will be sampled at a higher rate are ...
2
votes
0answers
61 views

Dealing with Numerical Issues from Computing Weighted Sample Covariance Matrix

I have vector-valued samples $\mathbf{x}_1, \mathbf{x}_2, \ldots, \mathbf{x}_N$ with normalized weights $w_1, \ldots, w_N$. I'm trying to compute a weighted sample covariance matrix: $$ \sum_i w_i (\...
0
votes
2answers
117 views

Learning weights for variables based on cluster assignments

I need an approach to assign weights to variables based on the cluster. I am not trying to learn the clusters, as I already have them, but I would just like to now understand how "important" each ...
0
votes
0answers
199 views

How to calculate the standard deviation to be used for a confidence interval on a weighted data set?

Although there were some similar questions on the board, none seemed to quite be the same as what I'm asking. I have a set of survey values (satisfaction ratings) that have been weighted for non-...
1
vote
0answers
427 views

Test difference of population proportion and weighted sample proportion

I want to test for differences of the known proportion of the population vs. the proportion of a sample. The sample is weighted by sampling weights. The sum of weights is equal to the size of the ...
4
votes
0answers
1k views

Calculating a confidence interval for a weighted sample

In a nutshell, I'd like to compute a confidence interval for some weighted sample day where the final value I'm seeking is a sum of different weighted samples. Illustrated here with a toy example set ...
2
votes
0answers
255 views

Does there exist a variant of ARIMA allowing for weighted samples?

I have a univariate time series exhibiting strong periodicity that I want to forecast, and I plan to use ARIMA. However due to specifics of the prediction task that I'm interested in performing, some ...
0
votes
0answers
90 views

relative frequency

I have a number of data such as the following. Each represents a trial where two options (V1 and V2) were given to a participant ("judge") and they were asked to pick one over the other. V1 and V2 can ...
2
votes
1answer
62 views

Application of Bayesian Averaging for Ranking

I have a sample with two metrics and one ratio per attribute. I am trying to rank the attributes based on the ratio and variable amounts and from my research I have found that most people find ...
4
votes
2answers
2k views

Observations weight-age in a Machine Learning model

I want to know is there any way in R/Python to specify to the model to emphasize its learning more on specific subset of data , while it considers the whole data. For example - i have sales behavior ...
0
votes
1answer
45 views

Weighting prior proposals based on distance function in approximate Bayesian computation

The typical approach in approximate Bayesian computation (ABC) is to propose parameters from the prior, simulate data $\chi'_\text{sim}$ and then accept data that minimises the data misfit $\lambda$ ...
1
vote
0answers
78 views

How does one choose which factors to weight on?

How does one choose which factors to weight on? The context is the following: I am running a satisfaction survey. The key outcome metric is satisfaction. The survey has moderate to significant under-...