Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [weighted-sampling]

If you have survey data with weights, please use "survey-sampling" instead. If you need to draw Monte Carlo samples from a distribution that is intractable/inconvenient, and have to use a sampler from a simpler distribution that you would then correct with weights, please use "importance-sampling", "monte-carlo" and/or "simulation" instead.

0
votes
0answers
10 views

Can a ratio of performance be used to measure dependence and assign weights in a weighted average?

With only a modest grasp of inferential stats and maths in general, I've been wondering how one might derive the weights for a weighted average from a variable that isn't as straightforward as a ...
2
votes
0answers
23 views

How to compute confidence intervals from *weighted* samples?

Imagine we have a webserver, which serves a total of N static URLS. There are users visiting the URLs every day. At the end of each day, we have data like this: ...
1
vote
1answer
25 views

why sampling weights that I have range from 1, not 0?

I am looking at a dataset from Pew Research Center. Inside the dataset, different survey waves have their own weight variable with sampling weights. I thought in general it is supposed to range from 0 ...
0
votes
0answers
16 views

when and where need deep stratification?

as working in sampling i understood some methods like, stratification and post stratification, as post stratification is good when simple random sampling provides underestimate results then we adjust ...
0
votes
1answer
24 views

How to use derivatives of a function to better estimate its variance over the domain?

How to use derivatives of a function to better estimate its variance over the domain? I have a scalar smooth function $f(x)$ and a multivariate random variable $x$ with known distribution (e.g. ...
0
votes
0answers
24 views

Computing the Sample Size for the sum of Bernoulli RVs with different probabilites times a constant

I have the following statistic for which I need to figure out a sample size: $$S= \frac{1}{n}\sum_{i=1}^n \left(c_i+\sum_{j=1}^{100} b_{ij}X_i\right)$$ where $c_i$ and $b_{ij}$ are constants and $...
1
vote
0answers
84 views

Which is the right way to handle imbalanced data in a regression problem?

I'm working on a regression problem with imbalanced data, and I would like to know if I'm weighting the errors correctly. I'll try to illustrate the concept with a simple example. Imagine I'm ...
0
votes
1answer
13 views

Uncertainty-minimizing stratified sampling strategy

Suppose there is a school, and I want to know what proportion of students like the color red better than green, or vice versa (suppose there is no "other" option, just a binary variable). The school ...
0
votes
1answer
16 views

Resampling to get equal predictive power per observation

Cross posted from data science due to lack of response This is probably a thing I am just not searching for correctly, but essentially my idea is this: given some machine learning classification $C$ ...
1
vote
0answers
30 views

Sampling - optimal allocation for quantiles?

Say we have variable with a typical heavy tailed distribution following the Pareto principle. We divide the population into two strata (following the 80 / 20 rule) and use a stratified sampling design ...
0
votes
0answers
60 views

Stratified Bootstrapping of sub-populations

I want to know how to calculate the quartiles of two sub-populations, which are indistinguishable in each sampling area but have different distributions across the sampling areas and so different ...
0
votes
0answers
25 views

Aggregate Data from Unequal Sample Size

I need to aggregate land price data from several station areas for a later regression analysis. However, each station area has different sample size and total area. ...
1
vote
1answer
238 views

How to calculate importance weights for update step of an SIR (Sequential Importance Resampling) Particle filter?

I understand that one may use a particle filter to solve the filtering problem (estimating the hidden state of a system which can be described as a Hidden Markov Model). If I have a system where I ...
2
votes
0answers
140 views

Equivalence of svyglm and glm for simple random surveys

I have been exploring the use of the svyglm function in R's survey package to analyse surveys with both equal and unequal sampling probabilites. For an unequal ...
3
votes
1answer
146 views

Finding median without raw data?

I have only summary statistics for each state in the United States. I have the mean and median prices for each state and that’s it. How can I estimate an “overall” median price for the nation? I ...
2
votes
1answer
51 views

Application of Bayesian Averaging for Ranking

I have a sample with two metrics and one ratio per attribute. I am trying to rank the attributes based on the ratio and variable amounts and from my research I have found that most people find ...
1
vote
0answers
77 views

What is the Effect of Weighting observations when training a Classifier and how it can be combined with Subsampling?

My question is what is the effect of assigning weights to observations when training a Classifier such as a Logistic Regression model. The glm function documentation in R for example states: Non-...
0
votes
1answer
921 views

Equivalent to weighted random sample? [closed]

Let's say that you have a list of numbers and a weight for each number e.g. X = [(1, 2342), (2, 55), (3...] In the above example, 2342 and 55 are weights. Is weighted random sampling N items from ...
2
votes
0answers
133 views

Hypothesis testing on Weighted Poisson Binomial Distribution

Suppose I have $i$ coins, all of which are weighted to have a different probability $p$ of flipping heads. This results in $i$ Bernoulli distributions with different $p_i$. Cumulatively, this results ...
0
votes
1answer
58 views

Can someone point me towards research works relevant to Importance or Weighting Datapoints like SAW(Stepwise adaptation of weights) technique?

I am working on Fitness case importance for Symbolic Regression and found a Paper "Step-wise Adaptation of Weights for Symbolic Regression with Genetic Programming" which talks about weights of ...
0
votes
1answer
157 views

Stratified Random Sampling with 2 factors

Anyone in here knows how Proportion gets calculated?
1
vote
0answers
65 views

what is weight vector and bias in svm [duplicate]

I'm trying to understand the SVM algorithm but not able to understand what weight vector and bias is ? Could anyone explain it in laymen terms.
3
votes
1answer
1k views

SE of weighted mean

$X$ is a random variable with unknown distribution. A number of experiments are conducted to estimate $X$. Each experiment has a different reliability measure in estimating $X$. These $n$ experiments ...
1
vote
1answer
322 views

Survey sampling : normalised weights or not?

I work in epidemiology on a sample which is stratified, and 2 – level cluster (801 individuals). I use the "survey" package on R for data analysis. In the sample, the sum of weights is the size of ...
0
votes
1answer
395 views

How Do I Weight a Dataset to Match Needed Demographics in R?

If I have the attached file of 100 people with this dataset with the following demographics/regions: Region: Center: 12%, East: 62%, West: 26% Sex: Female: 82%, Male: 18% Party: D: 89%, R: 4%, O: 7% ...
0
votes
1answer
123 views

Question about the Distance Weighted Sample Covariance

I recently got interested in spatial statistics. I had tried to get a distance based covariance matrix by using distance weight function and weighted sample covariance through this reference. I ...
2
votes
1answer
138 views

Ranking with weighting amount of data

I'll try to ask this question in a form of a hypothetical: I have 10 different advertising spaces and I want to rank them according to their conversion rate (conversions/views). BUT, I have an uneven ...
3
votes
0answers
97 views

Appropriate sample size for weighted sample

I have a population that is sampled such that each item has a different probability of being selected. That probability is separate and independent of the value of any given item. How do I determine ...
1
vote
1answer
69 views

Test fitness proportionate selection algorithm implementation

I implemented a couple of algorithms for fitness proportionate selection (roulette-wheel, alias method and roulette-wheel via stochastic acceptance) and now I want to write a test to ensure that ...
4
votes
1answer
454 views

“representative sampling” from a distribution [closed]

I'm drawing samples from a distribution to train a machine learning classifier (training it via mini-batches of 32 samples at a time). It's just a toy dataset, so I know that the samples are coming ...
1
vote
1answer
32 views

Evaluating quality of a sampler on a small subset of the entire sample space

Assume a multi-dimensional discrete sample space $X$, which is "large", e.g. millions of possible objects. Function $f: X \rightarrow (0;1]$ that assigns a "reward" to each object $x \in X$ (the ...
1
vote
0answers
50 views

Biased coin randomization with adjustments over time

Assume an experiment that was going to consider different randomization strategies to address different challenges. The experiment involves an experimental (active treatment) and control group (usual ...
0
votes
1answer
78 views

Power Analysis Weighted Data

Suppose an outcome depends on the intensity of a treatment intervention $\pi$, where $\pi \in [0,1]$. Given intensity of treatment $\pi$, the data generating process is $$Y_i = \beta_0 + \beta_1 \pi +...
3
votes
0answers
289 views

For classification w unbalanced datasets, is class-weighing the same as oversampling?

in unbalanced classification problems, I find myself using class_weigh = "auto" or similar parameters often, but I don't think I'm fully understanding what it's doing. I know that it's the industry ...
0
votes
0answers
579 views

Ranking based on weight for multiple variables and objectives

First of all, I know very little about statistics. I need help ranking a set of multiple solutions (about 138 solutions) based on three objectives. I work on the architectural design field, I analyzed ...
3
votes
0answers
62 views

What is the Big O of rejection sampling from large sets of weighted items (like billions of records)?

On average, how many "rejections" will I get before I get an acceptance (for large sets using weights)? This answer suggested O(log n)?
3
votes
0answers
994 views

How to generate a specific number of random binaries with probablities proportional to given values?

I have such a matrix in an excel sheet. It has 140 cells. I want to generate 20 binaries randomly. However, the probability of generating "1" in each cell is proportional to cell's value. Namely, ...
0
votes
1answer
1k views

Probability for selecting centroids - K-means++

K-means++ selects centroids one by one, where each point has the chance to become next centroid with probability proportional to distance to closest centroid already selected. I implemented it like ...
4
votes
2answers
155 views

Sampling Methods for Review of Many Hours of Surveillance Video Footage?

I'm curious is anyone has experience with or can point me to some resources for sampling methodology for reviewing surveillance video footage. For example, say we have 6480 hours of video footage (3 ...
3
votes
1answer
2k views

Learning user behavior that changes over time

I am learning a model using SVM that will predict user behavior of some kind. Simplifying this model, each example in the feature space contains some features: $f_1,f_2,..,f_n$ and a class $a$ that ...
1
vote
1answer
206 views

Weighting particular cases in logistic regression

I am new to logistic regression and I am trying to determine whether it makes sense to weight a particular case in my data which is oversampled in order to model my data better. I'm not even sure if ...
4
votes
1answer
135 views

Expected number of replacements during weighted reservoir sampling?

Consider the problem of taking a weighted sample of size $K$ from a stream of unknown but finite size $N$ in a single pass. Reservoir sampling solves this by assigning each item from the stream with ...
2
votes
0answers
477 views

Distribution of weighted data

I want to fit a distribution to weighted data, and I have some issues, starting with the parameter estimation. How can I perform the MLE with the WEIGHTED data? The histogram of the weighted data fits ...
1
vote
0answers
2k views

Observation/case weighting in cluster analysis

Sampling weights, the inverse probability of a unit's selection into the sample, and other more complex and adjusted weights are very often used in the social sciences. There is statistical software ...
1
vote
1answer
19 views

Sample modification or weighting for partially paired data?

I want to analzye age and gender related differences of serum levels of an antibiotic agent, which was administered to 1200 patients during 1700 infectious episodes (1-12 infectious episodes and thus ...
1
vote
1answer
1k views

Can I get to an approximation of the population with knowledge of the expansion factor?

Say, I have a set of variables for different villages and for each village I have an expansion factor so I would get a sample representative of each village -- the expansion factor does not change ...
1
vote
1answer
53 views

Correcting multiple marginals in a subsample of a survey

My question in short: How to assign weights to a subsample of a survey in order to fit multiple features simultaneously to their original marginals? ...and now the details: I have some data set X of ...
3
votes
2answers
92 views

Probability selection of elements from a set

If we have a set $S= \{s_1,s_2,..,s_n\}$ and each element in the set $S$ has an assigned probability $P_n$. Then a selection process is applied to the set $S$ such that, each element $S_n \in$ $S$ is ...
0
votes
1answer
114 views

Algorithm for sampling points according to weights

In my current problem, I need to sample points in proportion to the weights assigned to them and an original probability density function. Unfortunately, the weights aren't known ahead of time and can'...
5
votes
1answer
2k views

Weighted cases in a cluster analysis for cases in SPSS

I am conducting a cluster analysis (of cases) for a database which has weight attributed to the individual cases to ensure that it mirrors the general population in terms of sociodemographic ...