22
votes
Surveys: Is 25% of a large user base representative?
Think about surveys in the general population of say the US. If we need 50% of the population to determine the majority opinion we would need a sample of about 160 million, which is truly prohibitive. ...
- 21.5k
16
votes
When sampling a population for surveys we can often limit our sample size to hundreds, but when doing a Monte Carlo simulation we need way more. Why?
Surveys are relatively expensive, which determines a different balance between the cost and value. One instance of a Monte Carlo simulation is extremely cheap so you can repeat it more until the ...
- 1,057
12
votes
Accepted
Why it is important to make survey design object (svydesign function in R with id, strata, weights, fpc) from raw data and after clean data in object?
There are two separate issues here.
Sometimes, including with NHANES data, you do need to subset before defining the survey design object, because not all the records in the data set are part of the ...
- 46.8k
12
votes
Accepted
When sampling a population for surveys we can often limit our sample size to hundreds, but when doing a Monte Carlo simulation we need way more. Why?
I know that when doing surveys or polls, a sample size of mere hundreds or thousands is often sufficient, even for very large populations.
In the calculator you linked to, it only considers ...
- 4,603
11
votes
When sampling a population for surveys we can often limit our sample size to hundreds, but when doing a Monte Carlo simulation we need way more. Why?
Population size is misleading. Consider the following two experiments:
Experiment 1: A coin lands on heads with unknown probability $p$. We flip the coin $100$ times and try to estimate $p$.
...
- 401
10
votes
Accepted
Is a "trimmed" simple random sample still a simple random sample?
No, this is not a simple-random-sample
Except in the trivial case where all the auxiliary variables are different (which means that your second step consists of choosing any ten random units), this is ...
- 133k
9
votes
Accepted
Question on Covariance for sampling without replacement
Problems in sampling from finite populations without replacement can usually be solved in terms of the sample inclusion probabilities $\pi(x)$, $\pi(x,y)$, etc.
Let $\pi(x) = \Pr(X_1 = x)$ for any $x$...
- 334k
9
votes
Two worlds collide: Using ML for complex survey data
Update May 2022: In terms of accounting for survey weights, there's a nice pair of recent (2020?) articles on arXiv by Dagdoug, Goga, and Haziza. They list many ML-flavored methods and discuss how ...
- 4,603
8
votes
Accepted
Are the differences between sampling clusters and sampling strata, conceptual, methodological, neither or both?
Most U.S. health surveys (NHIS and its kiddo MEPS, NHANES, NSDUH) are stratified cluster surveys. The common representation of the public use data sets is a two-stage design with ~50 strata at the ...
- 32.3k
7
votes
Using post-stratification weights in R survey package
As @StasK says, the correct standard errors for raked/calibrated weights depend on the original weights and the auxiliary variables, and you don't get the right standard errors just by treating them ...
- 46.8k
7
votes
Surveys: Is 25% of a large user base representative?
By etymology "survey" (sur- from 'super', as in 'from above' and -vey from 'view') means to get an overview, not the full ...
- 400
7
votes
When sampling a population for surveys we can often limit our sample size to hundreds, but when doing a Monte Carlo simulation we need way more. Why?
One important side note:
This seems to suggest that any application of the Monte Carlo method can be concluded within several hundred/thousand of simulations as well. But I feel this cannot be true, ...
- 21.6k
6
votes
Why does weighted bootstrap have awful coverage even in toy example?
The weights argument in the boot function is looking for resampling importance weights, not inverse probability weights. When no weights are specified, the bootstrap assumes an importance resampling ...
- 1,197
6
votes
Accepted
Can I CUT a sample to become representative?
I'd not advise throwing out data, but instead doing post-stratification weighting. When you have multiple known demographic targets (e.g. sex, age group, location etc.), the "rake" of Deming ...
- 2,701
5
votes
Accepted
What does this sampling weight mean?
Let $N$ be the population size and $n$ the sample size, let $N_h$ and $n_h$ be the population and sample sizes for stratum $h$.
Then, the weight you defined is given by
$ W_h = \frac{N_h/N}{n_h/n} = ...
- 343
5
votes
Accepted
For the treatment assignment vector in causal inference, what is the difference between assuming a superpopulation vs. a finite-sample population?
For the super population, you don't need to see the potential outcomes as stochastic, neither as infinite. The difference is basically this: the "finite-sample" treatment effect is the causal effect ...
- 12.7k
5
votes
Accepted
Stratified survey calculations by hand and with survey package don't agree. Simulation results
Let's start with this
...
- 46.8k
5
votes
Are the differences between sampling clusters and sampling strata, conceptual, methodological, neither or both?
Stratified sampling is most efficient (in terms of variance of the estimate) when you have homogeneity WITHIN strata and heterogeneity BETWEEN strata. Think US states if your variable of interest were ...
- 1,598
5
votes
Accepted
Minimum sample size calculations for two-sample two-tailed proportion z test
I think the confusion arises because the premise of your question is flawed. In particular, as I understand it, you seem to think there is a discrepancy because you seem to be intuiting a notion that ...
- 2,515
5
votes
How to check the data is generated by machine or human?
Without any further information on the stipulated sampling method for the survey or the meaning of the three outcomes, any possible response could have some from humans or a machine, and there is no ...
- 133k
5
votes
Accepted
Why does the survey package in R and SPSS complex samples add-on give different standard errors?
The R syntax is correct: ~cl1+houseid specifies that cl1 values identify sampling units at stage 1 (PSUs) and ...
- 46.8k
5
votes
Accepted
comparing two samples drawn using two different sampling methods
The question is whether you can estimate the variances and covariance of the means (and the biases, potentially). For usual survey settings you will be able to estimate the variance of each mean, ...
- 46.8k
4
votes
Surveys: Is 25% of a large user base representative?
Another point of view comes from the theory of experiment design.
Statistical power is the probability of finding an effect if it’s
real (source)
Four factors affect power:
Size of the effect
...
- 1,230
4
votes
Two worlds collide: Using ML for complex survey data
R package glmertree allows for fitting decision trees to multilevel data. It allows for specifying a random effects structure, ...
- 3,162
4
votes
Accepted
Proof of the Horvitz-Thompson result
Let $\pi_{i}$ be the probability that unit $U_{i}$ is included in sample of size $n$ by a without replacement sampling procedure.
Now, define a random variable $t_{i}$, for $i=1,2,\cdots N$, by
\...
- 2,127
4
votes
Understanding svycontrast in R with simple random sampling
svycontrast computes "linear or nonlinear contrasts of estimates produced by survey functions (or any object with coef and vcov methods)."
That is, it takes the ...
- 46.8k
4
votes
log transform fixed PH in Cox model - how?
If you don't specify the correct linear form for a continuous predictor in a Cox proportional hazards (PH) model, it's quite possible to get this behavior. Tests for PH come after the regression ...
- 102k
4
votes
Proper subsetting of survey data
Here's a well-written blog post that directly answers your question:
https://notstatschat.rbind.io/2021/07/22/subsets-and-subpopulations-in-survey-inference/
Short answer:
The short answer is that, no,...
- 506
4
votes
Accepted
What are the differences and common points, if any, between oversampling as a survey design method and oversampling in a machine learning context?
You are correct about oversampling in survey design. If you want to read more about it, another useful search terms is "stratified sampling" (which is itself another term that survey stats ...
- 4,603
3
votes
Surveys: Is 25% of a large user base representative?
I sense two questions. One about the sample size (25%, why not a majority) and another about the sampling technique (is it truly random, sample 25% randomly on the entire company, sample 25% randomly ...
- 86.5k
Only top scored, non community-wiki answers of a minimum length are eligible