The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [sampling]

Creating samples from a well-specified population using a probabilistic method and/or producing random numbers from a specified distribution. As this tag is ambiguous, please consider [survey-sampling] for the former and [monte-carlo] or [simulation] for the latter. For questions regarding creating random samples from known distributions, please consider using the [random-generation] tag.

Filter by
Sorted by
Tagged with
228
votes
12answers
67k views

Why does a 95% Confidence Interval (CI) not imply a 95% chance of containing the mean?

It seems that through various related questions here, there is consensus that the "95%" part of what we call a "95% confidence interval" refers to the fact that if we were to exactly replicate our ...
92
votes
3answers
55k views

Can someone explain Gibbs sampling in very simple words? [duplicate]

I'm doing some reading on topic modeling (with Latent Dirichlet Allocation) which makes use of Gibbs sampling. As a newbie in statistics―well, I know things like binomials, multinomials, priors, etc.―,...
54
votes
5answers
14k views

Central limit theorem for sample medians

If I calculate the median of a sufficiently large number of observations drawn from the same distribution, does the central limit theorem state that the distribution of medians will approximate a ...
54
votes
8answers
11k views

Is sampling relevant in the time of 'big data'?

Or more so "will it be"? Big Data makes statistics and relevant knowledge all the more important but seems to underplay Sampling Theory. I've seen this hype around 'Big Data' and can't help wonder ...
46
votes
5answers
24k views

Statistical inference when the sample “is” the population

Imagine you have to do reporting on the numbers of candidates who yearly take a given test. It seems rather difficult to infer the observed % of success, for instance, on a wider population due to the ...
35
votes
4answers
58k views

How to sample from a normal distribution with known mean and variance using a conventional programming language?

I've never had a course in statistics, so I hope I'm asking in the right place here. Suppose I have only two data describing a normal distribution: the mean $\mu$ and variance $\sigma^2$. I want to ...
34
votes
5answers
190k views

Why does increasing the sample size lower the (sampling) variance?

Big picture: I'm trying to understand how increasing the sample size increases the power of an experiment. My lecturer's slides explain this with a picture of 2 normal distributions, one for the null-...
33
votes
5answers
37k views

Is a sample covariance matrix always symmetric and positive definite?

When computing the covariance matrix of a sample, is one then guaranteed to get a symmetric and positive-definite matrix? Currently my problem has a sample of 4600 observation vectors and 24 ...
32
votes
5answers
8k views

Why do political polls have such large sample sizes?

When I watch the news I've noticed that the Gallup polls for things like presidential elections have [I assume random] sample sizes of well over 1,000. From what I remember from college statistics ...
30
votes
5answers
6k views

Strategies for teaching the sampling distribution

The tl;dr version What successful strategies do you employ to teach the sampling distribution (of a sample mean, for example) at an introductory undergraduate level? The background In September I'll ...
28
votes
7answers
6k views

What is the name of the statistical fallacy whereby outcomes of previous coin flips influence beliefs about subsequent coin flips?

As we all know, if you flip a coin that has an equal chance of landing heads as it does tails, then if you flip the coin many times, half the time you will get heads and half the time you will get ...
28
votes
3answers
19k views

What does “independent observations” mean?

I'm trying to understand what the assumption of independent observations means. Some definitions are: "Two events are independent if and only if $P(a \cap b) = P(a) * P(b)$." (Statistical Terms ...
28
votes
3answers
7k views

What if your random sample is clearly not representative?

What if you take a random sample and you can see it is clearly not representative, as in a recent question. For example, what if the population distribution is supposed to be symmetric around 0 and ...
27
votes
4answers
4k views

Probability of not drawing a word from a bag of letters in Scrabble

Suppose you had a bag with $n$ tiles, each with a letter on it. There are $n_A$ tiles with letter 'A', $n_B$ with 'B', and so on, and $n_*$ 'wildcard' tiles (we have $n = n_A + n_B + \ldots + n_Z + n_*...
25
votes
8answers
4k views

How to estimate how many people attended an event (say, a political rally)?

A student asked me today, "How do they know how many people attended a large group event, for example, the Stewart/Colbert 'Rally to Restore Sanity' in Washington D.C.?" News outlets report estimates ...
24
votes
2answers
1k views

The paradox of i.i.d. data (at least for me)

As far as my aggregate (and scarce) knowledge on statistics permits, I understood that if $X_1, X_2,..., X_n$ are i.i.d. random variables, then as the term implies they are independent and identically ...
24
votes
3answers
21k views

Can non-random samples be analyzed using standard statistical tests?

Many clinical studies are based on non-random samples. However, most standard tests (e.g. t-tests, ANOVA, linear regression, logistic regression) are based on the assumption that samples contain "...
24
votes
2answers
12k views

Drawing from Dirichlet distribution

Let's say we have a Dirichlet distribution with $K$-dimensional vector parameter $\vec\alpha = [\alpha_1, \alpha_2,...,\alpha_K]$. How can I draw a sample (a $K$-dimensional vector) from this ...
24
votes
1answer
3k views

Computation of the marginal likelihood from MCMC samples

This is a recurring question (see this post, this post and this post), but I have a different spin. Suppose I have a bunch of samples from a generic MCMC sampler. For each sample $\theta$, I know the ...
24
votes
1answer
22k views

Explanation of finite correction factor

I understand that when sampling from a finite population and our sample size is more than 5% of the population, we need to a correction on the sample's mean and standard error using this formula: $\...
23
votes
1answer
782 views

Should sampling for logistic regression reflect the real ratio of 1's and 0's?

Suppose I want to create logistic regression model which can estimate a probability of occurrence of some animal species living on trees based on characteristics of trees (f.e. height). As always, my ...
22
votes
2answers
17k views

Generating data with a given sample covariance matrix

Given a covariance matrix $\boldsymbol \Sigma_s$, how to generate data such that it would have the sample covariance matrix $\hat{\boldsymbol \Sigma} = \boldsymbol \Sigma_s$? More generally: we are ...
22
votes
1answer
42k views

Why is the sampling distribution of variance a chi-squared distribution?

The statement The sampling distribution of the sample variance is a chi-squared distribution with degree of freedom equals to $n-1$, where $n$ is the sample size (given that the random variable of ...
21
votes
7answers
6k views

Can someone help to explain the difference between independent and random?

In statistics, does independent and random describe the same characteristics? What's the difference between them? We often come across the description like "two independent random variables" or "...
21
votes
1answer
6k views

Bootstrapping vs Bayesian Bootstrapping conceptually?

I'm having a trouble understanding what a Bayesian Bootstrapping process is, and how that would differ from your normal bootstrapping. And if someone could offer an intuitive/conceptual review and ...
21
votes
4answers
12k views

Sampling for Imbalanced Data in Regression

There have been good questions on handling imbalanced data in the classification context, but I am wondering what people do to sample for regression. Say the problem domain is very sensitive to the ...
21
votes
1answer
2k views

Did I just invent a Bayesian method for analysis of ROC curves?

Preamble This is a long post. If you're re-reading this, please note that I've revised the question portion, though the background material remains the same. Additionally, I believe that I've devised ...
20
votes
1answer
10k views

What is the standard error of the sample standard deviation?

I read from there that the standard error of the sample variance is $$SE_{s^2} = \sqrt{\frac{2 \sigma^4}{N-1}}$$ What is the standard error of the sample standard deviation? I'd be tempted to guess ...
20
votes
2answers
8k views

Gibbs sampling versus general MH-MCMC

I have just been doing some reading on Gibbs sampling and Metropolis Hastings algorithm and have a couple of questions. As I understand it, in the case of Gibbs sampling, if we have a large ...
20
votes
2answers
2k views

Simulating time-series given power and cross spectral densities

I am having trouble generating a set of stationary colored time-series, given their covariance matrix (their power spectral densities (PSDs) and cross-power spectral densities (CSDs)). I know that, ...
19
votes
3answers
850 views

How to sample from $c^a d^{a-1} / \Gamma(a)$?

I want to sample according to a density $$ f(a) \propto \frac{c^a d^{a-1}}{\Gamma(a)} 1_{(1,\infty)}(a) $$ where $c$ and $d$ are strictly positive. (Motivation: This could be useful for Gibbs ...
18
votes
4answers
739 views

How to generate a non-integer amount of consecutive Bernoulli successes?

Given: A coin with unknown bias $p$ (Head). A strictly positive real $a > 0$. Problem: Generate a random Bernoulli variate with bias $p^{a}$. Does anyone know how to do this? For instance, when ...
18
votes
1answer
354 views

Sampling model for crowdsourced data?

I'm working on an open health survey application, planned to be used in developing country. The basic idea is that survey interviews are crowdsourced - they are performed by unorganized volunteers ...
18
votes
3answers
1k views

Do optimization techniques map to sampling techniques?

From any generic sampling algorithm, one can derive an optimization algorithm. Indeed, to maximize an arbitrary function $f: \textbf{x} \rightarrow f(\textbf{x})$, it suffices to draw samples from $g ...
17
votes
5answers
2k views

Why we don’t make use of the t-distribution for constructing a confidence interval for a proportion?

To calculate the confidence-interval (CI) for mean with unknown population standard deviation (sd) we estimate the population standard deviation by employing the t-distribution. Notably, $CI=\bar{X} \...
17
votes
3answers
1k views

Generate points efficiently between unit circle and unit square

I'd like generate samples from the blue region defined here: The naive solution is to use rejection sampling in the unit square, but this provides only a $1-\pi/4$ (~21.4%) efficiency. Is there ...
17
votes
2answers
357 views

Is “every blue t-shirted person” a systematic sample?

I'm teaching an intro stats class and was reviewing the types of sampling, including systematic sampling where you sample every kth individual or object. A student asked if sampling every person ...
17
votes
4answers
15k views

Calculating required sample size, precision of variance estimate?

Background I have a variable with an unknown distribution. I have 500 samples, but I would like demonstrate the precision with which I can calculate variance, e.g. to argue that a sample size of 500 ...
17
votes
1answer
7k views

MCMC on a bounded parameter space?

I am trying to apply MCMC on a problem, but my priors(in my case they are $\alpha\in[0,1],\beta\in[0,1]$)) are restricted to an area? Can I use normal MCMC and ignore the samples that fall outside of ...
17
votes
4answers
13k views

Are “random sample” and “iid random variable” synonyms?

I have been facing hard time understanding meaning of "random sample" as well as "iid random variable". I tried to find out the meaning from several sources, but just got more and more confused. I am ...
17
votes
2answers
400 views

Sampling distribution from two independent Bernoulli populations

Let's assume that we have samples of two independent Bernoulli random variables, $\mathrm{Ber}(\theta_1)$ and $\mathrm{Ber}(\theta_2)$. How do we prove that $$\frac{(\bar X_1-\bar X_2)-(\theta_1-\...
16
votes
3answers
4k views

An unbiased estimate of the median

Suppose we have a random variable $X$ supported on $[0,1]$ from which we can draw samples. How can we come up with an unbiased estimate of the median of $X$? We can, of course, generate some samples ...
16
votes
1answer
31k views

Generating random samples from a custom distribution

I am trying to generate random samples from a custom pdf using R. My pdf is: $$f_{X}(x) = \frac{3}{2} (1-x^2), 0 \le x \le 1$$ I generated uniform samples and then tried to transform it to my custom ...
15
votes
5answers
50k views

How does the sampling distribution of sample means approximate the population mean?

I am trying to learn statistics because I find that it is so prevalent that it prohibits me from learning some things if I don't understand it properly. I am having trouble understanding this notion ...
15
votes
3answers
3k views

How can I estimate unique occurrence counts from a random sampling of data?

Let's say I have a large set of $S$ values which sometimes repeat. I wish to estimate the total number of unique values in the large set. If I take a random sample of $T$ values, and determine that ...
15
votes
1answer
11k views

Difference between Randomization test and Permutation test

In the literature the terms Randomization and Permutation are used interchangeably. With many authors stating "Permutation (aka randomization) tests", or vice versa. At best I believe the difference ...
14
votes
2answers
1k views

Can I use moments of a distribution to sample the distribution?

I notice in statistics/machine learning methods, a distribution is often approximated by a Gaussian, and then that Gaussian is used for sampling. They start by computing the first two moments of the ...
14
votes
2answers
3k views

Best suggested textbooks on Bootstrap resampling?

I just wanted to ask which are in your opinion the best available books on bootstrap out there. By this I don't necessarily only mean the one written by its developers. Could you please indicate ...
14
votes
2answers
40k views

What is the difference between sample variance and sampling variance?

What is the difference between sample variance and sampling variance? They seem same. Aren't they?
14
votes
2answers
5k views

What are some techniques for sampling two correlated random variables?

What are some techniques for sampling two correlated random variables: if their probability distributions are parameterized (e.g., log-normal) if they have non-parametric distributions. The data are ...