Creating samples from a well-specified population using a probabilistic method and/or producing random numbers from a specified distribution.
9
votes
3answers
270 views
Determine if a heavy tailed distributed process has improved significantly
I observe processing times of a process before and after a change in order to find out, if the process has improved by the change. The process has improved, if the processing time is reduced.
The ...
13
votes
4answers
2k views
Statistical inference when the sample “is” the population
Imagine you have to do reporting on the numbers of candidates who yearly take a given test. It seems rather difficult to infer the observed % of success, for instance, on a wider population due to the ...
8
votes
3answers
2k views
Calculating required sample size, precision of variance estimate?
Background
I have a variable with an unknown distribution.
I have 500 samples, but I would like demonstrate the precision with which I can calculate variance, e.g. to argue that a sample size of 500 ...
4
votes
2answers
227 views
Sample size for a variable number of answers
I have N different possible results (ex. red, yellow, or blue). I go to a population of infinite size and ask a bunch of people a question the answer to which is one of the N options.... How large ...
1
vote
3answers
135 views
Derive househould weights from a uniformly distributed person sample
The Swiss Public-Use Sample of the national census is a 5% sample drawn from the entire census survey. According to the documentation, the persons are sampled uniformly without replacement. Persons ...
11
votes
1answer
578 views
What are some techniques for sampling two correlated random variables?
What are some techniques for sampling two correlated random variables:
if their probability
distributions are parameterized
(e.g., log-normal)
if they have non-parametric
distributions.
The data ...
16
votes
2answers
305 views
Is “every blue t-shirted person” a systematic sample?
I'm teaching an intro stats class and was reviewing the types of sampling, including systematic sampling where you sample every kth individual or object.
A student asked if sampling every person ...
4
votes
3answers
250 views
How can I simulate census microdata for small areas using a 1% microdata sample at a large scale and aggregate statistics at the small area scale?
I would like to perform an individual-level multivariate analysis at small levels of geographic aggregation (Australian census collection districts). Clearly, the census isn't available at these ...
13
votes
4answers
396 views
How to generate a non-integer amount of consecutive Bernoulli successes?
Given:
A coin with unknown bias $p$ (Head).
A strictly positive real $a > 0$.
Problem:
Generate a random Bernoulli variate with bias $p^{a}$.
Does anyone know how to do this? For instance, ...
10
votes
3answers
464 views
Estimate the size of a population being sampled by the number of repeat observations
Say I have a population of 50 million unique things, and I take 10 million samples (with replacement)... The first graph is I've attached shows how many times I sample the same "thing", which is ...
4
votes
1answer
372 views
Random permutation of a vector with a fixed expected sample correlation to the original?
Suppose you have an $n$-vector $X$. For a fixed real number, $r$ between $-1$ and $1$, can one generate a random permutation of the integers $1,2,\ldots,n$, call it $i_1,i_2,\ldots,i_n$ such that the ...
1
vote
2answers
182 views
What is a representative sample?
I understand the rationale that underpins representative sampling (i.e. to avoid bias so that the sample 'broadly' represents the target population).
Suppose the size of the target population is 600 ...
9
votes
2answers
300 views
How to quickly sample X if exp(X) ~ Gamma?
I have a simple sampling problem, where my inner loop looks like:
v = sample_gamma(k, a)
where sample_gamma samples from the ...
8
votes
1answer
204 views
Definition of quantile
Given N sampled values, what does the "p-th quantile of the sampled values" mean?
7
votes
4answers
246 views
Variance of resistors in parallel
Suppose you have a set of resistors R, all of which are distributed with mean μ and variance σ.
Consider a section of a circuit with the following layout: (r) || (r+r) || (r+r+r). The equivalent ...
5
votes
4answers
871 views
How to interpret the margin of error in a poll?
Recently the media reported on a political poll that stated that "46% of Republican voters in Mississippi think that interracial marriage should be illegal". One example story (of many around the ...
3
votes
1answer
276 views
Posterior distribution for multinomial parameter
(topic moved from maths.stackexchange.com)
I'm currently developing an application integrating a probabilistic inference engine for Bayesian Networks. The Bayesian Network integrates some form of ...
3
votes
1answer
274 views
Sampling from a fixed population
Here's a real basic question. I'm trying to teach myself a bit of stats with Verzani's Using R for Introductory Statistics.
In question 5.13 he asks: A sample of 100 people is drawn from a population ...
1
vote
1answer
408 views
Statistical significance of a survey/poll?
Let's pretend I'm conducting a poll/survey. It's a simple yes/no poll (i.e. everyone only gives 1 of 2 answers). I have asked N people so far, and X of them have said "yes".
I would like to stop ...
0
votes
1answer
965 views
Random sampling (Real data) is so important, why?
How it is understandable for computer (or software) that a sampling method is Random? In fact, it is possible to data sampling by random method in N cases but one of the N cases been like as ...
-1
votes
1answer
405 views
Question regarding sampling, estimation and accuracy [closed]
Let's say there are N chunks of metal, each are 4 inches thick. They are to be hammered and reduced to following best possible sizes: 1, 2, 3 or 4 inches. If we were to use sampling to get an estimate ...
27
votes
8answers
2k views
Is sampling relevant in the time of 'big data'?
Or more so "will it be"? Big Data makes statistics and relevant knowledge all the more important but seems to underplay Sampling Theory.
I've seen this hype around 'Big Data' and can't help wonder ...
13
votes
2answers
401 views
Managing error with GPS routes (theoretical framework?)
I'm looking for the appropriate theoretical framework or speciality to help me deal with understanding how to deal with the errors that the GPS system has - especially when dealing with routes.
...
10
votes
1answer
2k views
Can someone explain Gibbs sampling in very simple words?
I'm doing some reading on topic modeling (with Latent Dirichlet Allocation) which makes use of Gibbs sampling. As a newbie in statistics -- well, I know things like binomials, multinomials, priors etc ...
8
votes
2answers
528 views
How to calculate sample size for simulation in order to assert some level of goodness in my results?
I am a stats newbie, so apologies in advance if I'm asking a braindead question. I have searched for answers to my question, but I find that many of the topics are either too specific, or quickly go ...
5
votes
2answers
405 views
Recommend references on survey sample weighting
Let's aim for some at an introductory level, some articles and some textbooks. Applied is more helpful, including R code is great. Thanks!
5
votes
2answers
484 views
Do confidence intervals apply to quota sampling?
French polling institutes are currently facing a major crisis after they recently published what can only be called the most ridiculous poll so far on the 2012 presidential election horse race. The ...
15
votes
3answers
360 views
How to sample from $c^a d^{a-1} / \Gamma(a)$?
I want to sample according to a density
$$
f(a) \propto \frac{c^a d^{a-1}}{\Gamma(a)} 1_{(1,\infty)}(a)
$$
where $c$ and $d$ are strictly positive.
(Motivation: This could be useful for Gibbs ...
8
votes
2answers
376 views
Solving a simple integral equation by random sampling
Let $f$ be a nonnegative function. I am interested in finding $z \in [0,1]$ such that
$$ \int_0^{z} f(x)\,dx = \frac{1}{2}\int_0^1 f(x)\,dx$$ The caveat: all I can do is sample $f$ at points in ...
6
votes
2answers
283 views
How can I estimate unique occurrence counts from a random sampling of data?
Let's say I have a large set of $S$ values which sometimes repeat. I wish to estimate the total number of unique values in the large set.
If I take a random sample of $T$ values, and determine that ...
5
votes
1answer
201 views
How do I determine how well a dataset approximates a distribution?
Quite simple, I have some probability distribution p(x), how can I measure whether one empirical density (set of delta masses) is a better approximation than another. I know that KL-divergence is a ...
4
votes
5answers
1k views
What exactly does 'representative sample' refer to?
When reading passages like the following:
Based on a representative sample of 88 recent raids, we show that the Turkana sustain costly cooperation in combat at a remarkably large scale, at ...
4
votes
1answer
216 views
Assessing the representativeness of population sampling
I am looking for some suggestions about assessing the representativeness of a particular dataset I am analyzing.
In this dataset I am looking at the relationship between two variables (e.g., X and ...
4
votes
1answer
307 views
How to correct uneven sampling distribution when calculating the mean?
Suppose I have a function f, and I want to sample it at 100 points in the interval [0, 100]. For some reason (that seemed smart ...
4
votes
1answer
2k views
Explanation of finite correction factor
I understand that when sampling from a finite population and our sample size is more than 5% of the population, we need to a correction on the sample's mean and standard error using this formula:
...
3
votes
1answer
319 views
With simple random sampling, how to approximate variance of R=avg(Y)/avg(X)?
Recenly I am reading "Mathematical statistics and data analysis" written by Rice myself. At page 207, theorem A said:
With simple random sampling, approimxate variance of ...
3
votes
1answer
153 views
Optimal importance sampling with ratio estimator
This is probably a stupid question, but here goes: so we want to approximate the following expectation:
$$\mathbb{E}[h(x)] = \int h(x)\pi(x) dx$$
Where $h(x)$ is an arbitrary function and $\pi(x)$ is ...
3
votes
6answers
338 views
Inference to the population when the survey response rate is only 30%
I have conducted a survey in which the questionnaires were sent out to 450 individuals, but only 30% of them answered the questionnaires.
Is it still valid to interpret the usual inference analysis ...
2
votes
1answer
88 views
Most efficient way of sampling a product of distributions $k \cdot f(x) \cdot g(x)$, given that PDFs $f(x)$ and $g(x)$ can easily sampled?
Is there an efficient approach tailored to sampling from a PDF $k \cdot f(x) \cdot g(x)$ (where $k$ is a normalizing constant) that would perform better than naive Metropolis, slice sampling, etc.? ...
2
votes
0answers
113 views
Directly compare subpixel shifts between two spectra — and get believable errors!
I have two spectra of the same astronomical object. The essential question is this: How can I calculate the relative shift between these spectra and get an accurate error estimate on that shift?
Some ...
2
votes
1answer
141 views
Do low-discrepancy sequences work in discrete spaces?
Low-discrepancy sequences in a real space ($[0,1]^n$) seem like a really excellent tool for evenly sampling a sample space. As far as I can tell, they generalise well to any real space, if you use an ...
2
votes
3answers
926 views
Interpreting a negative confidence limit for a proportion
The margin of error is driven by the size of the sample.
In a consultant's report (which is confidential at this stage), they collected responses from 10 store managers (out of a total of 200 store ...
1
vote
2answers
198 views
Calculating % unsampled in sampling with replacement
You sample N of N items with replacement.
How do you calculate the expected percent not sampled from original population ...
19
votes
3answers
984 views
What if your random sample is clearly not representative?
What if you take a random sample and you can see it is clearly not representative, as in a recent question. For example, what if the population distribution is supposed to be symmetric around 0 and ...
7
votes
2answers
497 views
Using MCMC to evaluate the expected value of a high-dimensional function
I am working on a research project that is related to optimization and recently had an idea to use MCMC in this setting. Unfortunately, I am fairly new to MCMC methods so I had several questions. I'll ...
4
votes
2answers
236 views
How to sample uniformly from an intersection of simplices?
The title says it all. Specifically: given strictly positive real numbers $a_1,\dots,a_T$ and $b_1,\dots,b_T$, I want to sample from $$\mu:=\text{Uniform}(\{p\in[0,\infty)^T : \sum_{t=1}^T a_t p_t = ...
4
votes
1answer
114 views
Information content of examples and undersampling
As I have written in my question "How much undersampling should be done?", I want to predict defaults, where a default is per se really unlikely (average ~ 0.3 percent). My models are not affected by ...
4
votes
1answer
186 views
How much undersampling should be done?
My aim is to predict quarterly customer-default probabilities: I have data on ~ 2 million individuals, who default on average with a probability of ~ 0.3 percent.
Therefore I am thinking about ...
3
votes
1answer
348 views
Correcting sample bias
I am studying the correlation of two observed variables (call them $A$ and $B$). the underlying distribution for $A$ is symmetric around $0$ (for sure), however in my sample I have $411$ observations ...
3
votes
3answers
624 views
Identifying the population and samples in a study
I apologise for the silliness of this question. I ran an e-learning experiment using a class of undergrads. Participation was voluntary, so only half of the class participated.
I know that my sample ...
