Questions tagged [sampling]

Creating samples from a well-specified population using a probabilistic method and/or producing random numbers from a specified distribution. As this tag is ambiguous, please consider [survey-sampling] for the former and [monte-carlo] or [simulation] for the latter. For questions regarding creating random samples from known distributions, please consider using the [random-generation] tag.

Filter by
Sorted by
Tagged with
4
votes
1answer
259 views

How to sample uniformly from an intersection of simplices?

The title says it all. Specifically: given strictly positive real numbers $a_1,\dots,a_T$ and $b_1,\dots,b_T$, I want to sample from $$\mu:=\text{Uniform}(\{p\in[0,\infty)^T : \sum_{t=1}^T a_t p_t = 1\...
6
votes
0answers
412 views

Choosing the number of latin hypercube samples [closed]

I'm using latin hypercube sampling to draw parameter values for an epidemiological simulation model (about 30 parameters, following various distributions), and so far I haven't been able to find any ...
4
votes
0answers
654 views

Sample size to achieve given confidence level

I am trying to estimate number of different unique visitors who visited a given website (online store). There are hundreds of millions of visits to the store and so this task is too difficult to ...
1
vote
2answers
751 views

Expectation of the variance of the sampling set without replacement

Select $n$ numbers without replacement from the set $\{1,2,...,m\}$, and generate the set $S=\{a_1,a_2,...,a_n\}$. I want to calculate the expectation of the variance for the sampling set $\mathbb{E}[...
4
votes
2answers
1k views

How to calculate the number of trials for a 'meaningful' result

Apologies for newbie question but have failed to understand what I've read about in this area so pointers to the correct methods would focus my attention. I have a piece of software which when tested ...
9
votes
3answers
8k views

Acceptance rates for Metropolis-Hastings with uniform candidate distribution

When running the Metropolis-Hastings algorithm with uniform candidate distributions, what is the rationale of having acceptance rates around 20%? My thinking is: once the true (or close to true) ...
7
votes
1answer
455 views

Estimating the variance of poker win rates

Suppose you have a casino with n poker players. Each player has a win rate - the amount of money he wins or loses per hand. We assume that these win rates are normally distributed with a mean of 0. (...
5
votes
1answer
589 views

How to combine two independent repeated experiments with different success probabilities?

Repeating an experiment (about which I asked before) with $n$ possible outcomes $t$ times independently, where all but one outcomes have probability $\frac{1}{n+1}$ and the other outcome has the ...
9
votes
1answer
6k views

What is a propensity weighting sampling / RIM?

I have come across the sampling method called "Propensity Weighting Sampling/RIM", but I do not have a good idea of what these survey methods are all about. What references in the literature cover ...
3
votes
2answers
194 views

Bias in sampling for set intersections

Say I have 2 sets, $A$ and $B$ with $n_{A}$ and $n_{B}$ elements respectively, which I assume is known. I would like to estimate $| A \bigcup B |$ using samples of $\tilde{A} \subset A$ and $ \tilde{...
3
votes
0answers
399 views

Exponential moving average with sub-interval relevance / varying timeframe

I need to calculate an exponential moving average for a series of data. The intended sampling interval is fixed (say 1s) but the data stream has varying intervals (data intervals vary from 0.01s to ...
4
votes
1answer
1k views

Monte carlo integration in spherical coordinates

I was playing around with writing a code for Montecarlo integration of a function defined in spherical coordinates. As a first simple rapid test I decided to write a test code to obtain the solid ...
9
votes
3answers
3k views

Recommend references on survey sample weighting

Let's aim for some at an introductory level, some articles and some textbooks. Applied is more helpful, including R code is great. Thanks!
8
votes
1answer
237 views

Who first developed the idea of “sampling distributions”?

Who created sampling distributions? I have looked everywhere and I'm writing a paper, but, so far, all I've been able to come up with is the theory and the definition. Please help me find the who, ...
10
votes
1answer
370 views

Definition of quantile

Given N sampled values, what does the "p-th quantile of the sampled values" mean?
92
votes
3answers
55k views

Can someone explain Gibbs sampling in very simple words? [duplicate]

I'm doing some reading on topic modeling (with Latent Dirichlet Allocation) which makes use of Gibbs sampling. As a newbie in statistics―well, I know things like binomials, multinomials, priors, etc.―,...
5
votes
1answer
384 views

Modeling multinomial problems with unknown sample size in BUGS

I'm trying to estimate the design effect of a series of relatively small sample size surveys ($n\sim 70$) with multiple responses. Design effects roughly correspond to how much larger actual sample ...
1
vote
2answers
222 views

Sampling dataset, choosing among N dimensions

I'm new in stats, and maybe this is a duplicated question, but I could not find a similar one. I'm trying to reduce a dimension of my dataset. Maybe reduce is not a good word. I need to sample some ...
1
vote
1answer
2k views

How to handle skewed binary target variables? [duplicate]

Possible Duplicate: Supervised learning with “rare” events, when rarity is due to the large number of counter-factual events I am trying to predict diabetes using the BRFSS dataset by ...
1
vote
1answer
4k views

How to compute margin of error with a given confidence interval?

I was given the following question: A survey found that 89% of a random sample of 1024 American adults approved of cloning endangered animals. Find the margin of error for this survey if we want 90% ...
2
votes
2answers
28k views

Sampling error example

I've read the sampling error examples in Wikipedia, but I still don't understand. Could you give me a simple and characteristic example for sampling error?
6
votes
2answers
198 views

Resampling within a survey to account for missing data

Suppose I have survey responses that look like this: ...
3
votes
3answers
404 views

How to check reliability of a given sampling technology?

Very basic question, I suspect. It's sort of the inverse of this one, and the same disclaimers about my own ignorance apply, but here goes: I have a sampling technology. I know the total population, ...
2
votes
2answers
108 views

Estimating the number of registered interaction designers in a given territory

I plan to conduct a research study about Interaction Designers. Prior to starting on sampling methods I am finding it very hard to get my hands on some demographics and statistics, e.g. How many ...
0
votes
2answers
127 views

How do I compare a matrix of population values to a matrix of sample values?

Let's say I have a matrix where the underlying data is from the entire population. Let's say I also have a matrix where the underlying data has been sampled from population. How would I analyze how ...
2
votes
1answer
999 views

Nonparametric expected value estimation of sample from unknown distribution

I have a data sample (in this case an EEG data sample, but my question refers to any type of data samples of prior unknown distributions). I would like to do a nonparametric estimate of the expected ...
8
votes
1answer
2k views

How to use/interpret empirical distribution?

First of all I'd like to apologize for the vague title, I couldn't really formulate a better one just now, please feel free to change, or advice me to change, the title to make it better fit the core ...
10
votes
3answers
2k views

Using MCMC to evaluate the expected value of a high-dimensional function

I am working on a research project that is related to optimization and recently had an idea to use MCMC in this setting. Unfortunately, I am fairly new to MCMC methods so I had several questions. I'll ...
8
votes
2answers
1k views

Do confidence intervals apply to quota sampling?

French polling institutes are currently facing a major crisis after they recently published what can only be called the most ridiculous poll so far on the 2012 presidential election horse race. The ...
3
votes
2answers
2k views

Choosing sample size to achieve pre-specified margin-of-error

If I want to achieve a margin-of-error of <= 5 % for a representative population sample, how large a sample do I need when: The interviewees are picked from X regions and from Y age groups? That is,...
12
votes
2answers
1k views

How to quickly sample X if exp(X) ~ Gamma?

I have a simple sampling problem, where my inner loop looks like: v = sample_gamma(k, a) where sample_gamma samples from the ...
3
votes
1answer
1k views

How to calculate confidence interval when only a part of the samples are valid?

I will simplify our problem in this way. Say there are 100,000 cases in total to examine. Due to the time limitation, we randomly selected 2,000 of them. Then we found 1,000 of them are invalid, so we ...
1
vote
2answers
2k views

Percent error or percent difference?

Using either percent error or percent difference, I want to compare one of my measured values from a set to the mean of the set. By reading the Wikipedia article on percent difference, it's still not ...
9
votes
1answer
292 views

Can I estimate the frequency of an event based on random samplings of its occurrence?

Some edits made... This question is just for fun, so if it isn't fun then please feel free to ignore it. I already get a lot of help from this site so I don't want to bite the hand that feeds me. It'...
5
votes
1answer
835 views

Assessing the representativeness of population sampling

I am looking for some suggestions about assessing the representativeness of a particular dataset I am analyzing. In this dataset I am looking at the relationship between two variables (e.g., X and Y)...
14
votes
2answers
5k views

What are some techniques for sampling two correlated random variables?

What are some techniques for sampling two correlated random variables: if their probability distributions are parameterized (e.g., log-normal) if they have non-parametric distributions. The data are ...
9
votes
2answers
19k views

How to make representative sample set from a large overall dataset?

What are the statistical techniques to create a sample set, which is representative of the entire population (with a known confidence level)? Also, How to validate, if the sample fits the overall ...
4
votes
1answer
683 views

Sampling from a fixed population

Here's a real basic question. I'm trying to teach myself a bit of stats with Verzani's Using R for Introductory Statistics. In question 5.13 he asks: A sample of 100 people is drawn from a population ...
4
votes
1answer
1k views

How to correct uneven sampling distribution when calculating the mean?

Suppose I have a function f, and I want to sample it at 100 points in the interval [0, 100]. For some reason (that seemed smart ...
4
votes
2answers
249 views

Checking the distribution of classes in a sample of a dataset

I am new here, so I might have missed a similar question. I am trying to do things good in my research and using proper statistical approaches, but in computer science, we had quite a weak training ...
4
votes
1answer
147 views

Information content of examples and undersampling

As I have written in my question "How much undersampling should be done?", I want to predict defaults, where a default is per se really unlikely (average ~ 0.3 percent). My models are not affected by ...
6
votes
4answers
917 views

Which one should be applied first: data sampling or dimensionality reduction?

I am working on binary classification problem. Data set is very large and highly imbalanced. Data dimensionality is also very high. Now I want to balance data by under-sampling the majority class, and ...
6
votes
3answers
2k views

Multistage sampling in R

I've got a dataset similar to this: ...
4
votes
1answer
555 views

How much undersampling should be done?

My aim is to predict quarterly customer-default probabilities: I have data on ~ 2 million individuals, who default on average with a probability of ~ 0.3 percent. Therefore I am thinking about ...
17
votes
4answers
15k views

Calculating required sample size, precision of variance estimate?

Background I have a variable with an unknown distribution. I have 500 samples, but I would like demonstrate the precision with which I can calculate variance, e.g. to argue that a sample size of 500 ...
17
votes
2answers
357 views

Is “every blue t-shirted person” a systematic sample?

I'm teaching an intro stats class and was reviewing the types of sampling, including systematic sampling where you sample every kth individual or object. A student asked if sampling every person ...
3
votes
1answer
279 views

Sample selection and variation in the variable of interest when using panel data

I wonder how to reason when selecting samples. I am doing a panel data regression analysis about how the euro membership correlates with the budget deficit in the member countriues. Using R that is: ...
6
votes
1answer
924 views

Balanced sampling for network training?

I have a very unbalanced sample set, e.g. 99% true and 1% false. Is it reasonable to select a balanced subset with a 50/50-distribution for neural network training? The reason for this is, that I ...
4
votes
2answers
648 views

Optimal stopping from an unknown distribution

The Secretary problem has an algorithm for fixed N and immediate accept/reject (that is, reject reject ... accept one, stop). There are several variants; in mine, secretaries or samples come from a ...
2
votes
2answers
299 views

Survey Sampling: What am I missing with this plan?

Working off a fairly limited statistical understanding, so apologies in advance if this question is ludicrously basic. This has to have happened to other people who have successfully solved this ...