The Stack Overflow podcast is back! Listen to an interview with our new CEO.

# Questions tagged [sampling]

Creating samples from a well-specified population using a probabilistic method and/or producing random numbers from a specified distribution. As this tag is ambiguous, please consider [survey-sampling] for the former and [monte-carlo] or [simulation] for the latter. For questions regarding creating random samples from known distributions, please consider using the [random-generation] tag.

2,165 questions
Filter by
Sorted by
Tagged with
55 views

### Sampling from/near boundary of a region in R^n

Suppose $\Omega$ is a region in $\mathbb R^n$, and suppose we are given a function $\chi(x)$ with $\chi(x)=1$ if $x\in \Omega$ and $\chi(x)=0$ otherwise. If it helps we can assume $\Omega\subseteq B$ ...
27 views

### How can I generate a completely balanced combination

Attaching a code that generates unique combinations of 8 letters but, it still lacks a condition wherein all letters should have equal counts per column. For my code, basically, letter a to h should ...
15 views

### Mean of repeated samples

Let's assume we have random variables $X_1\sim N(\mu_1,1),\cdots,X_n\sim N(\mu_n,1)$. Now we take one sample from each and get $X_1 = x_1,\cdots,X_n = x_n$. We order them and calculate the mean of top ...
5 views

### Can Crombach's alpha be used to argue for the representativeness of non-probability sample?

My understanding is that Crombach's alpha measures the reliability of survey items in a scale, when they are supposed to be a measure of a construct or concept. However, I'm reading a non-published ...
20 views

### relation between location of a sample mean in sampling distribution and the standard error

Can someone explain the following statement with an example We'll describe the location of the sample mean by calculating how many standard errors it is away from the center of the sampling ...
12 views

### Sampling and Standardization of data before applying dimentionality reduction?

I'm trying to solve a classification problem with 4 parameters, next_action - binary variable(0/1) total_visits- numerical value days_Since_last_visit - numerical lead_source- categorical variable (5 ...
20 views

### Categorical sampling without instantiating probability vector

I want to sample from a discrete distribution with probability vector $p \in \mathbb R^n$, where $n$ is large. Suppose that $p_i = f_i / Z$, where $Z$ is a normalization constant. I can compute the ...
12 views

### An algorithm of step-wise sampling from higher dimensional joint distribution, is it valid?

In a simple example, assume joint distribution $\{X,Y,Z\}\sim F_{XYZ}$ The algorithm works as follows: Sample $x$ from the marginal distribution of $X$, $F_X$. Sample $y$ from the distribution of $Y$...
26 views

### Antithetic variate for Chi-squared distribution?

I am using antithetic sampling for variance reduction. I know for standard normal $z$, it's antithetic variate is $-z$ ($1-U$ for uniform etc). But I cannot figure out what would be antithetic ...
12 views

### Explain Sequential Importance Sampling to someone with basic statistics knowledge?

Assume I've taken only basic statistics and a good amount of math. How would you explain Sequential Importance Sampling to a beginner in a way they can understand?
44 views

### Determining sample size for given confidence interval and margin of error

I want to estimate the true runtime of a class of programs that run on a platform that introduces virtualization-related variance into the runtime. Quantitatively, my goal is to be able to state the ...
10 views

### sampling for estimating poplation weighted average

I have a very large population, or a set X. there is a probability function on X (that is, each member of X is assigned a probability). also, each member of X has a certain numeric property V. I want ...
7 views

### .632+ bootstrap implementation in the mlxtend package

I'm probably missing something, so I'm posting this question assuming someone could explain my misunderstanding. After reading Improvements on cross-validation: the 632+ bootstrap method as well as ...
30 views

### how to determine correct sample size?

I have 2,000 transactions and I have to check the accuracy of those transactions. I want to select a sample, check all transactions in this sample and make conclusions on population with 95 percent of ...
20 views

### Study Setup for a research project

I have 180 observations that were collected in a healthcare environment. The observations were identified in a prospective manner. Outcomes to be measured are accuracy of group A assessment when ...
73 views

### What is the “Monte Carlo” part in MCMC algorithms like Metropolis-Hastings and Gibbs sampling?

I know that Monte Carlo is used to approximate an integral by sampling. I also learnt MCMC algorithms such as Metropolis-Hastings and Gibbs sampling but I don't know where the "Monte Carlo" part is in ...
33 views

### Estimating Required Data for NLP Classification Models

Are there general guidelines for how much data is required for natural language processing (NLP) classification models? I understand this may depend on the text quality, text length, how accurate the ...
14 views

### Aggregate data for machine learning. Weights or fake disaggregation?

I have a dataset of medical centers and I need to predict their infection rate, based on the center characteristics and aggregated patient data (eg. percentage of patients which underwent a certain ...
21 views

### Does a class survey count as a random sampling?

If we had a class of 200 people and we made a survey in which anyone in the class can choose to participate or not, does this count as random sampling? I feel like it's not, but I can't put it into ...
20 views

### Sampling to validate a cluster

I am trying to compute the silhouette coefficient in a clustered dataset which is 550k, but as the process is computational intensive I run out of memory and I cannot compute the silhouette ...
18 views

### Data sub-set question

I have a data set of 15,000 with the following characteristics: median : 3.3 avg : 4.3 st dev : 3.5 kurtosis : 19.05 skew : 3.94 Now from within that I have a smaller sub-set. There are 1,160 in ...
10 views

### Margin of error in total area estimate

Suppose I have a sample of size $n$ where each unit is a soybean producer from a specific state. I want to estimate the total cultivated area in this state. I already have an estimate of the total ...
17 views

### Sampling from Gaussian distribution subject to a quadratic inequality constraint

Would it be possible to generate samples $x \in \mathbb{R}^n$ from $\mathcal{N}(\mu, \Sigma)$ subject to an inequality constraint $x^\top Q x/2 + b^\top x \le c$, $Q = Q^\top \succeq 0$. We also know, ...
57 views

### What does it mean to obtain a sample $S$ of size $n$ according to a distribution $D$ over a set $X$ in machine learning?

What does it mean to obtain a sample $S$ of size $n$ according to a distribution $D$ over a set $X$ in machine learning?
22 views

### Estimating from Biased Sample

ORIGINAL: This is a rather convoluted problem, but I will try to explain it as clearly as I can. I have a discrete, known population with 1,000,000 possible values (call this set A), with roughly ...
7 views

### Removing Bias/Selecting Stable Subsample From a Panel of Longitudinal Observational Data?

We have a large observational panel dataset (3 million individuals over 5 years) and are trying to use aggregated quantities from the panel to predict an external time series which we know to be the ...
29 views

### How to address large sampling units but small sample sizes?

I've got data from eight one-hectare tropical forest plots, with all trees greater than a certain size measured and identified within them. These plots are different forest ages with two replicates ...
16 views

### how to determine sample size for continuous metric

many online sample size calculator like http://www.evanmiller.org/ab-testing/sample-size.html only have sample size calculation for discrete metric(like conversion rate) how to calculate sample size ...
26 views

### Representativity of a sample to a reference population with a mixed distribution

I a have a series of small samples taken not randomly from a reference population with a complex distribution over a categoric variable with 21 levels and a continuous one in the x > 0 domain. I ...
57 views

### Prove that the variance of the sample mean is smaller than that of the mean of a simple random sample of the same size n drawn with replacement

In statistics, a simple random sample is a subset of individuals chosen (one by one) from a population. Each individual is chosen randomly such that each individual has the same probability of being ...
14 views

### Expected value for a sequence of group samplings from a fixed population

Suppose I've 100 apples where 25 of them are bad and the remaining 75 are good. I draw apples 20-by-20 from this group of apples. That is, I draw the first set of 20 apples from 100 apples, second ...
11 views

### Clarification on Central Limit Theorem (Sample Mean vs. Sample Distribution)

I know the CLT says that as sample size (n) increases, the distribution of the sample mean has a mean that approaches the population mean and the variance of those means approaches (σ^2/n). The way I ...
35 views

### Validation Set Accuracy Significantly Higher than Hold out Test Set

I'm building a binary classifier, where each record is a task, and the response variable is whether it was completed on time. I'm using random forest My data set spans from 2000-2015 My hold out ...