The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [sampling]

Creating samples from a well-specified population using a probabilistic method and/or producing random numbers from a specified distribution. As this tag is ambiguous, please consider [survey-sampling] for the former and [monte-carlo] or [simulation] for the latter. For questions regarding creating random samples from known distributions, please consider using the [random-generation] tag.

Filter by
Sorted by
Tagged with
1
vote
1answer
55 views

Sampling from/near boundary of a region in R^n

Suppose $\Omega$ is a region in $\mathbb R^n$, and suppose we are given a function $\chi(x)$ with $\chi(x)=1$ if $x\in \Omega$ and $\chi(x)=0$ otherwise. If it helps we can assume $\Omega\subseteq B$ ...
0
votes
1answer
27 views

How can I generate a completely balanced combination

Attaching a code that generates unique combinations of 8 letters but, it still lacks a condition wherein all letters should have equal counts per column. For my code, basically, letter a to h should ...
2
votes
0answers
15 views

Mean of repeated samples

Let's assume we have random variables $X_1\sim N(\mu_1,1),\cdots,X_n\sim N(\mu_n,1)$. Now we take one sample from each and get $X_1 = x_1,\cdots,X_n = x_n$. We order them and calculate the mean of top ...
0
votes
0answers
5 views

Can Crombach's alpha be used to argue for the representativeness of non-probability sample?

My understanding is that Crombach's alpha measures the reliability of survey items in a scale, when they are supposed to be a measure of a construct or concept. However, I'm reading a non-published ...
1
vote
1answer
20 views

relation between location of a sample mean in sampling distribution and the standard error

Can someone explain the following statement with an example We'll describe the location of the sample mean by calculating how many standard errors it is away from the center of the sampling ...
0
votes
1answer
12 views

Sampling and Standardization of data before applying dimentionality reduction?

I'm trying to solve a classification problem with 4 parameters, next_action - binary variable(0/1) total_visits- numerical value days_Since_last_visit - numerical lead_source- categorical variable (5 ...
1
vote
1answer
20 views

Categorical sampling without instantiating probability vector

I want to sample from a discrete distribution with probability vector $p \in \mathbb R^n$, where $n$ is large. Suppose that $p_i = f_i / Z$, where $Z$ is a normalization constant. I can compute the ...
0
votes
0answers
12 views

An algorithm of step-wise sampling from higher dimensional joint distribution, is it valid?

In a simple example, assume joint distribution $\{X,Y,Z\}\sim F_{XYZ}$ The algorithm works as follows: Sample $x$ from the marginal distribution of $X$, $F_X$. Sample $y$ from the distribution of $Y$...
0
votes
1answer
26 views

Antithetic variate for Chi-squared distribution?

I am using antithetic sampling for variance reduction. I know for standard normal $z$, it's antithetic variate is $-z$ ($1-U$ for uniform etc). But I cannot figure out what would be antithetic ...
0
votes
0answers
12 views

Explain Sequential Importance Sampling to someone with basic statistics knowledge?

Assume I've taken only basic statistics and a good amount of math. How would you explain Sequential Importance Sampling to a beginner in a way they can understand?
2
votes
1answer
44 views

Determining sample size for given confidence interval and margin of error

I want to estimate the true runtime of a class of programs that run on a platform that introduces virtualization-related variance into the runtime. Quantitatively, my goal is to be able to state the ...
0
votes
0answers
10 views

sampling for estimating poplation weighted average

I have a very large population, or a set X. there is a probability function on X (that is, each member of X is assigned a probability). also, each member of X has a certain numeric property V. I want ...
0
votes
0answers
7 views

.632+ bootstrap implementation in the mlxtend package

I'm probably missing something, so I'm posting this question assuming someone could explain my misunderstanding. After reading Improvements on cross-validation: the 632+ bootstrap method as well as ...
2
votes
1answer
30 views

how to determine correct sample size?

I have 2,000 transactions and I have to check the accuracy of those transactions. I want to select a sample, check all transactions in this sample and make conclusions on population with 95 percent of ...
0
votes
0answers
20 views

Study Setup for a research project

I have 180 observations that were collected in a healthcare environment. The observations were identified in a prospective manner. Outcomes to be measured are accuracy of group A assessment when ...
2
votes
1answer
73 views

What is the “Monte Carlo” part in MCMC algorithms like Metropolis-Hastings and Gibbs sampling?

I know that Monte Carlo is used to approximate an integral by sampling. I also learnt MCMC algorithms such as Metropolis-Hastings and Gibbs sampling but I don't know where the "Monte Carlo" part is in ...
0
votes
0answers
33 views

Estimating Required Data for NLP Classification Models

Are there general guidelines for how much data is required for natural language processing (NLP) classification models? I understand this may depend on the text quality, text length, how accurate the ...
0
votes
0answers
14 views

Aggregate data for machine learning. Weights or fake disaggregation?

I have a dataset of medical centers and I need to predict their infection rate, based on the center characteristics and aggregated patient data (eg. percentage of patients which underwent a certain ...
1
vote
1answer
21 views

Does a class survey count as a random sampling?

If we had a class of 200 people and we made a survey in which anyone in the class can choose to participate or not, does this count as random sampling? I feel like it's not, but I can't put it into ...
1
vote
1answer
20 views

Sampling to validate a cluster

I am trying to compute the silhouette coefficient in a clustered dataset which is 550k, but as the process is computational intensive I run out of memory and I cannot compute the silhouette ...
0
votes
0answers
18 views

Data sub-set question

I have a data set of 15,000 with the following characteristics: median : 3.3 avg : 4.3 st dev : 3.5 kurtosis : 19.05 skew : 3.94 Now from within that I have a smaller sub-set. There are 1,160 in ...
0
votes
0answers
10 views

Margin of error in total area estimate

Suppose I have a sample of size $n$ where each unit is a soybean producer from a specific state. I want to estimate the total cultivated area in this state. I already have an estimate of the total ...
3
votes
0answers
17 views

Sampling from Gaussian distribution subject to a quadratic inequality constraint

Would it be possible to generate samples $x \in \mathbb{R}^n$ from $\mathcal{N}(\mu, \Sigma)$ subject to an inequality constraint $x^\top Q x/2 + b^\top x \le c$, $Q = Q^\top \succeq 0$. We also know, ...
2
votes
1answer
57 views

What does it mean to obtain a sample $S$ of size $n$ according to a distribution $D$ over a set $X$ in machine learning?

What does it mean to obtain a sample $S$ of size $n$ according to a distribution $D$ over a set $X$ in machine learning?
0
votes
0answers
22 views

Estimating from Biased Sample

ORIGINAL: This is a rather convoluted problem, but I will try to explain it as clearly as I can. I have a discrete, known population with 1,000,000 possible values (call this set A), with roughly ...
0
votes
0answers
7 views

Removing Bias/Selecting Stable Subsample From a Panel of Longitudinal Observational Data?

We have a large observational panel dataset (3 million individuals over 5 years) and are trying to use aggregated quantities from the panel to predict an external time series which we know to be the ...
0
votes
1answer
29 views

How to address large sampling units but small sample sizes?

I've got data from eight one-hectare tropical forest plots, with all trees greater than a certain size measured and identified within them. These plots are different forest ages with two replicates ...
0
votes
0answers
16 views

how to determine sample size for continuous metric

many online sample size calculator like http://www.evanmiller.org/ab-testing/sample-size.html only have sample size calculation for discrete metric(like conversion rate) how to calculate sample size ...
0
votes
0answers
26 views

Representativity of a sample to a reference population with a mixed distribution

I a have a series of small samples taken not randomly from a reference population with a complex distribution over a categoric variable with 21 levels and a continuous one in the x > 0 domain. I ...
1
vote
1answer
57 views

Prove that the variance of the sample mean is smaller than that of the mean of a simple random sample of the same size n drawn with replacement

In statistics, a simple random sample is a subset of individuals chosen (one by one) from a population. Each individual is chosen randomly such that each individual has the same probability of being ...
0
votes
0answers
14 views

Expected value for a sequence of group samplings from a fixed population

Suppose I've 100 apples where 25 of them are bad and the remaining 75 are good. I draw apples 20-by-20 from this group of apples. That is, I draw the first set of 20 apples from 100 apples, second ...
0
votes
0answers
11 views

Clarification on Central Limit Theorem (Sample Mean vs. Sample Distribution)

I know the CLT says that as sample size (n) increases, the distribution of the sample mean has a mean that approaches the population mean and the variance of those means approaches (σ^2/n). The way I ...
0
votes
0answers
35 views

Validation Set Accuracy Significantly Higher than Hold out Test Set

I'm building a binary classifier, where each record is a task, and the response variable is whether it was completed on time. I'm using random forest My data set spans from 2000-2015 My hold out ...
0
votes
0answers
29 views

Generating variables using R

Question: Assume 2 treatments, 1 and 0. Given 120 participants, generate underlying truth with Age as a discrete uniform distribution over integers between [20-60] Gender (G) as $I\{G = F\} \sim \...
2
votes
1answer
33 views

Is this a valid way to get a simple random sample?

I have a data set of about 200,000 observations about establishments in my district and their operating time. I need to establish the correctness of my data. Consequently, I am trying to take a simple ...
0
votes
0answers
17 views

Calculate proper number of elements in a sample

Suppose that I own several convenience stores across the U.S. and I want to infer the share of market for different brands by just analyzing their sell behavior in my stores (ex. I might infer that ...
2
votes
1answer
21 views

Why systematic random sample for exit polling?

I was sitting in class today and we were going over how systematic random samples are used in election exit polling e.g. say we want to sample 1/15 people that come to vote. choose some n between 1 ...
0
votes
0answers
8 views

Marginalisation when conditioning on the Fourier transform of a random variable

I am looking to sample from a distribution $p(y)$ defined by the following expectation: $$p(y) = \mathbb{E}_{p(u)} \left[ p(y|u) \right]$$ Both $p(u)$ and $p(y|u)$ are multivariate Gaussians: $$p(u)...
0
votes
0answers
6 views

Sampling trying to keep as much multivariate variance as possible

We can use PCA for dimensionality reduction, but at the cost of getting "uninterpretable" variables. I was thinking if anyone considered a sampling technique that would try to aim keeping as much of ...
0
votes
0answers
10 views

Calculating the error of data sampling in intersecting sets

I am faced right with a simple (I imagine!) statistics problem as part of my job, but as non-statistician I have no idea how to get a hold of it. Here it is: Let $X_i \subset X, 0<i<n$ for some ...
0
votes
0answers
27 views

Nested sampling: estimate of bulk posterior support over prior

Going through the details of the Nested sampling Skilling paper, and I've encountered an estimate in Section 5 which I cannot reproduce. Rephrasing what's mentioned in the paper: we assume to have a ...
0
votes
0answers
24 views

Error on a probability with unknown distribution

This question is broken into a couple pieces. I'll go through my approach to puzzling out the question and am looking for some help on places where my logic is just off. I have a biology experiment ...
0
votes
0answers
21 views

Random Samples from Spliced Distribution

I am studying Clauset, Shalizi, and Newman, Power Law Distributions in Empirical Data (preprint available here) in R. Packages used: ...
4
votes
1answer
34 views

Conducting “inference” on Titanic data set (and other non-random/“population-encompassing” data sets alike)

Presume I'm given a data set like Titanic, where the data on all the passengers is available (hence "population-encompassing" in the title). Then, by inertia, I proceed to conduct statistical ...
0
votes
1answer
21 views

What is difference in sampling distribution and sampling from distribution?

Do sampling distribution and sampling from distribution mean the same thing? I am interested in x~N($\mu$, $\sigma$).
0
votes
0answers
11 views

Uneven number of samples across categories for a chi-squared test

I have two categories that I'm trying to preform a chi-squared test on with threat level being the dependent variable and durability as the independent. Categories for threat level include: low, ...
2
votes
1answer
44 views

Hidden-Markov Model for Markov-Chain with Sequential Bernoulli State Sampling

Consider a finite discrete-time Markov chain whose state is sampled at the times determined by the outcome of a Bernoulli process. That is, in each time period I flip a biased coin. If it comes up as "...
0
votes
1answer
30 views

Accept-reject algorithm , why is c>1? [duplicate]

In an accept-reject algorithm, we need to find c such that pj/qj≤ c for all j for which pi > 0 . And, the probability of accepting in any iteration is 1/c. Why is c guaranteed to be more than 1?
0
votes
0answers
21 views

Correction for sampling from infinite population?

I am currently creating a procedure to simulate the generation of data from an unknown, finite population (size $N$). Let's say data generation for a sample of $n = 100$ is done by drawing from a ...
-1
votes
1answer
86 views

Central limit theorem and normal distribution confusion

So I'm trying to study for a test and I'm stuck on two textbook questions. I'm having trouble grasping the concept of sampling distributions and when to apply certain rules/when to categorize a ...