# Questions tagged [sampling]

Creating samples from a well-specified population using a probabilistic method and/or producing random numbers from a specified distribution. As this tag is ambiguous, please consider [survey-sampling] for the former and [monte-carlo] or [simulation] for the latter. For questions regarding creating random samples from known distributions, please consider using the [random-generation] tag.

2,941 questions
Filter by
Sorted by
Tagged with
15 views

### how does subsample parameter work in boosting algorithms like xgboost and lightgbm?

From what I know, both of them are sequential learners and only the 1st tree in the sequence gets built on the data and all the following trees that get built are to correct the mistakes of previous ...
31 views

### How to determine the number of strata from stratified sampling? Any rule of thumb?

I have a continuous variable (proxy for difficulty) that I would like to discretize for stratified sampling. My sampling approach is equal allocation (I will choose an equal proportion of samples per ...
24 views

### When can we get unbiased estimate given biased data?

There was a recent "hot take" tweet by Andrej Karpathy (without any comment or clarification from the author): real-world data distribution is ~N(0,1) good dataset is ~U(-2,2) It provoked ...
• 113k
20 views

### When taking a sample of a population with clusters, how do we set a threshold so that the clusters are significant?

For example, say we have a population of about 200,000 datapoints which can be sorted into about 10 clusters. However, performing the clustering process on the entire population is not feasible. Thus ...
• 11
38 views

### How could proving the result of variance of estimator in Latin Hypercube Sampling

Ok i almost freak out of this!!!! i've to doing some presented about this paper (M.D.MCKAY,J.BECKMAN, W.J.CONOVER (1979). It's about comparing methods for selecting values (Random Sampling, Stratifed ...
33 views

### How to propagate uncertainties for parameters whose upper and lower uncertainties are different?

If $A = A_0 \pm \sigma_A$, $B = B_0 \pm \sigma_B$, and $f = \frac{A}{B}$, the normal error propagation goes like (following Wikipedia) \sigma_f = \sqrt{\sigma_A^2 + \sigma_B^2 - 2\...
25 views

### Extrapolating the total number of different molecules (equivalent of marble sampling) [closed]

We have a total material, 100%. We measured 2 samples of it, each 4.7%. We found 1538 different molecules in each sample, 1061 found in both, and 477, 477 found exclusively in either (but not both) ...
• 103
23 views

### Estimate the Numbers of samples required for each variable

I am tasked to help with organizing the driving experiments for a robot. The goal is that for each Terrain Class like forest, wet-meadows, dry-meadows we will measure the time it takes the robot to ...
23 views

• 355
39 views

### Log-normal mean and standard deviation change after sampling

I have the log-normal standard deviation and the mean that I want to use to sample from a log-normal distribution in Python. However after I do the sampling and compute the arithmetic mean and ...
43 views

### Convert from log-normal distribution to normal distribution [duplicate]

For purposes of sampling I have the variance v and mean m of a log-normal distribution, I don't want to directly sample from the log-normal distribution so my question is how can I convert this into a ...
21 views

### the mean and standard deviation aren't the same as those of the input data i provided after sampling

I have a log-normal mean and a standard deviation. after i converted them to the underlying normal distribution's parameters mu and sigma, I sampled from the log-normal distribution however when i ...
52 views

### uniformly sample from gaussian distributed data

I have data that is roughly gaussian distributed, bounded on a range of [x0, x1], w/ mean m and stadard deviation s. I want to ...
• 1,218
26 views

### Inverse transform sampling : comparing bias, variance and mse for an estimator

Starting from the PDF of the Pareto distribution, f_{\theta_1, \theta_2}(x) = \begin{cases} \frac{\theta_1 \theta_2^{\theta_1}}{x^{\theta_1 + 1}}, &\quad x \geq \theta_2 \...
1 vote
33 views

### How to sample and compute the likelihood from a Mollified Uniform distribution?

I want to draw samples from the mollified Uniform distribution presented in another Cross Validated thread, cf the answer from whuber. What is the best way to do so?...
11 views

### Population statistics

Let’s say one has a small random size of 100 people, and the distribution of this small population is 60% male and 40% female. Now, say, there are 1,000,000 people that live in this region in total. ...
• 1,405
18 views

### Estimating total number of consumers based on observed transactions

I read through most other questions of this type and didn't find a relevant situation. I want to predict the total number of consumers and repeat consumers that shopped in the past month, however, ...
61 views

### Showing that $E[\hat{\tau}_D] = P(n_D > 0)\tau_D$ and $\vert E[\hat{\tau}_D] - \tau_D\vert \leq \tau_D(1-\frac{N_D}{N})^n$

Consider the following double sampling scheme: We have a population of size $N$ with variable of interest $y_i$ for each $i \in \{1,\dots,N\}$, and (fixed) subpopulation $D$ of size $N_D$. Let $S$ ...
• 147
15 views

### Sampling population to replicate another population

Disclaimer: This question is quite general and I assume there are multiple answers equally valid, and at this point I have no a priori opinion on what is "similar" population so everything ...
• 281
28 views

### Can you give an example of Metropolis and Metropolis-Hastings algorithm?

I have studied many books and tried to understand both the Metropolis and Metropolis-Hastings algorithm. Everywhere it is written in the context of the Ising model or Lenard-Jones Energy. I am having ...
• 1,518
48 views

### Monte Carlo and function minimization (simulated annealing)

I posted this question on math.stackexchange, but did not get an answer and a limited amount of views, so I removed the question there and post it here. Recently I was asking myself some basic ...
• 663
1 vote
22 views

### Is there a method to select elements negatively proportionate to a categorical distribution?

I have a categorical distribution which depicts the probability of bad choices. I want to select elements negatively proportionate to that categorical distribution, i.e., if my categorical ...
• 199
23 views

### How to choose a sample of a text dataset to label?

I was reading about active learning recently and active learning seems to be used after the first model is generated. So, I was wandering if there are technics to choose what to label before ...
• 163
19 views

### Likelihood A>B, Given Margin of Errors For A and B

Let's say in a video game there are 2 different attacks I want to test, A and B. They have a RNG component so damage varies from one sample to next. Both follow a standard distribution. I test by ...
30 views

### Sampling Puzzle

There is a bag with N = 50 balls. Among which M = 10 balls are red, and N-M = 40 balls are blue. Further, say the red balls are numbered among themselves from 1 to 10, and the blue balls are numbered ...
20 views

### Probability of failure of Uniform Sampling [duplicate]

Say I have a bag with 10 numbered balls, and I pick one ball at each time step and then put it back in the bag. Since each ball is equally likely, therefore the current situation represents a uniform ...
1 vote
35 views

### What is the correct way to sample from a large dataset with several years of data?

I'm a student doing a machine learning project. I'm using the Lending Club Dataset 2007-2018 to predict loan default. The original dataset contains over 2 million rows. I want use a sample from it to ...
16 views

### How to calculate sampling error for proportionate sampling?

I have done sampling using Proportionate Stratified Random Sampling. The table below shows the proportion of each groups in the sample and population. This is the formula for Standard Error (...
• 73
19 views

### How to deal with limited sampling when analyzing a probability distribution

I constructed a probability histogram for my data and wish to construct a log-log plot to look at the distribution, but the sampling (I'm not quite sure how to phrase this) is very poor at the tail ...
• 121
1 vote
16 views

### Is my sample sufficient for PCA?

I’m working on a project for my stats class and I’m wondering if my sample size is sufficient for PCA; I keep seeing mixed recommendations so I figured I’d ask. My sample size is 50 (unit of analysis ...
1 vote
17 views

### Draw a stratified sample in panel data

Question: How can we draw a stratified sample in panel data context when the stratifying variables are not fixed over time? Background: I have panel data on workers (id) over time (t). I want to draw ...
• 253
1 vote
34 views

### How to check if the sample represents the population data?

The below table is similar to the population ...
• 73
93 views

### 8 out of 10 Cats - Optimal Sample

I was watching a lecture earlier by Marcus du Sautoy called: 'Thinking Better with Mathematics.' Marcus discusses the statistical sample sizes to verify statements about the population. In his own ...
• 174