Questions tagged [sampling]

Creating samples from a well-specified population using a probabilistic method and/or producing random numbers from a specified distribution. As this tag is ambiguous, please consider [survey-sampling] for the former and [monte-carlo] or [simulation] for the latter. For questions regarding creating random samples from known distributions, please consider using the [random-generation] tag.

Filter by
Sorted by
Tagged with
3
votes
1answer
43 views

Difference between Sampling a population Vs Bootstrapping

I am finding it difficult to understand the concept of Bootstrapping in statistics . I know what sampling is , that is , taking a 'sample_size' number of observations from a population to estimate ...
0
votes
1answer
45 views

What is the Sampling Design and how to take sampling weights?

I wan to know what is is the sampling design here (situation is also depicted in the figure below) City is divided into urban and rural domains Each domain is divided into blocks Each block contains ...
0
votes
1answer
37 views

Cross Validation in an Imbalanced data set [duplicate]

What is the point of oversampling an imbalanced data set if the ratio of the classes needs to be preserved in Cross validation ? If I have 1000 rows in a data set where 800 rows belong to one class ...
1
vote
1answer
43 views

Why check for normality of data in a sample?

from what I understand the assumption of normality (that must be assumed if one wants to use parametric tests) refers to the sampling distribution of the mean. It does not imply that the distriubtion ...
2
votes
1answer
37 views

Are random sampling with replacement methods (like bootstrapping) really representative of the population?

In many statistics courses, bootstrapping (and other random sampling with replacement methods) are suggested as ways to improve the confidence level in a statistic and improve our inference. Some even ...
0
votes
0answers
18 views

what is the best sampling procedure to use? (stratified)

all, i am a little confused on what sampling method to use. i am doing a classification problem where i am trying to classify a person as having cancer or not. there is apparently huge variance in ...
3
votes
1answer
145 views

Metropolis-Hastings: target distribution with two modes; deterministic transformation

I'm trying to construct a Metropolis-Hastings algorithm to sample a target distribution $p(x)$ with two different and isolated modes. The example I'm working with is \begin{equation} p(x) = \frac{\...
0
votes
0answers
23 views

Interpretation of sampling distribution as the main distinction between Bayesian and classical statistics (Leamer)

In Hendry et al. (1990) p. 187-188, Edward Leamer says: To me the essential difference between the Bayesian and a classical point of view is not that the parameters are treated as random variables, ...
0
votes
0answers
54 views

Uniformly distributed VAE samples

I am currently working on a VAE to generate images (for simplicity MNIST). If I understand the theory correctly, the latent variables follow a gaussian normal distribution in the dimensions of the ...
0
votes
0answers
9 views

Finding a representative sample with the same mean/distribution of a numeric variable

I'm trying to find a representative sample from a population of about 3100. I checked out sample-size calculators and a sample of about 1,000 seems to be the size I want for confidence/margin of error....
11
votes
2answers
443 views

Central Limit Theorem - Rule of thumb for repeated sampling

My question was inspired by this post which concerns some of the myths and misunderstandings surrounding the Central Limit Theorem. I was asked a question by a colleague once and I couldn't offer an ...
0
votes
0answers
16 views

Math support for implementation of weighted random sampling

I read a paper introducing an implementation of WRS(weighted random sampling) https://utopia.duth.gr/~pefraimi/research/data/2007EncOfAlg.pdf However I was struggling to understand if algorithm A is ...
0
votes
1answer
14 views

How to simulate random sample over large set by randomly sampling non-overlapping subsets

I've got a large corpus of video, and I want to randomly sample frames over the whole corpus. A naive way to do this is convert all videos to a giant set of images with ...
3
votes
1answer
28 views

Terminology: Data sample vs statistical sample

I am writing an eye tracking paper that involve both "samples" as used in signal theory (e.g "the sampling rate was of 50 samples per second) and statistical samples. A reviewer is ...
0
votes
0answers
8 views

Multi-categorical set balancing

I have a particular mathematical problem that I would name as multi-categorical set balancing. I don't think this is a new problem but I do not know the correct term for it, therefore I am also ...
0
votes
1answer
27 views

Sampling with python statsmodels ARIMA package [closed]

Assume I have a model following ARIMA(p,q,d) with statsmodels package of python. Given a time series given by a numpy array "serie", the code looks like: ...
5
votes
2answers
96 views

Optimization as sampling for stochastic functions

Given an input space $X$ and a function $f: X\rightarrow \mathbb R$, we want to find $x^*=argmin_{x\in X} f(x)$. One way is to cast this problem as a sampling, where we define a distribution $p(x)\...
0
votes
1answer
25 views

What is meant by “hypothetical” repetitions of a sampling experiment?

given this definition of standard error: "the standard error quantifies how much an estimator varies in hypothetical repetition of a sampling experiment" I just wanted to clarify exactly ...
0
votes
0answers
12 views

Dealing with an imbalanced dataset in text mining

As an English major with no traditional training in statistics, I am having a very rough time with this, so any help would be greatly appreciated. My problem is that only 849 books out of my 6360 book ...
0
votes
1answer
17 views

How can I calculate sample size for case control study?" [closed]

Please, I want to understand how to calculate the sample size the formula for the project entiled, "Evaluation of renal function in type2 diabetes mellitus patients attending federal medical Like&...
0
votes
0answers
12 views

How to choose the number of samples to sample from an unbalanced dataset?

I have two unbalanced dataset with binary classes. One dataset has $13000$ samples for class 1 and $14000$ samples for class 2. Another dataset has $20000$ samples for class 1 and $40000$ samples for ...
0
votes
0answers
3 views

How can I estimate a percentage in a two-stage sample design?

I'm trying to solve a exercise that I have to do. In this, there is a two-stage sampling design. The first stage is a Poisson design and in the second stage is an IS. The problem that proposes the ...
6
votes
3answers
208 views

Expectation of sample averages from normal distribution

Let $n\geq 2$ i.i.d. normally distributed variables $s_i\sim\mathcal{N}\left(0,\sigma^2\right)$, with $i\in\left\{1,2,\dots,n\right\}$. I draw two samples of $k<n$ variables, without replacement. ...
0
votes
0answers
27 views

Can you use the isolation forest algorithm on large sample sizes?

The original isolation forest paper states that the algorithm works best on small subsamples, but is it okay to use it on large sample sizes or are other anomaly detection algorithms better?
0
votes
1answer
21 views

Is it possible to estimate the accuracy with increasing sample size?

I've just done an experiment where I sample 10% of a population and note the results of each sample. Without much context I do KNOW that the true mean of this population parameter is 50.06%. After ...
2
votes
0answers
22 views

Ways to sample from a distribution that are more efficient than random

I am trying to sample from a known distribution (somewhat complicated in that a transformed random variable has random noise from a scale mixture of normals added to it and is then back-transformed - ...
1
vote
0answers
38 views

Can you use the isolation forest algorithm on a large sample size?

I've been using the scikit learn sklearn.ensemble.IsolationForest implementation of the isolation forest to detect anomalies in my datasets that range from 100s of ...
0
votes
0answers
22 views

python gibbs sampler for bivariate normal distribution, failing to converge

I've been trying to understand Gibbs sampling for some time. Recently, I saw a video that made a good deal of sense. https://www.youtube.com/watch?v=a_08GKWHFWo The author used Gibbs sampling to ...
8
votes
3answers
181 views

Clarification on the definition of “population”

In stats classes, I've always learned that a population is always a very broad, almost unquantifiable group (e.g., all voters in a country, all consumers of a company, all viewers of a TV channel), ...
0
votes
1answer
34 views

Sampling from a continuous 2 dimensional probability distribution function for importance sampling

I just want to clarify a few points with regarding to sampling from a continuous 2-dimensional probability density function. If I want to sample from this pdf, I could sample from a 1D pdf, $P(x)$, ...
1
vote
0answers
5 views

Determining variance due to measurement errors between two data sets of the same population

I have a set of n objects, and for each object i, the property C(i) has been measured. The measuring device has low precision, so the value actually recorded from the measurement is C(i) + X(i), where ...
1
vote
1answer
39 views

What does standard error of the mean ACTUALLY show?

I'm brushing up on my stats, so please bare with me (and correct me) for any mistakes. I really hope someone can help me out! Let's consider two separate experiments that are designed to measure the ...
1
vote
1answer
25 views

Estimating Fourier spectrum from multiple time series of a system

I have a set of N time series, each of length T, that describe separate realisations of a single physical system. For each series, I can compute an FFT to find the Fourier spectrum up to a period 2/T, ...
0
votes
0answers
22 views

Empirical conditional density of continuous variables

I have a dataframe, with data of several continuous variables. The variables are not independent. My goal is to sample from the distribution that generated this data. What's a relatively easy and ...
0
votes
1answer
16 views

Model Error estimation

Assume we have some credit model, which has calibrated (up to some error) score of how credit-worthy any individual person is. For example, if the model’s estimate is 83% then we can assume the ...
4
votes
2answers
46 views

What is the relationship between Boltzmann / Gibbs sampling and the softmax function?

I'm looking at sampling functions in the context of reinforcement learning; specifically the explore/exploit problem. A method I've seen pretty often is to derive the action by assigning a score to ...
0
votes
1answer
45 views

Can I use the sample mean to check whether non-normally distributed populations are different?

I've run a user-test of a tool where 86 users participated and scored the a according to the System Usability Scale. The scale goes from 0-100, with 0 being the worst and 100 being the best. The ...
0
votes
0answers
15 views

How to sample biased distribution from another distribution?

I have a PSNR histogram distribution for 6k images and I want to sample a few images from this distribution. But as you can see the distribution is right-skewed and the samples will less PSNR values ...
2
votes
2answers
31 views

Accept-reject and subsets of iid samples

I have some confusion about subsets of iid samples being distributed as the original sample. As an illustration, consider the accept-reject algorithm to produce iid samples from a pdf $f(x)$. We draw,...
0
votes
0answers
10 views

Mapping changed in synthetic sampling (SMOTE)

I have an imbalanced class dataset (0.3%-99.7%), I have used SMOTE (https://arxiv.org/pdf/1106.1813.pdf) to create a proportion of 30%-70%. But in the SMOTE dataset my product-segment mapping seems ...
1
vote
1answer
38 views

Rejection sampling - total probability of acceptance [closed]

I am given the following pdf $$f(x)=3 x^{2}, \quad 0 \leq x \leq 1$$ which i need to simulate by using rejection sampling. I have used the following code below in R. ...
2
votes
0answers
20 views

Sampling from deep belief networks

DBNs are generative models, and usually you sample by thermalising the deepest layer (as it's a restricted Boltzmann Machine), and then forward propagating a sample towards the visible layer to get a ...
1
vote
1answer
13 views

Simple preprocessing large columns set

I have a huge dataset about the effect of the drug on cancerous cell lines with a 17k column. I need to prepare a simple regression, but I don't know, how to pick the most important columns. I ...
-1
votes
1answer
51 views

What do we commonly call a Sampler ? and the link between MonteCarlo, Metropolis-Hasting method, MCMC method and Fisher formalism

Sorry to ask multiples questions but they are all related to the same problematic. I would like to get explanations/clarifications as much as possible since I am going to use all these methods in the ...
0
votes
0answers
39 views

Additivity of sample (rather than population) variances

I'm trying to use the fact that variances are additive to derive the variance of an unknown random variable. Say that A and B are both independent normally distributed random variables. A third ...
0
votes
0answers
9 views

How to partition a sample into representative subsamples?

The problem is the following: take a sample $X$ of the general population $\Omega$, whose distribution is known. Each element of $X$ is described by a vector of characteristics, each characteristics ...
1
vote
1answer
30 views

Number of samples needed to estimate population mean to given margin of error

The population distribution is unknown, but is probably multi-modal. The required margin of error is given in advance. Any number of samples can be drawn from the population. Given enough samples, ...
0
votes
0answers
4 views

selecting number of folds in an unevenly distributed small dataset

I'm working on a facial expression dataset with 7 classes with the following sample distribution. I'd like to perform a k-fold but some classes have less than 10% number of samples than the largest ...
1
vote
0answers
5 views

Making a Valid Analysis with a Convenience Sample

I am interested in doing an analysis on the thickness of a particular coin. There are inexpensive forgeries of the coin, and the market is flooded with these forgeries, but one can occasionally find ...
0
votes
1answer
72 views

Inverse transform sampling

I have the following P.D.F function: $$g(x)=4 \cdot 38^{4} x^{-5}, \quad x \geq 38$$ By taking the inverse of the CDF i get: $$G^{-1}(u)=\left\{\begin{array}{ll} \sqrt[4]{-38/u}, & \text { if } ...

1
2
3 4 5
50