Questions tagged [sampling]

Creating samples from a well-specified population using a probabilistic method and/or producing random numbers from a specified distribution. As this tag is ambiguous, please consider [survey-sampling] for the former and [monte-carlo] or [simulation] for the latter. For questions regarding creating random samples from known distributions, please consider using the [random-generation] tag.

Filter by
Sorted by
Tagged with
0
votes
1answer
41 views

Problems with using Gibbs Sampling for Bayesian DAGs

Assume we want to sample from the variables of Bayesian belief network, which is a Directed Acyclic Graph (DAG), where we observe some of the variables, and do not observe the others. We can usually ...
2
votes
0answers
67 views

Why gradient boosting uses sampling without replacement?

In Random Forest each tree is built selecting a sample with replacement (bootstrap). And I assumed that Gradient Boosting's trees were selected with the same sampling technique. (@BenReiniger ...
1
vote
0answers
31 views

Sampling states of an “unnatural” Hamiltonian System

I would like to sample from a Gibbs distribution given by $$f(p, q) = \frac{1}{\mathcal{Z}}e^{-H(p, q; \omega, J)}$$ where $H$ is the Hamiltonian on generalized coordinates $(p,q)\in \mathbb{R}^{2n}...
5
votes
1answer
137 views

Sampling from dataset according to distribution obtained from another dataset

Suppose we have dataset $A$ with several categorical and numerical features: $A_{cat_1}$, $A_{cat_2}$, $\ldots;$ $A_{num_1}$, $A_{num_2}$, $\ldots;$ Also we have another dataset $B$ with the same ...
5
votes
1answer
225 views

How big should be a sample to be statistically significant?

This is my problem: I need to sample from a random variable that I assume to have a gaussian distribution. I want to estimate the mean and the std of the variable by doing as little sample as possible....
0
votes
3answers
74 views

In spatial data, does unevenly distributed data create model bias?

The scenario: Suppose there is a square shaped region with three cities in it. A dataset is collected by citizens, who collect data where they live. Thus, data is collected much more heavily in those ...
0
votes
0answers
24 views

How to sample in Bayesian inference model?

I have the following problem to model in R. If we have $X_1, ..., X_n \sim N(\theta,1)$ and want to estimate $\theta$, we assume $\theta$ has a prior distribution, and the Bayes estimate of $\theta$ ...
0
votes
0answers
19 views

Chaos Theory, and Logistic Function 𝑋_{𝑛+1}=𝑟⋅𝑋_{𝑛}⋅(1−𝑋_{𝑛}) For Sampling music

This is a curiosity question, and I'd appreciate some insight and sources to delve deeper. The question is about the logistic function and sampling signals. I just saw a video about the logistic map/...
0
votes
0answers
24 views

How to sample from 1 data set to match distribution of variable in another data set?

I have dataset 1 and another dataset 2. All values from dataset 2 are in dataset 1, however, dataset 1 has a variable X that does not match the distribution of data set 2. Is there a way I can sample ...
4
votes
0answers
724 views

Detailed Balance for Hamiltonian Monte Carlo

I am trying to understand the detailed balance proof present in this paper: https://arxiv.org/abs/hep-lat/9208011v2 (page 4). My question: Why do we consider the volume of a neighborhood of points ...
0
votes
0answers
15 views

Stratified Sampling: use and interpret Instance Weight for EDA and modelling

Context I am working on a data mining project (EDA + Predictive Modelling) using the US census income data, which consists of training and test instances obtained using stratified sampling over the ...
2
votes
0answers
37 views

Inverse sampling a direction over the hemisphere of a surface?

Let $S^2:=\{x\in\mathbb R^3:|x|=1\}$ denote the unit 2-sphere, $$\omega_{x\to y}:=\frac{y-x}{|y-x|}\;\;\;\text{for }x,y\in\mathbb R^3\text{ with }x\ne y,$$ $M\subseteq\mathbb{R}^3$ be the disjoint ...
0
votes
0answers
26 views

random of sample of more than you plan to use?

My client is asking me to randomly sample 20 clusters when they only plan to use 10. They are doing this in case something happens and some cannot participate. I'm thinking this is no longer a ...
0
votes
0answers
6 views

How do I set a benchmark for a laboratory measurement from known data?

A colleague is trying to gain formal control of a laboratory process and needs my help. The process involves transferring a liquid into an empty vial using a qualified liquid delivery device (e.g. ...
0
votes
0answers
44 views

Sample conditional multivariate random variable?

There's multivariate random variable, future prices of assets, 5 years from now: $$X = [Gold, Silver, SP500]$$ There's historical prices for $X$ available for last 50 years. It's possible to fit ...
0
votes
0answers
12 views

Does respondent-driven sampling benefit by designing recruitment to encouraging mixing?

In respondent-driven sampling (RDS), recruitment of respondents (aside from the initial seed respondents) is accomplished by incentivizing respondents who have participated in a study to recruit ...
2
votes
0answers
50 views

Inverse transform method with piecewise pdf

I am having trouble using the inverse transform method with the generalized inverse $$F^{-1}(u) = \inf \{x : F(x) \geq u\}$$ In this case, I have a piecewise pdf $$f(x) = \begin{cases}x, & 0 \...
1
vote
0answers
15 views

How to sample from a mixture of densities of transformed random variables?

Suppose we are given a set of $m$ random variables, $X_1$, $X_2$, ..., $X_m$, defined over the same set $\mathcal{X}$, with known densities $p_{X_i}$, for $i=1$, $2$, ..., $m$. Assume that getting ...
2
votes
1answer
62 views

Sample size calculation for correlated count data

I am wondering how I can simulate Poisson data that is correlated. Let's say, I collect data at two time points. At both time points, the data is Poisson distributed, at time point 1 with $\lambda=4$, ...
0
votes
0answers
31 views

Two-stage stratified cluster sampling with “overlapping” PSUs

Let's say I want to do a survey getting a representative sample for an entire population P, as well as groups A and ...
0
votes
0answers
30 views

Sampling big dataset

I am currently running some models on a very large dataset (instacart basket analysis, ~134MM rows) and was wondering on how to pick the right sample size to iterate and evaluate different models ...
4
votes
1answer
32 views

How do I sample a multivariate posterior when I can sample the likelihood and prior?

Suppose I want to sample the posterior distribution of a multivariate $\beta$ given some scalar $x$. By Bayes' theorem, this distribution is $$P(\beta|x) \propto P(x|\beta)P(\beta) $$ I don't have ...
0
votes
1answer
55 views

Parameter Estimation via MCMC

In general, we use MCMC method to sample from a distribution which is hard to compute. In Bayesian setting, we sample from the posterior distribution of the random parameters defining the underlying ...
2
votes
3answers
85 views

Random undersampling: is there a way to chose the best majority samples?

I'm modeling credit fraud, where I have a small number of samples that result in fraud (1), and most samples that are not fraud (0). I am creating a models for detecting fraud based on new data. I'm ...
0
votes
0answers
27 views

Gibbs Sampling for Order Restricted Bayesian Inference

I am trying to learn Gibbs Sampling for Bayesian Inference, I am very confused about the following setting. Suppose we observe $X :=(x_1 , \dots, x_N)$ where $x_i $ is binomially distributed ...
2
votes
2answers
610 views

Can a sample larger than 5,000 data points be tested for normality using shapiro.test by applying the test to a subsample?

I have a number of samples that I would like to test for normality. One of the samples exceeds 5,000 data points, the limit up to which the shapiro test accepts samples. This is the data: ...
1
vote
1answer
61 views

Estimating population defects from a sample size

2 questions: Question #1: I have a lot size of 240 and have drawn out a sample size of 20. Assuming I test the 20 units for go/no-go (i.e. it's binary, pass/fail) and I have Zero failures.....what is ...
3
votes
0answers
25 views

How Does Variance Propagate From Likelihood Function To MCMC Posterior?

Suppose we are trying to obtain the posterior distribution of three parameters that influence a discretely observed population. The likelihood function is unfortunately intractable, as it is a mix of ...
0
votes
1answer
61 views

Standard deviation of sample proportions (standard error) in R

This resource states that the standard deviation of the sampling distribution (the standard error) is equal to: They provide an example where a population has p=0.6 and samples of n=25 are drawn from ...
0
votes
0answers
44 views

how to choose a sample?

I need to review 600 exams with burbles by hand (I have a software to read them all). How many exams do I need to review in order to have 95% of chance of my software is working properly? I think ...
1
vote
1answer
49 views

Problem of unbalanced data [duplicate]

unbalanced data is an issue that can effect the performnce of classification model ,several remides can be done to balance the data two of them are upsampling and downsampling , my questions is : how ...
2
votes
1answer
29 views

probability sampling calculating probabilities

Consider a population with 10 elements , N:{0,1,2,3,4,5,6,7,8,9}. The probability of drawing ...
0
votes
0answers
7 views

Seasonality affects samples on a heterogeneous system?

In a time window of a year, suppose that a social networks has a bimodal normal distribution that peaks at April and October approximately. Being a heterogenous social network, it has different types ...
1
vote
1answer
33 views

Standard deviation in Direct-Sampling

In a computational physics course, I was asked to do direct-sampling for the numerical value of $\pi$ and then I estimated the standard deviation of $\pi$ , ...
0
votes
0answers
17 views

statistic significance of proportions

Let's assume someone buys a bag of smarties, and finds that it includes 22 brown candies 19 red candies 16 yellow candies 15 blue candies 8 green candies How can we use a test of significance to ...
2
votes
0answers
23 views

How is random sampling done in practical terms with a large population (e.g,. people suffering from arthritis in the US)?

my first post here. I'm studying statistics and we are learning about random sampling but I don't understand how what I am learning is applicable to the real world. Are there stat books or articles ...
0
votes
0answers
10 views

Multi-arm RCT Sample Size External and Internal valid

I am trying to figure out the sample size for a multi-arm Randomized Control Trial (non-clinical) that will have internal validity and representative of the population, say; population X Any ...
0
votes
0answers
32 views

Is that possible that I use an arbitrary prior distribution that is not conjugate to the likelihood and do mcmc sampling?

In Bayesian Statistics, we can use conjugate priors to the likelihood functions, then we can get the posterior distributions which are the same distributions as the prior distributions. I wonder ...
0
votes
0answers
23 views

Gains on test data set higher than that on training data set post balancing

I have an imbalanced data set (96-4 split between No and Yes cases). I am trying to build a decision tree model for classification after balancing my data set(tried different thresholds for ...
0
votes
0answers
46 views

Correctly Specifying C=0 Sampling Plan

I looking for a little help with how to go about specifying/requiring the use of a C=0 sampling plan on an engineering drawing. In the past, we have typically been able to refer to a standard, for ...
4
votes
1answer
396 views

Random Walk Metropolis Hastings implementation in R using log scale

Context I looked literally everywhere but I couldn't find a complete implementation of the Random Walk Metropolis-Hastings algorithm using the log scale. By log scale I mean that we are working with ...
0
votes
0answers
15 views

sample size advice for matching nominal variables

I have two databases of patients and medications. Each database has the same patients. Both have millions of records. My job is to select a sample from each and confirm that the medications match. ...
1
vote
1answer
47 views

Sampling from unknown probability distribution [duplicate]

I'm reading about Monte Carlo methods. Suppose that $X_1,...,X_n$ are i.i.d $p(x_i|\theta)$, where $\theta$ is an unknown parameter of interest. My textbook states: Suppose we could sample some number ...
0
votes
0answers
41 views

Binomial distibution with mean 0

I give the following context. Test hypothesis $H_0:p=p_o$ for Bernoulli distributed random variable $X$, with parameter $p$ and samples $X_i$. For estimate $\hat{p}=\frac{1}{n}\sum_{k=1}^n X_i$, $n\...
0
votes
0answers
24 views

Use of the inversion method in sequential sampling to “invert” a random walk

Let $M\subseteq\mathbb R^3$ be Borel measurable, $\lambda$ be a $\sigma$-finite measure on $\mathcal B(M)$, $k\in\mathbb N$, $I:=\{0,\ldots,k\}$, $q$ be a probability density on $\left(E^I,{\mathcal E}...
3
votes
1answer
31 views

Sampling from conditional multivariate normal distribution (“conditioned on ellipses” / stratified sampling)

I have an issue related to stratified sampling. To explain, I need to start with one-dimensional case: Let $N\sim \mathcal{N}(0,1)$, i.e., a standard normal variable with cdf $F$. Partition $\mathbb{R}...
0
votes
1answer
12 views

Sampling small set of indices from a very large set

I have a set containing 100 millions of indices. In each iteration, I choose $k$ sub-samples from this set, and once I select the $k$-samples, I increase or decrease the probability for the selected ...
0
votes
0answers
46 views

Can we sample from the wrapped normal distribution and evaluate the density of the sample simultaneously?

In a computer program (written in C++), given $x\in[0,1)$ and $\sigma>0$, I need to sample $y$ from the wrapped normal distribution $\mathcal W_{x,\:\sigma^2}$ with mean $x$ and variance $\sigma^2$ ...
0
votes
1answer
29 views

Likes/Retweets in Twitter data depends on time of extraction

Currently I scheduled a task to extract twitter data of a week (every sunday) to predict the stock market for the following days. The number of likes of a tweet is not static and changes over time. ...
0
votes
0answers
19 views

How biased is a statistical study in which sampling was purposely made without repeats? [duplicate]

[Cross posted on Math.SE] It is understood in mathematical statistics that a sample (as in sampling distribution) may very well contain repeatedly the same item/subject. In practice though, it would ...

1
3 4
5
6 7
50