# Questions tagged [sampling]

Creating samples from a well-specified population using a probabilistic method and/or producing random numbers from a specified distribution. As this tag is ambiguous, please consider [survey-sampling] for the former and [monte-carlo] or [simulation] for the latter. For questions regarding creating random samples from known distributions, please consider using the [random-generation] tag.

3,247 questions
Filter by
Sorted by
Tagged with
1 vote
5 views

### ABC (Approximate Bayesian Computation) Sampling, Simulating data from Complex models

In ABC sampling methods, Rejection, MCMC and SMC, when we sample potential parameter values from the prior/proposal, we then use those parameters on our model and simulate data values. This can be ...
14 views

### Sample from a piecewise exponential distribution

Given a distribution $p(x)\propto \exp(\min_{i=1}^N [a_i^Tx + b_i])$, where $x$ and $a_i$ are both D-dimensional vectors, $b_i$ is a scalar, and $(a_i,b_i)$ are known. How can I sample from it? This ...
1 vote
63 views

### Generate Correlated Bernoulli Samples in Python

Suppose I have $M$ Bernoulli distributions with parameters $p_i$, pairwise correlation $\rho_{ij}$ for $i\neq j$. I would like to generate $N$ samples from the joint distribution. The case of $M=2$ ...
27 views

### Sampling a mortality table

I have some mortality table: age_range_0 yearly_probability_0 age_range_1 yearly_probability_1 ... age_range_k yearly_probability_k over_range_k yearly_probability_over For example: [35,45) 0.001 [45-...
16 views

### Calculating true/population standard deviation from bootstrap standard deviation/standard error

I am using the coffee_ratings dataset to do a proof-of-concept calculation to estimate the population (i.e., coffee_ratings) ...
234 views

### Gibbs Sampling vs. Using Raw Probability in Contrastive Divergence

In Hinton's Practical Guide to Training Restricted Boltzmann Machines, Section 3, he discusses different situations in which one should take a sample from the Gibbs sampling process, and other ...
176 views

### Sampling with specified covariance matrix and distribution

Given a positive semi-definite $n\times n$ matrix $C$ I would like to construct $n$ random variables $X_1,\dots,X_n$ drawn from $n$ fixed distributions such that $\mathrm{corr}(X_i,X_j) = C_{ij}$. I ...
1k views

### Is Central Limit Theorem about multiple samples or just one?

I've studied CLT and my understanding is that multiple samples will generate a normal distribution centered in the mean of the population. However, today, one post in Linkedin was saying that "...
49 views

### permutation test in edgeR

I have a simple RNA-seq experiment with treatment and control, each with 3 biological repeats. I run my data through edgeR and obtained differentially expressed genes (DEGs). Due to the low sample ...
30 views

### Sampling from a binomial and get stuck in boundaries

I am trying to use the Metropolis-Hasting in order to obtain a sample for X that is a vector of length N of values that go from 0 to K (in this case K=3). So X ~ Binom(K, p) and p ~ Beta(1,1). For ...
12 views

### Predict quality of a product, from production line, by looking at a sample instead of the whole batch

I have a machine that produces cartons. Every batch that is produced is of 3000 cartons. I want to determine what percentage of cartons are GOOD and what percentage are NOT GOOD ( marks on the carton, ...
84 views

### Why weighted importance sampling is a biased estimator?

By simple math, we can have $$E_P[f(X)] = \sum_X f(x)p(x) = \sum_X f(x)\frac{p(x)}{q(x)}q(x) = E_Q[f(X)\frac{P(X)}{Q(X)}],$$ which can be approximated by Monte Carlo sampling in two ways. 1. Normal (...
6 views

### Sample from a population such that it follows a given Dirichlet distribution

I have a store of probability scores (~ around 2 million scores) in the following format - [[0.1,0.2,0.7],[0.4,0.15,0.45], ...] I want to sample from these scores ...
334 views

### Two-sample t-test between coefficient from regression on two different samples

I am trying to perform a two sample t-test on the regression coefficients for event dummies between two portfolio's (portfolio 1: Top portfolio and portfolio 2: Bad portfolio) The Do file I wrote for ...
15 views

### Estimating Marketing Contribution from Opted-In Users

I'm trying to estimate how much marketing contributes to the usage of our product, but because of consent requirements, I can only collect data on a subset of users - the ones who have consented to us ...
9 views

### Sampling: when is multivariate random variable interpretation dissimilar to repeated realization of single random variable interpretation

After reading many different posts on this site regarding the relationship between random variables and samples, I still have one lingering question (my apologies if I've missed any post explicitly ...
117 views

### Solutions to downsampling imbalanced time series dataset in time series forecasting for regression model

I have an imbalanced time series dataset for use in a time series forecasting problem for regression (forecast 1 video of 24 hour data (144 7x7 images) given a 1 video of 24 hour data (144 7x7 images))...
27 views

### Do I use $P(y_t|y_{1:t-1})$ instead of $P(y_t|z_{t},y_{1:t-1})$ for prediction in a Hidden Markov Model?

I am confused what to sample for getting the prediction $y_{t}$ if I have access to the previous observations $y_{1:t-1}$ and the hidden states $z_{1:t}$. I want to predict the observation at time $t$ ...
121 views

### Generate Gamma distributed values with upper bound

I need to generate N random numbers from a Gamma distribution, but with an upper bound Pmax using Matlab. Right now, I see two ...
29 views

### Biased Sampling from a Non-Normal Dataset

For my analysis, I'm interested in a particular subset from a non-normally distributed population. I would therefore like to generate a sample from that population. The sample will have drastically ...
1 vote
292 views

### Compare two datasets and whether they agree

I have two datasets and they both have the same set of independent variables: 9 of them are on scale from 0 till 100 3 of them are categorical(1 with two types categories, 1 with three types of ...
144 views

### Metropolis-Hastings to sample from dependent random variables

Imagine the goal is sampling from $p(X,Y)$ and X and Y are dependent real-valued random variables, i.e. $p(X|Y)\neq p(X)$. Now the question is how can we apply Metropolis-Hastings algorithm on the ...
17 views

### How to sample with the 1-norm?

I am currently working on ridge regression, which can be interpreted using Bayesian statistics (DOI: 10.1016/j.electacta.2015.03.123). In particular, I know that the maximum-a-posteriori (MAP) ...
1 vote
46 views

### Sampling from a very high dimensional Gaussian

I would like to a sample from a Gaussian $N(0,K)$ where $K$ is a kernel gram matrix, so that $K=[K_{ij}]$ with $K_{ij} = k(x_i,x_j)$ for some positive definite function $k$. The first issue is that ...
1 vote
252 views

### Why can stratified sampling to testing/training sets on strata that contain less than 10% of the entire dataset be statistically risky?

I'm trying to split my data into a testing and a training set. There are lots of variables that I want to ensure are well represented in both the training and testing sets (say, 15 covariates). But ...
17 views

### Sampling in case of imbalanced dataset [duplicate]

From a course of AI for Medical Diagnosis, it is explained that validation and test sets should be balanced, 50-50 cases of both cases 0 and 1, so that the performance of the model can be assessed. ...
1 vote
23 views

### randomisation issue/crisis

I have randomized into three groups using a randomization aid and groups were examined at 2 time points. however in the analysis found that there was a significant difference at baseline in one of the ...
1k views

### Using SMOTE with grouped, paneled, or categorical data?

Let's say that I am building a classifier on imbalanced data. A sample of the data set looks like: ...
I was wondering if it might be possible to generate 2 correlated $Beta$ random variables? In other words, I want to generate two Beta random variables which can be said to have come from two Beta ...