Questions tagged [sampling]

Creating samples from a well-specified population using a probabilistic method and/or producing random numbers from a specified distribution. As this tag is ambiguous, please consider [survey-sampling] for the former and [monte-carlo] or [simulation] for the latter. For questions regarding creating random samples from known distributions, please consider using the [random-generation] tag.

Filter by
Sorted by
Tagged with
2 votes
1 answer
27 views

Sampling a mortality table

I have some mortality table: age_range_0 yearly_probability_0 age_range_1 yearly_probability_1 ... age_range_k yearly_probability_k over_range_k yearly_probability_over For example: [35,45) 0.001 [45-...
Carmen's user avatar
  • 21
0 votes
0 answers
16 views

Calculating true/population standard deviation from bootstrap standard deviation/standard error

I am using the coffee_ratings dataset to do a proof-of-concept calculation to estimate the population (i.e., coffee_ratings) ...
OzkanGelincik's user avatar
0 votes
0 answers
14 views

Sample from a piecewise exponential distribution

Given a distribution $p(x)\propto \exp(\min_{i=1}^N [a_i^Tx + b_i])$, where $x$ and $a_i$ are both D-dimensional vectors, $b_i$ is a scalar, and $(a_i,b_i)$ are known. How can I sample from it? This ...
user avatar
5 votes
1 answer
176 views

Sampling with specified covariance matrix and distribution

Given a positive semi-definite $n\times n$ matrix $C$ I would like to construct $n$ random variables $X_1,\dots,X_n$ drawn from $n$ fixed distributions such that $\mathrm{corr}(X_i,X_j) = C_{ij}$. I ...
deepfloe's user avatar
  • 151
0 votes
1 answer
49 views

permutation test in edgeR

I have a simple RNA-seq experiment with treatment and control, each with 3 biological repeats. I run my data through edgeR and obtained differentially expressed genes (DEGs). Due to the low sample ...
Netanel Cohen's user avatar
7 votes
4 answers
1k views

Is Central Limit Theorem about multiple samples or just one?

I've studied CLT and my understanding is that multiple samples will generate a normal distribution centered in the mean of the population. However, today, one post in Linkedin was saying that "...
Andrew Joplh's user avatar
1 vote
1 answer
63 views

Generate Correlated Bernoulli Samples in Python

Suppose I have $M$ Bernoulli distributions with parameters $p_i$, pairwise correlation $\rho_{ij}$ for $i\neq j$. I would like to generate $N$ samples from the joint distribution. The case of $M=2$ ...
zigs211567's user avatar
0 votes
0 answers
30 views

Sampling from a binomial and get stuck in boundaries

I am trying to use the Metropolis-Hasting in order to obtain a sample for X that is a vector of length N of values that go from 0 to K (in this case K=3). So X ~ Binom(K, p) and p ~ Beta(1,1). For ...
Bibi's user avatar
  • 57
0 votes
0 answers
12 views

Predict quality of a product, from production line, by looking at a sample instead of the whole batch

I have a machine that produces cartons. Every batch that is produced is of 3000 cartons. I want to determine what percentage of cartons are GOOD and what percentage are NOT GOOD ( marks on the carton, ...
Aran's user avatar
  • 1
0 votes
0 answers
6 views

Sample from a population such that it follows a given Dirichlet distribution

I have a store of probability scores (~ around 2 million scores) in the following format - [[0.1,0.2,0.7],[0.4,0.15,0.45], ...] I want to sample from these scores ...
honeybadger's user avatar
  • 1,562
0 votes
1 answer
84 views

Why weighted importance sampling is a biased estimator?

By simple math, we can have $$ E_P[f(X)] = \sum_X f(x)p(x) = \sum_X f(x)\frac{p(x)}{q(x)}q(x) = E_Q[f(X)\frac{P(X)}{Q(X)}], $$ which can be approximated by Monte Carlo sampling in two ways. 1. Normal (...
Fernando Zhu's user avatar
0 votes
0 answers
9 views

Sampling: when is multivariate random variable interpretation dissimilar to repeated realization of single random variable interpretation

After reading many different posts on this site regarding the relationship between random variables and samples, I still have one lingering question (my apologies if I've missed any post explicitly ...
S.C.'s user avatar
  • 71
2 votes
1 answer
15 views

Estimating Marketing Contribution from Opted-In Users

I'm trying to estimate how much marketing contributes to the usage of our product, but because of consent requirements, I can only collect data on a subset of users - the ones who have consented to us ...
Ben's user avatar
  • 21
0 votes
0 answers
27 views

Do I use $P(y_t|y_{1:t-1})$ instead of $P(y_t|z_{t},y_{1:t-1})$ for prediction in a Hidden Markov Model?

I am confused what to sample for getting the prediction $y_{t}$ if I have access to the previous observations $y_{1:t-1}$ and the hidden states $z_{1:t}$. I want to predict the observation at time $t$ ...
wd violet's user avatar
  • 777
2 votes
1 answer
29 views

Biased Sampling from a Non-Normal Dataset

For my analysis, I'm interested in a particular subset from a non-normally distributed population. I would therefore like to generate a sample from that population. The sample will have drastically ...
Linkray's user avatar
  • 21
0 votes
0 answers
117 views

Solutions to downsampling imbalanced time series dataset in time series forecasting for regression model

I have an imbalanced time series dataset for use in a time series forecasting problem for regression (forecast 1 video of 24 hour data (144 7x7 images) given a 1 video of 24 hour data (144 7x7 images))...
Marco's user avatar
  • 51
1 vote
0 answers
46 views

Sampling from a very high dimensional Gaussian

I would like to a sample from a Gaussian $N(0,K)$ where $K$ is a kernel gram matrix, so that $K=[K_{ij}]$ with $K_{ij} = k(x_i,x_j)$ for some positive definite function $k$. The first issue is that ...
WeakLearner's user avatar
  • 1,481
0 votes
0 answers
17 views

How to sample with the 1-norm?

I am currently working on ridge regression, which can be interpreted using Bayesian statistics (DOI: 10.1016/j.electacta.2015.03.123). In particular, I know that the maximum-a-posteriori (MAP) ...
user avatar
0 votes
0 answers
17 views

Sampling in case of imbalanced dataset [duplicate]

From a course of AI for Medical Diagnosis, it is explained that validation and test sets should be balanced, 50-50 cases of both cases 0 and 1, so that the performance of the model can be assessed. ...
Félix Francisco Enríquez Romer's user avatar
1 vote
1 answer
23 views

randomisation issue/crisis

I have randomized into three groups using a randomization aid and groups were examined at 2 time points. however in the analysis found that there was a significant difference at baseline in one of the ...
ketchup28's user avatar
0 votes
0 answers
15 views

Is this a correct way of resampling the MCMC chain?

Please understand I am not familiar to the statistical languages. All I want is to resample a probability distribution from an existing sample drawn from another distribution using MCMC, without ...
Hojin Cho's user avatar
  • 131
1 vote
0 answers
11 views

Can I use a subset defined from characteristics of individuals from original sample as a sample itself?

I have stratified random sample of individuals which represents a large geography. Some of these individuals are self-employed. Can I take subset of these surveyed individuals who are self-employed ...
Kiran Limaye's user avatar
0 votes
1 answer
92 views

How on earth is a random sample representative of the entire population?

Suppose there are 10k people in the population. You want to find out how many of them brush their teeth. You take 1k of them entirely randomly. They all say "yes". But the rest 9k say "...
curiousCprogrammer1231's user avatar
2 votes
1 answer
36 views

Random Sampling from Distribution

I have data of lengths ranging from 25 to 135 and I would like to determine the distribution of this data so that I can randomly generate values from this distribution. My data contains 443 values, ...
Hal's user avatar
  • 21
2 votes
1 answer
51 views

How to get sample size for purposive sampling?

How do I get a sample size if I'm using purposive sampling for my study? It's a sensory evaluation for a food product development but there are exclusion criteria so my adviser suggested to use ...
Junio Princess Ana V Ana's user avatar
2 votes
1 answer
163 views

Can Random Sampling Help Improve Data Quality Problems?

I am working with a hospital dataset. Patients come in and provide information (i.e. var1, var2, var3) about themselves and their partner (if they have one). Later, their partner also come and provide ...
Uk rain troll's user avatar
5 votes
1 answer
490 views

Does the Law of Large Numbers work better for some Distributions? [closed]

Here are two popular principles in Statistics: 1) Law of Large Numbers: If $X$ is a random variable with a probability density function $f(x)$ and an expected value $E[X] = \mu$. If we take a sample ...
Uk rain troll's user avatar
2 votes
1 answer
28 views

Should I normalise my dependent variable by a measured variable that I want to be constant?

This is a question about the validity of applying the results of a statistical test, and unfortunately will require some explanation of the experiment in question. Essentially, I want to measure how ...
Rachel Jones's user avatar
3 votes
3 answers
369 views

Gaps in a histogram in sampling distribution

This histogram shows the sampling distribution of 5000 sample proportions each based on 50 persons. Why are there gaps between the bars in the histogram?
Happy Cretine's user avatar
2 votes
2 answers
73 views

Is R's weighted sample without replacement function misleading?

Background The 2023 article "Remarks on some misconceptions about unequal probability sampling without replacement" by Tillé suggests the sample function ...
LBogaardt's user avatar
  • 582
0 votes
1 answer
47 views

Statistical testing on non-random sample?

I am working with a non-random sample in an observational study but would like to do statistical testing to show certain trends and processes. Statistical tests assume random sampling, which is not ...
manaTEA's user avatar
  • 103
2 votes
1 answer
78 views

In stratified sampling, why is the stratum population variance obtained by dividing by 1 less than the stratum size

I am aware that flavors of this question get asked a lot, for e.g., here. I am fine with the sample variance being divided by $n-1$ and that is what makes it an unbiased estimator of the population ...
Tryer's user avatar
  • 275
0 votes
0 answers
13 views

Sampling for model validation against a full dataset

I have two models for estimating impact of some intervention on some metric. They take some sample as an input and return a single value. Full sample has 170k observations, which are a representative ...
Jean Broc's user avatar
1 vote
1 answer
65 views

Sample many correlated random matrices all with the same pairwise correlation coefficient

I am looking to generate $K$ different correlated random matrices, of which the elements all have the same pairwise correlation coefficient. That is, let $A_1, A_2, \ldots, A_K$ be $N \times N$ random ...
Johannes Nauta's user avatar
1 vote
1 answer
51 views

Input-Output Correlations using PCA

I want to understand the correlation between a set of $\color{red}{\textrm{inputs}}$ and a set of $\color{blue}{\textrm{outputs}}$ deriving from numerical simulations, and possibly reduce dimensions ...
lukewarn's user avatar
0 votes
0 answers
16 views

Consider an example with two strata for n=300. Suppose optimum allocation is sample sizes of 200 and 100 in the two strata. The approx sample sizes

Consider an example with two strata for n=300. Suppose the optimum allocation requires sample sizes of 200 and 100 in the two strata. The approximate sample sizes actually used are 180 and 120, ...
Gadin Naidu's user avatar
1 vote
0 answers
14 views

Sampling quantities with a fixed sum ("string cutting"), but those quantities have to be discrete

I would like to sample 6 quantities that are guaranteed to add up to 600, each with a mean of 100. I want to be control the amount of variance around 100 (same variance for all 6 quantities, but need ...
Luke Strickland's user avatar
3 votes
4 answers
677 views

What is the difference between sampling error and measurement error?

The terms sampling error and measurement error in statistics field are very relevant and have a paramount importance for data analysis and reporting.
Dr. Subhash Chander's user avatar
2 votes
0 answers
55 views

Power sizing when sampling from a group with known subgroups

I need to do a power analysis of a group that is composed of three subgroups. The measurements to sample are difference measurements between two dogs of the same breed rated side by side. Dogs are ...
Estimate the estimators's user avatar
1 vote
0 answers
92 views

About paper "Deep Unsupervised Learning using Nonequilibrium Thermodynamics"

Here is the paper link related to the question from my title. In Appendix B, it computes the entropy of $p(X^T)$ and says "By design, the cross entropy to $\pi(x^t)$ is constant under our ...
user405729's user avatar
0 votes
0 answers
14 views

What weight to use for using NAMCS and NHAMCS together?

I am interested in analyzing the total prescription of aspirin in NAMCS and NHAMCS (https://www.cdc.gov/nchs/ahcd/index.htm) during a given year for all visits. NAMCS and NHAMCS each had a weight and ...
user45765's user avatar
  • 1,416
8 votes
3 answers
260 views

Sampling from $P(x) \propto \cosh^{m}(a x) e^{-x^{2}/2}$

Is there an efficient algorithm to draw samples $x \sim P(x)$ from the PDF: $$ P(x) \propto \cosh^{m}(a x) e^{-x^{2}/2} $$ where $a\ge0$ is a real parameter, and $m$ a positive integer? Since this is ...
a06e's user avatar
  • 4,420
0 votes
0 answers
21 views

Aggregating daily sampling evidence into a monthly estimate

Suppose there's a business process that's sampled daily for quality review purposes, i.e., a population of items is generated in daily batches, and randomly sampled every day to identify production ...
st1led's user avatar
  • 242
2 votes
1 answer
74 views

Sequential random sampling

This question is from Rice's Mathematical Statistics and Data Analysis. (I'm not a student, so you don't need to worry about helping me "cheat" by answering this!) Suppose you have a ...
Adam Cataldo's user avatar
2 votes
0 answers
35 views

How to estimate the age of players correctly?

I have the data of players active on a gaming console and the playtime hours corresponding to the games they have played and their age. I want to analyze the top (say 10) games that the people between ...
Ritik P. Nayak's user avatar
1 vote
0 answers
22 views

The AFT and PH model

According to the following - https://dylanspicker.com/courses/STAT437/content/041%20+%20044.%20Lecture%20Notes.pdf the PH and AFT models are equivalent for Weibull distribution. Does this implies the ...
Kozolovska's user avatar
  • 1,355
1 vote
0 answers
30 views

Sampling inquiry for thesis [closed]

I have a mixed-method thesis ongoing and I plan collecting data on my own college (namely college X), specifically from students and faculty members on my department. Evidently, that would be ...
Kenzo's user avatar
  • 41
2 votes
2 answers
74 views

Estimate number of bad actors

This might be a silly question, but it's got me stumped. I am trying to estimate the number of bad actors in a system. Let's suppose we have 100 users, and some percentage of them are bad actors. In ...
Michael Petrochuk's user avatar
1 vote
0 answers
27 views

Sample Random Effects from Mixed-effects Model

I have a mixed effects model that I'd like to simulate random effects from using a MVN approximation (sampling from the predictive distribution). My question is what is the advantage of simulating ...
user13317's user avatar
  • 675
2 votes
1 answer
44 views

Monte carlo estimator using samples from normal distribution

I am following the Physically Based Rendering: From Theory To Implementation book. In the exercises from Chapter 2 (Monte carlo integration) they ask to write a program to compute numerical ...
ElPotac's user avatar
  • 51

1
2 3 4 5
65