Questions tagged [sampling]
Creating samples from a well-specified population using a probabilistic method and/or producing random numbers from a specified distribution. As this tag is ambiguous, please consider [survey-sampling] for the former and [monte-carlo] or [simulation] for the latter. For questions regarding creating random samples from known distributions, please consider using the [random-generation] tag.
3,250
questions
0
votes
0
answers
14
views
Is it possible to estimate the number of positives from precision and recall values?
Let's say, I have a binary predictor, and its performance in precision and recall is known from the previous study. Now, we apply the predictor on the new (unknown) dataset with 1000 samples, and got ...
1
vote
0
answers
19
views
Sample a random subgraph from an undirected, unweighted graph, what's the probability of "every two nodes's distance is at least 3 in the subgraph"?
This may be a problem in sampling theory or graph theory. I have done many research but I still didn't find valid solutions.
I know a simple random sample is representative of the population. Now I ...
-1
votes
0
answers
5
views
How do conduct a sample size for qualitative and quantitative, if the population is not known [closed]
1.4. Research Question
How does peer support programs on the sexual health outcomes of AGYW living with HIV, impact and influence their knowledge, attitudes, and beliefs related to HIV prevention and ...
3
votes
1
answer
26
views
ABC (Approximate Bayesian Computation) Sampling, Simulating data from Complex models
In ABC sampling methods, Rejection, MCMC and SMC, when we sample potential parameter values from the prior/proposal, we then use those parameters on our model and simulate data values. This can be ...
2
votes
1
answer
38
views
Sampling a mortality table
I have some mortality table:
age_range_0 yearly_probability_0
age_range_1 yearly_probability_1
...
age_range_k yearly_probability_k
over_range_k yearly_probability_over
For example:
[35,45) 0.001
[45-...
0
votes
0
answers
16
views
Calculating true/population standard deviation from bootstrap standard deviation/standard error
I am using the coffee_ratings dataset to do a proof-of-concept calculation to estimate the population (i.e., coffee_ratings) ...
0
votes
0
answers
15
views
Sample from a piecewise exponential distribution
Given a distribution $p(x)\propto \exp(\min_{i=1}^N [a_i^Tx + b_i])$, where $x$ and $a_i$ are both D-dimensional vectors, $b_i$ is a scalar, and $(a_i,b_i)$ are known. How can I sample from it? This ...
5
votes
1
answer
179
views
Sampling with specified covariance matrix and distribution
Given a positive semi-definite $n\times n$ matrix $C$ I would like to construct $n$ random variables $X_1,\dots,X_n$ drawn from $n$ fixed distributions such that $\mathrm{corr}(X_i,X_j) = C_{ij}$. I ...
0
votes
1
answer
56
views
permutation test in edgeR
I have a simple RNA-seq experiment with treatment and control, each with 3 biological repeats. I run my data through edgeR and obtained differentially expressed genes (DEGs). Due to the low sample ...
7
votes
4
answers
1k
views
Is Central Limit Theorem about multiple samples or just one?
I've studied CLT and my understanding is that multiple samples will generate a normal distribution centered in the mean of the population. However, today, one post in Linkedin was saying that "...
2
votes
1
answer
68
views
Generate Correlated Bernoulli Samples in Python
Suppose I have $M$ Bernoulli distributions with parameters $p_i$, pairwise correlation $\rho_{ij}$ for $i\neq j$. I would like to generate $N$ samples from the joint distribution. The case of $M=2$ ...
0
votes
0
answers
30
views
Sampling from a binomial and get stuck in boundaries
I am trying to use the Metropolis-Hasting in order to obtain a sample for X that is a vector of length N of values that go from 0 to K (in this case K=3).
So X ~ Binom(K, p) and p ~ Beta(1,1).
For ...
0
votes
0
answers
12
views
Predict quality of a product, from production line, by looking at a sample instead of the whole batch
I have a machine that produces cartons. Every batch that is produced is of 3000 cartons.
I want to determine what percentage of cartons are GOOD and what percentage are NOT GOOD ( marks on the carton, ...
0
votes
0
answers
6
views
Sample from a population such that it follows a given Dirichlet distribution
I have a store of probability scores (~ around 2 million scores) in the following format - [[0.1,0.2,0.7],[0.4,0.15,0.45], ...]
I want to sample from these scores ...
0
votes
1
answer
84
views
Why weighted importance sampling is a biased estimator?
By simple math, we can have
$$
E_P[f(X)] = \sum_X f(x)p(x) = \sum_X f(x)\frac{p(x)}{q(x)}q(x) = E_Q[f(X)\frac{P(X)}{Q(X)}],
$$
which can be approximated by Monte Carlo sampling in two ways.
1. Normal (...
0
votes
0
answers
9
views
Sampling: when is multivariate random variable interpretation dissimilar to repeated realization of single random variable interpretation
After reading many different posts on this site regarding the relationship between random variables and samples, I still have one lingering question (my apologies if I've missed any post explicitly ...
2
votes
1
answer
15
views
Estimating Marketing Contribution from Opted-In Users
I'm trying to estimate how much marketing contributes to the usage of our product, but because of consent requirements, I can only collect data on a subset of users - the ones who have consented to us ...
0
votes
0
answers
27
views
Do I use $P(y_t|y_{1:t-1})$ instead of $P(y_t|z_{t},y_{1:t-1})$ for prediction in a Hidden Markov Model?
I am confused what to sample for getting the prediction $y_{t}$ if I have access to the previous observations $y_{1:t-1}$ and the hidden states $z_{1:t}$.
I want to predict the observation at time $t$ ...
2
votes
1
answer
30
views
Biased Sampling from a Non-Normal Dataset
For my analysis, I'm interested in a particular subset from a non-normally distributed population. I would therefore like to generate a sample from that population. The sample will have drastically ...
0
votes
0
answers
119
views
Solutions to downsampling imbalanced time series dataset in time series forecasting for regression model
I have an imbalanced time series dataset for use in a time series forecasting problem for regression (forecast 1 video of 24 hour data (144 7x7 images) given a 1 video of 24 hour data (144 7x7 images))...
1
vote
0
answers
46
views
Sampling from a very high dimensional Gaussian
I would like to a sample from a Gaussian $N(0,K)$ where $K$ is a kernel gram matrix, so that $K=[K_{ij}]$ with $K_{ij} = k(x_i,x_j)$ for some positive definite function $k$. The first issue is that ...
0
votes
0
answers
17
views
How to sample with the 1-norm?
I am currently working on ridge regression, which can be interpreted using Bayesian statistics (DOI: 10.1016/j.electacta.2015.03.123). In particular, I know that the maximum-a-posteriori (MAP) ...
0
votes
0
answers
17
views
Sampling in case of imbalanced dataset [duplicate]
From a course of AI for Medical Diagnosis, it is explained that validation and test sets should be balanced, 50-50 cases of both cases 0 and 1, so that the performance of the model can be assessed.
...
1
vote
1
answer
23
views
randomisation issue/crisis
I have randomized into three groups using a randomization aid and groups were examined at 2 time points. however in the analysis found that there was a significant difference at baseline in one of the ...
0
votes
0
answers
16
views
Is this a correct way of resampling the MCMC chain?
Please understand I am not familiar to the statistical languages. All I want is to resample a probability distribution from an existing sample drawn from another distribution using MCMC, without ...
1
vote
0
answers
11
views
Can I use a subset defined from characteristics of individuals from original sample as a sample itself?
I have stratified random sample of individuals which represents a large geography. Some of these individuals are self-employed. Can I take subset of these surveyed individuals who are self-employed ...
0
votes
1
answer
92
views
How on earth is a random sample representative of the entire population?
Suppose there are 10k people in the population. You want to find out how many of them brush their teeth. You take 1k of them entirely randomly. They all say "yes". But the rest 9k say "...
2
votes
1
answer
36
views
Random Sampling from Distribution
I have data of lengths ranging from 25 to 135 and I would like to determine the distribution of this data so that I can randomly generate values from this distribution. My data contains 443 values, ...
2
votes
1
answer
55
views
How to get sample size for purposive sampling?
How do I get a sample size if I'm using purposive sampling for my study? It's a sensory evaluation for a food product development but there are exclusion criteria so my adviser suggested to use ...
2
votes
1
answer
163
views
Can Random Sampling Help Improve Data Quality Problems?
I am working with a hospital dataset.
Patients come in and provide information (i.e. var1, var2, var3) about themselves and their partner (if they have one). Later, their partner also come and provide ...
5
votes
1
answer
491
views
Does the Law of Large Numbers work better for some Distributions? [closed]
Here are two popular principles in Statistics:
1) Law of Large Numbers: If $X$ is a random variable with a probability density function $f(x)$ and an expected value $E[X] = \mu$. If we take a sample ...
2
votes
1
answer
28
views
Should I normalise my dependent variable by a measured variable that I want to be constant?
This is a question about the validity of applying the results of a statistical test, and unfortunately will require some explanation of the experiment in question.
Essentially, I want to measure how ...
3
votes
3
answers
376
views
Gaps in a histogram in sampling distribution
This histogram shows the sampling distribution of 5000 sample proportions each based on 50 persons.
Why are there gaps between the bars in the histogram?
2
votes
2
answers
73
views
Is R's weighted sample without replacement function misleading?
Background
The 2023 article "Remarks on some misconceptions about unequal probability sampling without replacement" by Tillé suggests the sample function ...
0
votes
1
answer
49
views
Statistical testing on non-random sample?
I am working with a non-random sample in an observational study but would like to do statistical testing to show certain trends and processes. Statistical tests assume random sampling, which is not ...
2
votes
1
answer
78
views
In stratified sampling, why is the stratum population variance obtained by dividing by 1 less than the stratum size
I am aware that flavors of this question get asked a lot, for e.g., here. I am fine with the sample variance being divided by $n-1$ and that is what makes it an unbiased estimator of the population ...
0
votes
0
answers
13
views
Sampling for model validation against a full dataset
I have two models for estimating impact of some intervention on some metric. They take some sample as an input and return a single value.
Full sample has 170k observations, which are a representative ...
1
vote
1
answer
65
views
Sample many correlated random matrices all with the same pairwise correlation coefficient
I am looking to generate $K$ different correlated random matrices, of which the elements all have the same pairwise correlation coefficient. That is, let $A_1, A_2, \ldots, A_K$ be $N \times N$ random ...
1
vote
1
answer
51
views
Input-Output Correlations using PCA
I want to understand the correlation between a set of $\color{red}{\textrm{inputs}}$ and a set of $\color{blue}{\textrm{outputs}}$ deriving from numerical simulations, and possibly reduce dimensions ...
0
votes
0
answers
16
views
Consider an example with two strata for n=300. Suppose optimum allocation is sample sizes of 200 and 100 in the two strata. The approx sample sizes
Consider an example with two strata for n=300. Suppose the optimum allocation requires sample sizes of 200 and 100 in the two strata. The approximate sample sizes actually used are 180 and 120, ...
1
vote
0
answers
14
views
Sampling quantities with a fixed sum ("string cutting"), but those quantities have to be discrete
I would like to sample 6 quantities that are guaranteed to add up to 600, each with a mean of 100. I want to be control the amount of variance around 100 (same variance for all 6 quantities, but need ...
3
votes
4
answers
679
views
What is the difference between sampling error and measurement error?
The terms sampling error and measurement error in statistics field are very relevant and have a paramount importance for data analysis and reporting.
2
votes
0
answers
55
views
Power sizing when sampling from a group with known subgroups
I need to do a power analysis of a group that is composed of three subgroups.
The measurements to sample are difference measurements between two dogs of the same breed rated side by side. Dogs are ...
1
vote
0
answers
94
views
About paper "Deep Unsupervised Learning using Nonequilibrium Thermodynamics"
Here is the paper link related to the question from my title.
In Appendix B, it computes the entropy of $p(X^T)$ and says "By design, the cross entropy to $\pi(x^t)$ is constant under our ...
0
votes
0
answers
15
views
What weight to use for using NAMCS and NHAMCS together?
I am interested in analyzing the total prescription of aspirin in NAMCS and NHAMCS (https://www.cdc.gov/nchs/ahcd/index.htm) during a given year for all visits. NAMCS and NHAMCS each had a weight and ...
8
votes
3
answers
261
views
Sampling from $P(x) \propto \cosh^{m}(a x) e^{-x^{2}/2}$
Is there an efficient algorithm to draw samples $x \sim P(x)$ from the PDF:
$$
P(x) \propto \cosh^{m}(a x) e^{-x^{2}/2}
$$
where $a\ge0$ is a real parameter, and $m$ a positive integer?
Since this is ...
0
votes
0
answers
21
views
Aggregating daily sampling evidence into a monthly estimate
Suppose there's a business process that's sampled daily for quality review purposes, i.e., a population of items is generated in daily batches, and randomly sampled every day to identify production ...
2
votes
1
answer
74
views
Sequential random sampling
This question is from Rice's Mathematical Statistics and Data Analysis. (I'm not a student, so you don't need to worry about helping me "cheat" by answering this!) Suppose you have a ...
2
votes
0
answers
35
views
How to estimate the age of players correctly?
I have the data of players active on a gaming console and the playtime hours corresponding to the games they have played and their age. I want to analyze the top (say 10) games that the people between ...
1
vote
0
answers
22
views
The AFT and PH model
According to the following - https://dylanspicker.com/courses/STAT437/content/041%20+%20044.%20Lecture%20Notes.pdf
the PH and AFT models are equivalent for Weibull distribution.
Does this implies the ...