Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
Make your voice heard. Take the 2019 Developer Survey now

Questions tagged [sampling]

Creating samples from a well-specified population using a probabilistic method and/or producing random numbers from a specified distribution. As this tag is ambiguous, please consider [survey-sampling] for the former and [monte-carlo] or [simulation] for the latter. For questions regarding creating random samples from known distributions, please consider using the [random-generation] tag.

0
votes
0answers
10 views

Renormalizing a distribution to reduce variance

I have a predictive model $M$ that generates an empirical predictive distribution $P_M$ via a set of samples. I cannot change the predictive model. I can evaluate the predictive performance using ...
0
votes
0answers
14 views

Should I use statistical inference on this “sample”?

The dataset gets its data from thousands of individuals throughout the US who update the same spreadsheet of about 5000 rows. This dataset contains address for individuals and is updated by the ...
0
votes
0answers
10 views

How to generate samples without replacement using mvrnorm in R? [on hold]

I'm using the mvrnorm function in R to generate positively correlated bivariate data with specified means, variances and correlation coefficient. I'm generating 50,...
0
votes
0answers
10 views

How to calculate the mean of a sub sample, given the mean of the super sample and the standard deviation of the population?

I need to run a simulation of cash flows for a project. We are selling a service. The service comes with a range of options. Depending on the specific options chosen, the cost of the service can ...
0
votes
0answers
11 views

Type I and II errors accounted for by one equation?

On this website: https://clincalc.com/stats/samplesize.aspx both type I and II errors are accounted for by the one equation. Surely, because a type I error involves too much positive variance and a ...
-1
votes
0answers
28 views

What is the Population? [closed]

So, in AP Statistics, we had the below question (we are in the sampling distributions chapter), which he decided to spring on us in a graded quiz with no previous examples similar to the below problem:...
-1
votes
0answers
13 views

Average of sample covariances [duplicate]

I have K datasets, each with N variables and M samples and assume they are coming from the same multivariate normal distribution. I am interested in estimating the covariance matrix. Now it can be ...
6
votes
1answer
233 views

How to generate samples of Poisson-Lognormal distribution

I would like to compute samples of the number of product purchased in a supermarket. I want to model it with a mixed Poisson lognormal distribution. Items purchased $x$ of a given consumer follow a ...
0
votes
0answers
3 views

probability sampling for retrospective chart review RCR/Medical Record Review

how best can you achieve probability sampling in a medical record review. In answering question, please use a real example. Say you have 4000 pregnancies to review, how many of these do you need to ...
1
vote
2answers
22 views

Evaluating proposed under-sampling method

I am currently working on an under-sampling procedure to tackle problems that arise when training and test distributions are different. Does the following experiment set-up enable performance ...
0
votes
1answer
12 views

Do pollsters use independent samples?

Is it a common practice for political pollsters to use random independent samples to conduct their polls?
0
votes
0answers
15 views

How to choose my next sample point in a 2D boundary problem?

I've got a real-world problem that I'm trying to solve with as few computations as possible. In this 2D problem, everything is parameterized on a unit square, $f(x,y); x, y \in [0, 1]$. Anecdotal ...
1
vote
1answer
26 views

I want to simulate a random sample of length n from DAG of correlated Bernoulli's

Suppose I have a DAG of 4 vertices. Each vertex consists of a Bernoulli of parameter $p$. It is the following: (Z) ---> (Y) (Z) ---> (W) (X) ---> (Y) ---> (W) I hope it is clear. Anyway, I ...
0
votes
0answers
21 views

combine samples of supersets to get a reservoir sample of the subset

Here's some background before my question (Bear with me! :D) I am working on a data pipeline that deals with a massive input log dataset; instead of saving the full input log, current version of this ...
0
votes
0answers
14 views

Calculating probability of voting results from small sample aize

2 million people voted in a poll for their favourite song. 35,000 votes have been counted so far, of which: 575 votes are for Song A 466 votes are for Song B 393 votes are for Song C (And the ...
1
vote
0answers
15 views

Using MAP inference in MRFs to estimate expectations

This question pertains to MRFs of the form $$ p(y | \theta) = \frac{1}{Z(\theta)} \exp \left( \sum_c \theta_c^\top \phi_c(y) \right) $$ with notation and nomenclature taken from [1]. Suppose that ...
1
vote
0answers
25 views

How to sample from a multivariate uniform distribution with constrained sum [closed]

Problem description: Let say that we have the following 4 uniform distributions: $u_1$~$U_1(-1, 1), u_2$~$U_2(0, 2)$, $u_3$~$U_3(0, 0.2)$ and $u$~$U[U_1, U_2, U_3]$ subject to $u_1 + u_2 + u_3 = 1$. ...
0
votes
0answers
32 views

Multiple correlation in the sampling variance of slope coefficient in multiple regression

Let's say we have the following multiple regression model, $$Y_i = \alpha + \beta_1 x_{i1} +\beta_2x_{i2} + ... + \beta_kx_{ik} + \varepsilon_i$$ with $ \varepsilon_i$ is $iid$ ~ $N(0,\sigma_{\...
0
votes
0answers
17 views

Determining probability of a sample proportion [ex. from Khan Academy]

I need a bit of clarification to understand sampling distributions through Khan Academy. My answer is widely different than the answers given in the sample The example question can be summarized as: ...
0
votes
0answers
13 views

Sampling methods - Stratified Vs. Probability Proportionate to Size Cluster

India has 29 States, each of which is further sub-divided into smaller administrative Districts of varying size and population. I need to download micro (firm-level) data from each state, but I do not ...
0
votes
1answer
44 views

Generating very few samples from a probability distribution using MCMC?

Since MCMC converges to target only after taking very large number of steps, what if I want to have just say 10 samples from target distribution? Do I still have to generate lots of samples, and then ...
1
vote
0answers
30 views

Resample from data with constraints to the marginal distribution

Motivation This problem comes from the situation where I have a non-random sample of individuals for which $p$ variables are measured. The target is to extract a subset of individuals which would be ...
2
votes
1answer
26 views

Should training data in each batch size be resample only one time or at each epoch when using mini-batch

I saw some related question regarding to the fact is one should use sampling with resampling or without when using minibatch. However my question is different. Let's assume that I use sampling ...
0
votes
0answers
58 views

How to use data_utils.WeightedRandomSampler and still be able shuffle training data in Pytorch?

I am working on the multi-label classification task in Pytorch and I have imbalanced data in my model, therefore I use data_utils.WeightedRandomSampler method ...
1
vote
0answers
18 views

sampling from multivariate distribution using copula

I'm trying to get a sample from a multivariate distribution which is constructed by copula. this is the steps that i go,is it true? at first i estimate the copula and the marginals then get a sample ...
2
votes
1answer
61 views

Sampling from empirical distribution

I have a vector of y (min is > 0, max could be 1), for which, i have no idea what distribution is. But based on the data we have, vector y, we can get the empirical cumulative probability distribution,...
0
votes
0answers
7 views

Measurement theory and sample size calculation for multivariate testing

Let $\mathbf{Y}$ be a vector of independent, normally-distributed random variables. Let $S_1$, $S_2$ and $S_3$ be three non-overlapping samples of sizes $N_1$, $N_2$ and $N_3$, respectively. Let $M_A$ ...
0
votes
0answers
6 views

Quality metric of sampled time series data

I have a time series that has too many points. I sample one in every 100 points, in order to reduce the amount of data I need to transmit from my measurement device. What accuracy metric can I use ...
1
vote
1answer
17 views

Gibbs sampler for Dirichlet Process concentration parameter

I am trying to implement a Gibbs sampler for Hierarchical Dirichlet process, but I cannot seem to correctly estimate the concentration parameters. I therefore started testing just this part of a ...
0
votes
0answers
9 views

coverage index?

Suppose I have a space of potential outcomes X with a probability distribution on it. I assume that there is a distance function between elements of X (e.g. X is a metric space). I also have a set S ...
4
votes
1answer
72 views

Can the Bayesian but not the frequentist “just add more observations”?

Since the frequentist's p-values are uniformly distributed under the null hypothesis, it is a highly problematic practice to add more and more data to your sample until you find a significant result. ...
0
votes
0answers
53 views

Use samples from each individual or pool samples from several individuals.

I have a system that can predict that can predict the blood pressure for hospitalized patients from zero to 120 minutes into the future. Now I want to see if my predictions are statistically ...
2
votes
2answers
72 views

Measure that takes samples that is minimized in expectation for a uniformly-distributed random variable?

I am having trouble thinking of a function that operates on a set of samples, that is, single-valued random variables between zero and one, $x_i \in (0,1), i\in\{1,2,...I\}$, and provides a measure of ...
1
vote
0answers
19 views

Total variation for draws from probability densities

I was wondering if there is a metric that is similar to that of total variation, which gives a measure of how different two probability distributions, $f$ and $g$ are, but works for samples? So ...
2
votes
1answer
82 views

Conditional Distribution to sample

Suppose I have six data points (n,x): (14,5), (13,4), (7,3), (10,5), (12,7), (20,13) which are realizations of binomial experiments on n trials with x successes respectively.. And I assume I ...
0
votes
2answers
55 views

Sampling distributions seem to be quite useless

I am studying estimation and found the concept of sampling distribution hard to grasp. The book I am reading claims that "sampling distributions" answers the following question: how confident should ...
0
votes
1answer
146 views

How to create a distribution and sample?

Suppose we are given some small set of data on bundles of electrical wires and increasing voltages run through them, and we note how many of the individual wires fail. So for example, a large data ...
0
votes
0answers
46 views

Obtain a random sample from a sum of two dependent random variables

Suppose $X$ and $Y$ are dependent random variables and I know the marginal densities of $X$ and $Y$ (if simple we can assume they are Gaussian). Using a copula I may be able to estimate the joint ...
2
votes
1answer
53 views

Machine learning algorithm test/evaluation sample size

I have recently implemented a machine learning algorithm as a part of a new credit risk scoring system. I would now like to evaluate the accuracy/performance of the algorithm when used in a "real ...
0
votes
0answers
15 views

Almost Sure Convergence and Subsamples

My actual question is in the last paragraph, but I will start with a basic example. In the book "A Course in Large Sample Theory" (Ferguson), they present the Strong Law of Large Numbers as the ...
3
votes
0answers
109 views

Confusion in terminologies for simple linear regression model [closed]

Please go through my draft summary below and let me know if my conventions are correct, comprehensible, and non ambiguous. Simple Linear Regression Model Let given observed sample set be $\{(x_1,...
0
votes
0answers
10 views

What are the differences between Domain Estimation and Poststratification

Both of these two methods will firstly select a simple random sample without replacement and decide which groups these observations belong to and use ratio estimation to get quantities of interest. It ...
1
vote
1answer
41 views

Variance of $Y|x$ from regression line

Using simple linear regression model, and sample correlation coefficient $r$, for a sample set $X,Y$, the true regression line could be given as below. $$ \hat{Y}|x = \overline{y} + r\dfrac{s_Y}{...
1
vote
0answers
14 views

Random sampling at two different frequencies

In a production line, random samples are drawn to be tested for properties A and B. A has to be tested at a higher cadence than B and the assumption is that A and B have zero correlation. Since the ...
0
votes
1answer
57 views

Long samples from Gaussian Process _prior_

I'm interested in being able to sample a long (N~10^5) sample from a Gaussian process. For a small sample I understand I can quite easily construct an NxN covariance matrix and then choose a random ...
1
vote
1answer
34 views

What is the probability/likelihood of a sample being drawn from a probability distribution over binary values

Suppose we have a known discrete probability distribution $X$ over $\{0,1\}^k$. Given a sequence of binary values $e = (e_1, ..., e_n)\text{, where } e_i\in \{0,1\}^k$, what is the probability (or the ...
0
votes
1answer
33 views

Algorithmic or structural limitations of space-filling Latin hypercube sampling

I'm new to Latin hypercube sampling, and am trying to understand if the somewhat odd sampling that results from the Matlab function lhsdesign is a limitation of the ...
1
vote
0answers
8 views

sampling for small population sizes and multiple outcomes

What's the right formula to determine sampling size (without replacement) for a small population and multiple groups? I have a population of N = few hundred events (per month) where I'd like to ...
0
votes
0answers
14 views

Fast Approximate Sampling from Multivariate Normal Parameterized by Precision Matrix

I want to efficiently sample $x \sim N(\mu, \Omega)$ where $\Omega$ is a precision matrix (e.g., the inverse of the covariance. The challenge is that the dimension of $x$ is massive (~ 100K to 10M) ...
0
votes
0answers
66 views

Is my Correlation reasoning correct?

I am trying to understand how to arrive at $r = \dfrac{Cov(X,Y)}{\sigma_X\sigma_Y}$ with a logical narrative. This in fact is kind of continuation from my this unanswered question. I see that by ...