The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [sampling]

Creating samples from a well-specified population using a probabilistic method and/or producing random numbers from a specified distribution. As this tag is ambiguous, please consider [survey-sampling] for the former and [monte-carlo] or [simulation] for the latter. For questions regarding creating random samples from known distributions, please consider using the [random-generation] tag.

Filter by
Sorted by
Tagged with
1
vote
0answers
29 views

Dealing with non-representative samples of a larger population

I'm seeking some advice about a project that I am working on. In this project, my colleagues and I are planning on collecting a sample that is intended to be representative of the national adult ...
0
votes
0answers
18 views

Sample from known function

I am wondering how is it possible to not be able to sample from a function that is known. E.g. suppose the expression of $f(x)$ is known but we need to approximate it in order to sample from it, why ...
0
votes
0answers
19 views

Best way to measure treatment effect across two treatments

I have two sets of patient-level data for two distinct treatments, unfortunately, the data is only over a 14 week period and I'm hoping to build a predictive model to estimate/simulate what the data ...
1
vote
0answers
33 views

sampling mechanism for correlated variables [closed]

Assuming Y=f(x1,..,Xn), while doing Monte Carlo simulation, I need to sample x1, ..xn based ...
2
votes
1answer
85 views

What does it mean to generate a random variable from a distribution when random variable is a function?

I am looking at a reference for sampling from a distribution, and the first step of the so-called algorithm states:http://www.columbia.edu/~ks20/4703-Sigman/4703-07-Notes-ARM.pdf Generate a random ...
1
vote
0answers
18 views

Randomized blocking “after the fact”

I have 5 treatments that I'm going to randomly assign to visitors to my website. I'll have 10,000 or more visitors. Is it statistically valid to analyze the results based on visitor characteristics ...
1
vote
1answer
50 views

Does standard error for a sample mean make assumptions about the distribution?

Suppose I calculate standard deviation of a sample mean as $ \sigma / \sqrt{n} $ where $\sigma$ is the population standard deviation and $n$ the sample size. Have I just made an assumption that the ...
0
votes
0answers
18 views

Why the ratio of (distance from expectation)^2 / expectation in a goodness fit test follow a chi-square distribution?

I know that the sum of square of normal random variables follow a chi-square distribution. But when I learn how to do a goodness-fit test I don't know why the ratio of (O-E)^2/E follows a chi-square. ...
1
vote
0answers
23 views

Gibbs Sampler for mixture models: shall I skip some samples to avoid to use correlated samples? [duplicate]

I am implementing a Gibbs sampler in order to estimate the parameters of a mixture model. Assuming that the parameters are contained in a vector $\theta$ what I will do is: Implement and run the ...
5
votes
1answer
348 views

What are some techniques to augment tabular data?

As we know we can perform data augmentation to "image dataset". We can apply random rotation, shifts, shear and flips over images. Are there techniques to augment tabular small dataset? I know the ...
0
votes
0answers
133 views

Normal Distribution with unknown mean and unknown variance [duplicate]

I'm new to Statistics, so bear with me. Let $X_1, \ldots. X_n$ be iid $N(\mu, \sigma^2)$ random variables, where $\mu$ and $\sigma^2$ are unknown. Let $\bar{X}_n = \sum_{i = 1}^n X_i / n$ and $\sigma' ...
0
votes
0answers
10 views

Renormalizing a distribution to reduce variance

I have a predictive model $M$ that generates an empirical predictive distribution $P_M$ via a set of samples. I cannot change the predictive model. I can evaluate the predictive performance using ...
0
votes
0answers
16 views

Should I use statistical inference on this “sample”?

The dataset gets its data from thousands of individuals throughout the US who update the same spreadsheet of about 5000 rows. This dataset contains address for individuals and is updated by the ...
0
votes
0answers
17 views

How to calculate the mean of a sub sample, given the mean of the super sample and the standard deviation of the population?

I need to run a simulation of cash flows for a project. We are selling a service. The service comes with a range of options. Depending on the specific options chosen, the cost of the service can ...
0
votes
0answers
13 views

Type I and II errors accounted for by one equation?

On this website: https://clincalc.com/stats/samplesize.aspx both type I and II errors are accounted for by the one equation. Surely, because a type I error involves too much positive variance and a ...
6
votes
1answer
587 views

How to generate samples of Poisson-Lognormal distribution

I would like to compute samples of the number of product purchased in a supermarket. I want to model it with a mixed Poisson lognormal distribution. Items purchased $x$ of a given consumer follow a ...
0
votes
0answers
7 views

probability sampling for retrospective chart review RCR/Medical Record Review

how best can you achieve probability sampling in a medical record review. In answering question, please use a real example. Say you have 4000 pregnancies to review, how many of these do you need to ...
1
vote
2answers
29 views

Evaluating proposed under-sampling method

I am currently working on an under-sampling procedure to tackle problems that arise when training and test distributions are different. Does the following experiment set-up enable performance ...
0
votes
1answer
17 views

Do pollsters use independent samples?

Is it a common practice for political pollsters to use random independent samples to conduct their polls?
0
votes
0answers
19 views

How to choose my next sample point in a 2D boundary problem?

I've got a real-world problem that I'm trying to solve with as few computations as possible. In this 2D problem, everything is parameterized on a unit square, $f(x,y); x, y \in [0, 1]$. Anecdotal ...
1
vote
1answer
35 views

I want to simulate a random sample of length n from DAG of correlated Bernoulli's

Suppose I have a DAG of 4 vertices. Each vertex consists of a Bernoulli of parameter $p$. It is the following: (Z) ---> (Y) (Z) ---> (W) (X) ---> (Y) ---> (W) I hope it is clear. Anyway, I ...
0
votes
0answers
28 views

combine samples of supersets to get a reservoir sample of the subset

Here's some background before my question (Bear with me! :D) I am working on a data pipeline that deals with a massive input log dataset; instead of saving the full input log, current version of this ...
0
votes
0answers
19 views

Calculating probability of voting results from small sample aize

2 million people voted in a poll for their favourite song. 35,000 votes have been counted so far, of which: 575 votes are for Song A 466 votes are for Song B 393 votes are for Song C (And the ...
1
vote
0answers
23 views

Using MAP inference in MRFs to estimate expectations

This question pertains to MRFs of the form $$ p(y | \theta) = \frac{1}{Z(\theta)} \exp \left( \sum_c \theta_c^\top \phi_c(y) \right) $$ with notation and nomenclature taken from [1]. Suppose that ...
0
votes
0answers
38 views

Multiple correlation in the sampling variance of slope coefficient in multiple regression

Let's say we have the following multiple regression model, $$Y_i = \alpha + \beta_1 x_{i1} +\beta_2x_{i2} + ... + \beta_kx_{ik} + \varepsilon_i$$ with $ \varepsilon_i$ is $iid$ ~ $N(0,\sigma_{\...
0
votes
0answers
23 views

Determining probability of a sample proportion [ex. from Khan Academy]

I need a bit of clarification to understand sampling distributions through Khan Academy. My answer is widely different than the answers given in the sample The example question can be summarized as: ...
0
votes
0answers
24 views

Sampling methods - Stratified Vs. Probability Proportionate to Size Cluster

India has 29 States, each of which is further sub-divided into smaller administrative Districts of varying size and population. I need to download micro (firm-level) data from each state, but I do not ...
0
votes
1answer
57 views

Generating very few samples from a probability distribution using MCMC?

Since MCMC converges to target only after taking very large number of steps, what if I want to have just say 10 samples from target distribution? Do I still have to generate lots of samples, and then ...
1
vote
0answers
56 views

Resample from data with constraints to the marginal distribution

Motivation This problem comes from the situation where I have a non-random sample of individuals for which $p$ variables are measured. The target is to extract a subset of individuals which would be ...
3
votes
1answer
240 views

Should training data in each batch size be resample only one time or at each epoch when using mini-batch

I saw some related question regarding to the fact is one should use sampling with resampling or without when using minibatch. However my question is different. Let's assume that I use sampling ...
0
votes
0answers
362 views

How to use data_utils.WeightedRandomSampler and still be able shuffle training data in Pytorch?

I am working on the multi-label classification task in Pytorch and I have imbalanced data in my model, therefore I use data_utils.WeightedRandomSampler method ...
1
vote
0answers
33 views

sampling from multivariate distribution using copula

I'm trying to get a sample from a multivariate distribution which is constructed by copula. this is the steps that i go,is it true? at first i estimate the copula and the marginals then get a sample ...
3
votes
1answer
1k views

Sampling from empirical distribution

I have a vector of y (min is > 0, max could be 1), for which, i have no idea what distribution is. But based on the data we have, vector y, we can get the empirical cumulative probability distribution,...
0
votes
0answers
19 views

Measurement theory and sample size calculation for multivariate testing

Let $\mathbf{Y}$ be a vector of independent, normally-distributed random variables. Let $S_1$, $S_2$ and $S_3$ be three non-overlapping samples of sizes $N_1$, $N_2$ and $N_3$, respectively. Let $M_A$ ...
0
votes
0answers
24 views

Quality metric of sampled time series data

I have a time series that has too many points. I sample one in every 100 points, in order to reduce the amount of data I need to transmit from my measurement device. What accuracy metric can I use ...
1
vote
1answer
119 views

Gibbs sampler for Dirichlet Process concentration parameter

I am trying to implement a Gibbs sampler for Hierarchical Dirichlet process, but I cannot seem to correctly estimate the concentration parameters. I therefore started testing just this part of a ...
3
votes
2answers
78 views

coverage index?

Suppose I have a space of potential outcomes X with a probability distribution on it. I assume that there is a distance function between elements of X (e.g. X is a metric space). I also have a set S ...
4
votes
1answer
82 views

Can the Bayesian but not the frequentist “just add more observations”?

Since the frequentist's p-values are uniformly distributed under the null hypothesis, it is a highly problematic practice to add more and more data to your sample until you find a significant result. ...
0
votes
0answers
62 views

Use samples from each individual or pool samples from several individuals.

I have a system that can predict that can predict the blood pressure for hospitalized patients from zero to 120 minutes into the future. Now I want to see if my predictions are statistically ...
2
votes
2answers
77 views

Measure that takes samples that is minimized in expectation for a uniformly-distributed random variable?

I am having trouble thinking of a function that operates on a set of samples, that is, single-valued random variables between zero and one, $x_i \in (0,1), i\in\{1,2,...I\}$, and provides a measure of ...
2
votes
1answer
86 views

Conditional Distribution to sample

Suppose I have six data points (n,x): (14,5), (13,4), (7,3), (10,5), (12,7), (20,13) which are realizations of binomial experiments on n trials with x successes respectively.. And I assume I ...
0
votes
2answers
76 views

Sampling distributions seem to be quite useless

I am studying estimation and found the concept of sampling distribution hard to grasp. The book I am reading claims that "sampling distributions" answers the following question: how confident should ...
0
votes
1answer
152 views

How to create a distribution and sample?

Suppose we are given some small set of data on bundles of electrical wires and increasing voltages run through them, and we note how many of the individual wires fail. So for example, a large data ...
0
votes
0answers
77 views

Obtain a random sample from a sum of two dependent random variables

Suppose $X$ and $Y$ are dependent random variables and I know the marginal densities of $X$ and $Y$ (if simple we can assume they are Gaussian). Using a copula I may be able to estimate the joint ...
3
votes
2answers
130 views

Machine learning algorithm test/evaluation sample size

I have recently implemented a machine learning algorithm as a part of a new credit risk scoring system. I would now like to evaluate the accuracy/performance of the algorithm when used in a "real ...
0
votes
0answers
27 views

Almost Sure Convergence and Subsamples

My actual question is in the last paragraph, but I will start with a basic example. In the book "A Course in Large Sample Theory" (Ferguson), they present the Strong Law of Large Numbers as the ...
3
votes
0answers
118 views

Confusion in terminologies for simple linear regression model [closed]

Please go through my draft summary below and let me know if my conventions are correct, comprehensible, and non ambiguous. Simple Linear Regression Model Let given observed sample set be $\{(x_1,...
0
votes
0answers
32 views

What are the differences between Domain Estimation and Poststratification

Both of these two methods will firstly select a simple random sample without replacement and decide which groups these observations belong to and use ratio estimation to get quantities of interest. It ...
1
vote
1answer
51 views

Variance of $Y|x$ from regression line

Using simple linear regression model, and sample correlation coefficient $r$, for a sample set $X,Y$, the true regression line could be given as below. $$ \hat{Y}|x = \overline{y} + r\dfrac{s_Y}{...
1
vote
0answers
19 views

Random sampling at two different frequencies

In a production line, random samples are drawn to be tested for properties A and B. A has to be tested at a higher cadence than B and the assumption is that A and B have zero correlation. Since the ...