Creating samples from a well-specified population using a probabilistic method and/or producing random numbers from a specified distribution.

learn more… | top users | synonyms

0
votes
0answers
4 views

Throwing Away extreme values from Calibrated Distribution

I have a parameter $\theta$ for a model such that $0 < \theta \leq 20$. I've calibrated the model to many data sets and fit several density functions to its histogram: According to this fitting, ...
0
votes
0answers
10 views

For classification w unbalanced datasets, is class-weighing the same as oversampling?

in unbalanced classification problems, I find myself using class_weigh = "auto" or similar parameters often, but I don't think I'm fully understanding what it's doing. I know that it's the industry ...
0
votes
0answers
18 views

how can I sampling by rows with partition? [on hold]

I have customer orders data,and I have defined a variable churned to specify who is churner and who is not churner,but there are big differences between the proportion of churner and nonchurner,so I ...
2
votes
0answers
12 views

rejection sampling in high dimensions

I read that rejection sampling might fail in high dimensional settings, as the rejection rate becomes too low. Intuitively - i can understand this - but i would like to understand the formal proof as ...
0
votes
1answer
17 views

Multivariate Box Muller

i am reading "machine learning - a probabilistic perspective" by Kevin Murphy - who states the following in the chapter on monte carlo inference. i understand that cov[y] = $\Sigma$, but i do not see ...
0
votes
1answer
26 views

How do I split a normal distributed sample into groups of percentiles but with an additional random noise component for uncertainty?

I have a sample of students that I want to divide into smaller groups based on a their IQ but with a certain random noise component - how can I do that? I need to cluster the best, the average and ...
0
votes
0answers
12 views

Resource allocation for many logistic models

This question is kind of vague but I will do my best to articulate it as best I can. I have many logistic regression models that are modeling polls of many different types of questions. All of the ...
0
votes
0answers
12 views

Selection bias in multilevel modeling

I want to fit a multilevel model from a big survey in which the respondent could have no education, medium education or high education. My dependent variable is a continuous variable from the group of ...
1
vote
2answers
49 views

How to sample a conditional PDF

Assuming I have two kind of balls, red and blue ones, and I pick randomly (with replacement) always a bunch of balls where the number of balls I pick is distributed according to the Poisson ...
1
vote
0answers
21 views

Sampling correlated categorical variables

I am looking for a way to sample correlated categorical (non-binary) variables, and in particular I am interested in the category counts: I have a set of $n$ correlated categorical random variables ...
1
vote
1answer
29 views

What is the standard way to distinguish the errors associated with sampling and measurement in statistics?

This is probably a very basic, yet not easy, question in applied science. I was just wondering, what are usually the standard ways to deal with it? Any pointers to further references are greatly ...
0
votes
1answer
35 views

Sampling from distribution using MATLAB / statistical packages like R

I am trying to read Latent Dirichlet Allocation model for Topic modeling. But I am not able to understand how they sample values from a distribution, draw a distribution from a process. In particular ...
1
vote
0answers
24 views

Confidence intervals for estimates generated from a non-probability sample?

From what I understand, to generate a margin of error to have confidence intervals for a given estimate one needs the standard error of the estimate (SE). For the SE one needs information about the ...
1
vote
1answer
17 views

Determination of sample size for a proportion

SCENARIO The parent company has other business units (think of these as franchises owned by the parent company). In order to quality assure procedures the parent company samples employees’ work at ...
2
votes
2answers
33 views

How to empirically show that a certain quantity approximately follows a normal distribution?

To motivate some theoretical work, I need to show that two certain variables (say $X$ and $Y$) approximately follow a normal distribution in actual datasets. I have one large dataset with about $300$ ...
0
votes
0answers
40 views

Standard deviation of sampling distribution of mean

If we take a sample and calculate the mean, we can calculate the standard deviation for the sampling distribution of the mean using this formula: $\sigma / \sqrt{n}$ But, how many samples do we need ...
0
votes
0answers
20 views

Experimental design for high dimensional model

I am running a computer model with a lot of inputs ( > thousands). I am not familiar with this area, so may I ask for any instructions/directions on how to make design on such high dimensional space? ...
1
vote
0answers
21 views

Uniformly sampling principal component scores to explore response surface

New simpler version of the question Consider a sample $\mathbf{X} \in \mathbb{R}^{n\times p}$ of $n$ points in $\mathbb{R}^p$ with $p$ small, say $p=5$, and $n$ large, say $n=3000$. Because they are ...
3
votes
1answer
53 views

About the central limit theorem and statistical testing

Wikipedia states that In probability theory, the central limit theorem (CLT) states that, given certain conditions, the arithmetic mean of a sufficiently large number of iterates of independent ...
0
votes
0answers
26 views

How to find this conditional probability in sampling without replacement in this experiment?

My problem is the following: I have an urn with 10 different names. Consider a particular name (ex. John). I want to estimate: 1) the probability to extract John 2) the probability to extract John ...
0
votes
0answers
8 views

Comparing conversion rate across differing group sizes

I'm trying to work out a formula for determining the normalised? conversion rate for a staff member - Converting traffic into a sale. This is to determine the most successful staff at conversion in ...
0
votes
0answers
15 views

Down-sampled training set with unbalanced test set

Data: https://www.kaggle.com/c/GiveMeSomeCredit/data (cs-training.csv) Training Tool: Python, Numpy, Pandas Balancing data with down sampling is a recommended solution to data imbalance. I balanced ...
0
votes
0answers
24 views

Sampling distribution of t values

In my experiment, analysis is performed on the single subject level such that for each subject I get a t value*. the degrees of freedom are identical for all subjects. I would now like to ask whether ...
0
votes
0answers
9 views

undersampling and class imbalance

What Data Mining software ( like SPSS, Weka) can be used to check/ verify a new under sampling approach such that the new proposed approach yields best results than the rest???
2
votes
0answers
24 views

Uniform Sampling from Intersections of Faces of Simplices

I'm trying to sample uniformly on the intersections of faces of several simplicies, with all coordinates being non-negative. That is, given constraints $$A\vec{w}=\vec{b} \ \ and \ \ \vec{w} \geq ...
0
votes
1answer
41 views

Random sample of a random sample from a population: Also a random sample?

From population P we draw an adequately sized random sample S1. From the sample S1 we draw an adequately sized random sample S2 (with replacement). Are the distributional characteristics of S2 ...
1
vote
0answers
20 views

Capture recapture model validation

In the absence of an actual population to test on, what are the best practices for validating capture recapture models?
1
vote
1answer
12 views

Sampling from matrix-normal distribution

I would like to sample from a matrix-Gaussian variable (not multivariate normal). so basically, let $X \in \mathbb{R}^{m\times n}$ be a random matrix distributed according to the matrix-normal law. ...
0
votes
1answer
33 views

Joint inclusion probability for Horvitz–Thompson estimator

There is a Wikipedia page for the Horvitz–Thompson estimator. It is an estimator for the population total. Unfortunately the page has failed to state the standard error. From here, the standard ...
0
votes
0answers
13 views

How to calculate sample size for PK/PD modeling?

I'm a PhD student of Pharmacology and would like to develop a PK/PD model on TB drugs. Of course I'm in my very early stage of learning the modeling science. But, for now what I want to ask is "Can ...
4
votes
1answer
24 views

Testing whether sampling (convex polytope) is uniform

Currently, I am sampling points from: i) a convex polytope (i.e. Ax <= b) ii) a high dimensional simplex The algorithms I am using are hit-and-run and a simple version of Bayesian bootstrap. I ...
0
votes
0answers
37 views

Sampling Distribution (Variance) of Weight Estimates

I am currently facing an issue regarding the sampling distribution of weight estimates. Problem Statement Given an estimate of a $n \times n$ covariance matrix $\hat{\Sigma}$ of $n$ random variables ...
1
vote
0answers
24 views

SRSWOR involving Weighting

A Simple Random Sample Without Replacement (SRSWOR) survey is conducted that included too many women and not enough men in the sample In the resulting weighting, each female is given a weight of 1 ...
2
votes
0answers
21 views

Box Muller vs Numpy?

I used the Box Muller transform and pythons uniform random number generator to sample random numbers in a given interval [a,b]. Here's the approach I used: Note: I know the average and the standard ...
2
votes
1answer
44 views

Choosing a sample rate for GBM models

I've created several GBM models to tune the parameters (trees, shrinkage and depth) to my data and the model performs well on the out-of-time sample. The data is credit card transactions (running into ...
1
vote
1answer
34 views

How accurate is my simple description of a confidence interval?

Here's my simple words-only description of a 95% confidence interval for the mean. How accurate is it? the sample mean comes from a distribution of possible sample means the sample mean might have ...
1
vote
2answers
60 views

Get quantile function of dynamic mixture model

I have a dynamic mixture distribution fitted to my risk data (i.e., I have all parameters) of Weibull and Generalized Pareto, with a Cauchy CDF mixing function, that can be written as: $mixture(x): ...
0
votes
1answer
27 views

Estimate population mean from sample

Population contains two independent parts: Group A & Group B with size $N_A$ nad $N_B$. $N = N_A + N_B$ Now sample from Group A and Group B separately. Sample size $n_A$, $n_B$. $n = n_A + ...
5
votes
0answers
111 views

Are the balls drawn randomly (independently of the number of balls existing in their colours)?

We have a big urn that contain $N_{Tot}$ balls. Balls are of $r$ different colours. The number of balls of the $i^{th}$ colour (before sampling) is $N_i$. John sampled $x$ balls in total (without ...
1
vote
1answer
35 views

optimal sequential sampling in gaussian process models

Let's say we have a one dimensional dataset of 24 points along with their responses. I am reserving three boundary points for testing (i=1,23,24) and i am fitting a Gaussian process model based on a ...
3
votes
2answers
107 views

Sampling from von Mises-Fisher distribution in Python?

I am looking for a simple way to sample from a multivariate von Mises-Fisher distribution in Python. I have looked in the stats module in scipy and the numpy module but only found the univariate von ...
4
votes
1answer
75 views

What is the standard error of the sample standard deviation?

I read from there that the standard error of the sample variance is $$SE_{s^2} = \sqrt{\frac{2 \sigma^4}{N-1}}$$ What is the standard error of the sample standard deviation? I'd be tempted to guess ...
1
vote
0answers
71 views

Simulate from a dynamic mixture of distributions, honoring the tail

This question is a follow-up to this other question, brilliantly answered by Xi'an. I have a dynamic mixture of Weibull and GPD distributions (with a CDF Cauchy mixing function). The mixture is ...
1
vote
1answer
39 views

Proof for the sampling variance of the Neyman Estimator

I'm going through Imbens and Rubin's new book and I just for the life of me can't figure out 1 minor detail in their proof for the sampling variance of the Neyman estimator $\bar{Y}^{obs}_{t} - ...
7
votes
3answers
93 views

Enlarging a random sample

In our project we have a population of 1000+ individuals. We picked a random sample of 107 individuals, but then we realized we needed more precision, so now we want to have a larger sample. The ...
1
vote
2answers
67 views

What is the problem of singular (non-invertible) covariance-variance matrix?

What exactly is the problem of having non-invertible covariance matrix? Why is getting the inverse of this matrix so important? This problem is often encountered when doing regressiong works on ...
1
vote
1answer
25 views

Reducing size of data sets while preserving their mathematical properties

So I'm looking at the paper $l_p$ Row Sampling by Lewis Weights in which the authors provide a method to sample the rows of a matrix while preserving their mathematical properties. More specifically ...
0
votes
0answers
19 views

Can I pool my data?

I collected some data about purchase decisions. For simplicity, lets say my IVs were Price and Quality, and DV Intent to Purchase My survey prompt was something like "Answer the survey questions ...
0
votes
0answers
35 views

Can we use normal distribution statistics on frequency distribution outputted by bootstrapping?

If I understand correctly, this is the bootstrapping procedure: Pretend sample is population Repeatedly resample from this pretend-population Calculate the mean of each resample. The output of ...
3
votes
1answer
55 views

How often will sampling distribution of the mean not be normally distributed?

Kabacoff 2015 suggests that if we're not willing to assume the sampling distribution of the mean is normally distributed, we should use bootstrapping to estimate the sampling distribution of the mean. ...