Creating samples from a well-specified population using a probabilistic method and/or producing random numbers from a specified distribution.

learn more… | top users | synonyms

2
votes
2answers
58 views

Why is it desirable to have low auto-correlation in MCMC?

I keep reading about the need to check for autocorrelation in MCMC. Why is it important that the autocorrelation is low? What does it measure in the context of MCMC?
1
vote
0answers
23 views

Using bootstrapping simulation to compare estimators in survey

I would like to estimate the mean of a population and select a best estimator with minimum variance of the estimated mean. Suppose that I have two estimators est1 and est2, and they could refer to any ...
0
votes
0answers
5 views

Brewer's method for sampling with unequal probabilities with n>2

I'm trying to use Brewer's method to sample 12 units out of a population of 73. I read on Brewer and Hanif's "Sampling with unequal probabilities" that the probability must be proportional to ...
1
vote
1answer
23 views

How a quota sample should be executed?

Let's say I have a population stratified by gender and age group. Having their proportions, how should I choose the sampling units for interview? How do I ensure the defined quotas are met? If the ...
0
votes
0answers
15 views

What is the best ensemble sampler for highly correlated parameter space?

I have a likelihood that I want to estimate the free parameters for it and I am using MCMC to estimate the parameters. Two of the free parameters are positions and I defined uniform priors and one has ...
1
vote
0answers
31 views

Simpson's Paradox with new sample

Suppose you constructing model whose training data is cumulative in nature; meaning each year you can add new observations with all prior observations being kept the same. (e.g. training set is ...
0
votes
0answers
28 views

How many samples do I need to prove that a classification algorithm is better than another?

I have two algorithms A and B, used to automatically classify each of N elements into K categories, N and K both being in the millions. Neither A or B is perfect, but it is relatively easy for a human ...
0
votes
1answer
21 views

Leave-one-subject-out cross validation in Caret

Hi Dear Colleagues, I wonder how to correctly setup a leave-one-subject-out cross validation (LOSO) for train() function in caret. Here is my example code: ...
0
votes
0answers
24 views

Estimating number of unique people

Assume from a prior experiment with have a known truth table of misclassification of logins on an individual basis. $$ \begin{array}{c|lcr} \text{Truth}/\text{Observed} & \text{Al (M)} & ...
2
votes
0answers
44 views

Birthday “Paradox” — with a different perspective

Background: Many people are familiar with the so-called Birthday "Paradox" that, in a room of 23 people, there is a better than 50/50 chance that two of them will share the same birthday. In its more ...
-1
votes
0answers
19 views

How to calculate sample size? [closed]

I need to know how many people of risk group do I have to screen, to get any statistical significance? For example renal cell carcinoma is diagnosed to 25 people from 100 000 population. If they have ...
1
vote
0answers
16 views

What are the statistical implications of assigning treatment and control in time series study under 2 different approaches?

I have a project wherein my group wants to measure the effect of counseling at-risk students on their academic outcomes. There are 1,000 students who will be enrolled during the current academic ...
0
votes
1answer
40 views

What are the null and alternate hypotheses? Will you use a left, right, or two-tailed test?

Ben has a coin which he claims is weighted in a way so that when he flips it, heads appears more often than 50% of the time. He tries to prove it to you by flipping the coin 100 times, which ...
0
votes
0answers
18 views

Inclusion probabilities in a sampling with unequal probabilities with Brewer's method, n > 2

I'm trying to estimate the total of a variable in a population using an unequal probability sample with $n>2$ and the Brewer's selection method. I draw the first unit with probability ...
2
votes
1answer
64 views

Optimal Allocation

A campus population of size N=9000 is to be surveyed by a stratified sample for the prevalence of a certain disease based upon three strata of respective sizes $N_h$ = 1000, 3000, and 5000 for h = 1, ...
2
votes
1answer
26 views

Expected coverage of a set of random samples

For a set A, I'm running 8 independent random samples, each with a probability of 1/8=12.5% and is without replacement. I know that the set formed by the union of these 8 samples will be of a size ...
3
votes
1answer
31 views

Pearson Correlation for Clustered data

I have a sample that was drawn using cluster sampling. Let say I have variables x and y, and I would like to check the significance of correlation between those two variables in SPSS 20 taking into ...
0
votes
0answers
16 views

Sampling Distribution of mean for Poisson Distributions

Suppose I have random variables $X_i$ which are Poisson distributed with mean $\mu$. I m interested in the sampling distribution of the variable $\frac{X_1+...+X_n}{n}$. We now that as $n$ goes to ...
1
vote
1answer
27 views

Verifying representativeness of a sample, after simple random sampling

I just used the standard formula to determine the sample size of a sample to match the mean of a pupulation with an error margin of 3 percentage points and with a 90% probability. I know would like ...
0
votes
0answers
10 views

How to improve results when using sampling in skewed binary classification?

I am using a data set with 18 features with True/False output (Related to mobile ad targeting). True values occurs only 0.4 % of the time. So, I have used sampling to keep the ratio of True and False ...
1
vote
1answer
24 views

sample frame representing the true population

(i) Is it true to say that if the sample frame is not a random selection (such as a convenience sample), then the sample frame is not representative of the true population? (ii) We can always do a ...
0
votes
0answers
12 views

Down-sampling with building models (specifically random forests)

I was wondering if anyone had ever used down-sampling to build random forests with data that has unbalanced classes. Basically down-sampling samples (with replacement) x*min from the population where ...
0
votes
0answers
30 views

How do I sample only non-null values in R? [migrated]

Is there was a way to sample only non-null values in R? Currently, I have this code. s <- sample(200000, replace=TRUE); m <- mydata$myvar[s] However, some ...
0
votes
1answer
19 views

Incomplete questionnaires

Is it recommented to delete the cases from a database if the responses are highly incomplete? Is there a percentage that can be taken into consideration? (for example, deleting a case if more than 50% ...
0
votes
0answers
20 views

Sobol sequence for a set of discrete numbers

Is it possible to create a sobol sequence for a set of numbers? I have 5 discrete variables x1, x2, x3, x4 and x5 and say for example x1 = {1,2,3,4,5}, x2 = {10,30,40,50,60}, x3 = {1,3,5,7,9}, x4 = ...
0
votes
1answer
44 views

Linear regression without intercept - sampling variance of coefficient

I am comparing linear regression with and without intercept for the general sampling case. For this, I have $n$ samples of two correlated random variables $X \sim N\left(0,\sigma_X^2\right)$ and $Y ...
0
votes
0answers
12 views

How to get a representative sample from a large data frame given a factor [migrated]

If one column is a factor, but each subset of the data frame has a different size -- maybe a subset has thousands of rows, while another has tens or hundreds of thousands of rows -- sampling done with ...
1
vote
0answers
78 views

Given $N$ samples from $p(x|y=y_0)$ how can I infer $y_0$?

I have $N$ samples $x_i \sim p(x|y)$ for $y = y_0$. I don't know apriori what $y_0$ is but I know its a fixed value. I do not have the analytic form of $p(x|y)$, $p(y|x)$, $p(x,y)$. Instead I have ...
1
vote
1answer
37 views

Is it true that values of parameters do not vary from sample to sample?

This is something I was told. However, it seems to me that parameters of a population can be functions of time, in certain situations, in which case the values of parameters could vary from sample ...
1
vote
1answer
49 views

burn in for Metropolis Hastings MCMC

I was wondering if there is a principled way to figure out how many samples to discard during the MH-MCMC burn-in stage. So, as I understand it, the initial samples can introduce bias in the ...
1
vote
1answer
38 views

Gibbs sampling versus general MH-MCMC

I have just been doing some reading on Gibbs sampling and Metropolis Hastings algorithm and have a couple of questions. As I understand it, in the case of Gibbs sampling, if we have a large ...
0
votes
0answers
9 views

Sampling a high dimensional function

I have a function $f$ that takes $N$ real-valued inputs and is very expensive to compute. I am studying the properties of $f$ by generating samples. Each input $n$ has a range of values $(n_{min}, ...
1
vote
1answer
23 views

Dirichlet sample by normalising Gamma RVs

I know that if you sample $K$ random variables $(X_1, X_2, \dots, X_K)$ from Gamma distributions using shape parameters $(\alpha_1, \alpha_2, \dots \alpha_K)$ and a scale parameter $\theta = 1$ such ...
3
votes
1answer
43 views

Ratio of correlated sample variances (gamma distributed)

for $N$ samples of two correlated random variables $X \sim N\left(0,\sigma_X^2\right)$ and $Y \sim N\left(0, \sigma_Y^2\right)$ with correlation $\rho$, I am analyzing the ratio of the sample ...
0
votes
2answers
36 views

Difference between Research Design and Experimental Design

What is the difference between Research Design and Experimental Design? I can't see any difference. Both of them need to establish Causality. Both of them are the arrangement for collection and ...
3
votes
0answers
83 views

recommendations to analyze a survey of the entire sample frame with a 20% response rate

we surveyed all 10,000 professionals in a particular industry. The industry is highly-regulated, so we have contact information for everyone in our population of interest. We attempted to contact ...
2
votes
1answer
28 views

Aggregation of random samples

Suppose I have two distributions $X$ and $Y$, and I have a statistic $T(X, Y)$. If I have a random sample (iid) $X_1, ..., X_N$ taken from $X$ and a random sample (iid) $Y_1, ..., Y_M$ taken from $Y$, ...
0
votes
1answer
63 views

Improving SVM classification

I have a classification problem (bioinformatics domain) where I have around 333 features. Currently, I am first selecting features (using importance feature of random forest) and then pushing the same ...
2
votes
1answer
32 views

Convenience sampling - Distribution forcing?

I am conducting some experiments on a data set that was collected by convenience. It is a data set based on historical data, most of which is not digitized. I know the exact distribution of the ...
4
votes
1answer
90 views

Confidence interval for a proportion estimated through stratified sampling

When estimating the confidence interval for a binomial proportion $p$, with $\hat{p}$ near 0 or 1, one has to use something other than the Wald interval to get a reasonable estimate (see for instance ...
0
votes
0answers
29 views

Drawing a random Sample on a Probability Distribution

Say I have multiple normal or beta distributions. So I have two questions. How do I Draw a random sample from a distribution? How do I compare determine which of my distributions has the largest ...
3
votes
1answer
44 views

Error Bars for Monte Carlo Experiment

Suppose we have a random variable $X$, where $\mathbb{E}(X)$ and $\text{Var}(X)$ are known. I have computed $N$ number of MC-type samples from the distribution of $X$. Let $\bar{x} = \frac{1}{N}\sum ...
1
vote
0answers
19 views

Calculating power for a survey to compare different metrics

I am currently planning a survey. The data will be used to test which of three metrics best captures some "true" underlying characteristic of the respondents. Getting data on the "true" underlying ...
3
votes
0answers
23 views

Injective uniform distribution on an n-sphere

For an application I'm working on, I need to go from some uniformly distributed variables to an uniform distribution on an n-sphere. The standard way to do this seems to be choose (n+1) normal ...
1
vote
0answers
20 views

Cholesky decomposition and confidence ellipsoid

I'm trying to construct an error ellipsoid from a covariance matrix (which exists for a 3D point) and then sample consistent xyz points in this region. (This question succeeds this one.) What I'm ...
3
votes
1answer
85 views

Sampling distribution of regression coefficients for normally distributed random variables

Based on $N$ realizations of two random variables $X \sim N(0,\sigma_X^2)$ and $Y \sim N(0, \sigma_Y^2)$ with correlation $\rho$, I conduct a simple linear regression $Y = \beta_0 + X\beta_1 + ...
3
votes
1answer
31 views

How to estimate the change in prevalence of a trait in the population from non-random samples?

Suppose I am interested in the prevalence of drug use in a population over time, but I am unable to conduct audits (sample some transactions at random and check to see if drugs were exchanged). I do ...
2
votes
0answers
39 views

Iterative or Lazy Reservoir Sampling

I'm fairly well acquainted with using Reservoir Sampling to sample from a set of undetermined length in a single pass over the data. One limitation of this approach, in my mind, is that it still ...
0
votes
0answers
49 views

Trying to use Cholesky decomposition of covariance matrix to sample error ellipsoid

I'm trying to construct an error ellipsoid from a covariance matrix (which exists for a 3D point) and then sample consistent xyz points in this region. In a previous question when I asked about this ...
0
votes
1answer
30 views

Sampling brownian motions

I wish to sample standard linear Brownian motions on the interval $[0,1]$. I do this by dividing the interval into $n$ equal sub-intervals, deciding $B(0)=0$, and letting ...