Creating samples from a well-specified population using a probabilistic method and/or producing random numbers from a specified distribution.

learn more… | top users | synonyms

0
votes
1answer
18 views

Probability of Two Samples Containing Overlapping Data

Quick question about sampling a data population that contains non-overlapping data: if I take two samples of 50 from a population of 100,000 unique data points, how would I calculate the probability ...
4
votes
1answer
95 views

Why is the sampling distribution of normal distributed variable automatically also normal distributed

I am currently reading about the standard error. I know about central limit theorem, but I don't understand why, if my variable is normally distributed in the population, the sampling distribution ...
1
vote
0answers
10 views

The effect of oversampling on the positive predictive value

I need to calculate the positive predictive value for a validation set for a rare event. The problem is that the validation set was oversampled for the rare event. The event occurs in 5 percent of the ...
0
votes
1answer
17 views

How to determine probability of score when selecting from pool of values?

The question seems simple but I just can't solve it. I have twelve test scores and three are to be picked at random to determine the overall test grade. How can I calculate the probability of my ...
0
votes
1answer
19 views

Updating set of probabilities for sampling with features importance

I'm currently working on some algorithm and I'm kinda out of idea for a problem I'm trying to tacle. Basically I'm trying to subsample the features of a dataset. I want to subsample that given this ...
1
vote
1answer
22 views

Addressing Non-reponse in a Convenience Sample

I am studying customer satisfaction in a large hierarchical organization. I plan to administer a voluntary survey to customers across the organization, and need to address non-response in my analysis. ...
0
votes
0answers
11 views

Choosing an appropriate proposal distribution for metropolis hastings

So far the only constraint I've found for sampling from some target distribution $\pi(x)$ is that the proposal distribution must include the support of $\pi(x)$. That's very vague. What makes a ...
3
votes
1answer
65 views

How to find the probability that a random sample of 10 points out of 52 points of data will have a mean close to the total mean?

I have a data set that consists of 52 weeks of data. I want to find the probability that any 10 weeks of data within the set of 52 weeks is close to the mean of the total 52 weeks. Close meaning ...
1
vote
2answers
21 views

Distribution of max of samples with replacement

Suppose you have a set of numbers $\{1,2,...,m\}$ where $m \ge 5$ . Now you randomly choose five of those elements with replacement, $\text{a}_1$ ... $\text{a}_5$. What is the distribution of ...
3
votes
1answer
27 views

Defining PSU in “Sampled with Replacement” Cluster Samples

I am trying to decide on an approach to estimate design effect for a multi stage cluster survey. The clusters were selected with probability proportional to size sampling WITH replacement. The ...
0
votes
0answers
48 views

Help with 'alternate' proof that $E[x_i] = \bar{X}$

Notation $x_i$ - value of the ith observation in the sample $X_j$ - value of the jth observation in the population $\bar{x}$ is the sample mean $\bar{X}$ is the population mean This follows ...
0
votes
1answer
38 views

How to show that the $E[x_i] = \bar{X}$ [duplicate]

I know that $E[x_i] = \sum X_iP_i= \bar{X}$ But I can't quite figure out the probability in the middle step. I can't find any material online to help clarify this. Any suggestions ?
1
vote
2answers
88 views

Help with proof that the expected value of $x_i$ is $\bar{X}$

I'm having a little trouble with the proof that the expected value of $x_i$ is $ \bar{X} $. What I have is $E[x_i]=\sum_{j=1}^{N}X_j Pr(x_i=X_j) $ Then, $Pr(x_i=X_j) = 1/N $ This is the bit I ...
3
votes
2answers
41 views

Comparing Bernoulli means across subpopulations in which the number of observed successes may be zero

I've got a binary characteristic and a population $S$ with size $n$ and $P[X] = p$ such that $p$ may be small and $n$ is extremely large. Within this population are subpopulations of various sizes ...
0
votes
0answers
10 views

Subsampling two datasets so that the new sets have similar joint prob. distribution

I want to subsample two equivalent (in terms of features/columns) data sets in a way that the new subsampled data sets have the same joint probability distribution. To explain it better on an example: ...
1
vote
1answer
31 views

How do I remove emotional bias in rating survey questions?

I want to do a survey about the quality of a slot game. At the end of a game session (when player cashes out), there will be a question on screen: "Please rate the game on a scale of 1 to 5" Player ...
0
votes
1answer
21 views

Why is Sampling Importance Resampling (SIR) better than Importance Sampling (IS)?

From what I understand, SIR is a mechanism for sampling from a distribution $p$ that works as follows: Approximate a target distribution $p$ using an importance sample $S$ from a proposal ...
1
vote
1answer
35 views

Calculating mean and variance with logarithmic sample weights

I have run into a problem that must be pretty simple, but I keep getting snagged somewhere. I have an algorithm that returns a sample and the logarithm of the sample weight (which get themselves ...
3
votes
1answer
52 views

Is my understanding of this “confidence interval” solution correct?

I was faced with a problem of automatically detecting regions of continuity in a vector. I have a lot of these vectors, hundreds. One example of such vector is here. Basically looking at the vector, ...
3
votes
0answers
19 views

Equivalence of simple random sampling with- and without- replacement

We know that in simple random sampling without replacement the probability of drawing n units from a population of N units(where ordering is considered) is 1/(N)n or 1/(N n) (where ordering is not ...
1
vote
0answers
13 views

Gap in testing random forest using samples with different construction

I'm using a random forest to predict a true/false classification. I have roughly 20,000 registers per month over a year. If I leave out of my train set a random 20% of the data, I get ~40% KS. If ...
1
vote
0answers
11 views

The relations between sampling and optimization

Assume that we have $n$ training data $x_1, ... x_n$ , generated by a probability model $P(x;\theta)$. We want to estimate the parameters $\theta$ of the model based on the observations. In ...
2
votes
1answer
30 views

How many times, in expectation, am I fetching a random sample from a predefined discrete probability distribution of elements?

As part of a simulation that I'm working on, I have a probability distribution over $n$ elements, from which I have to sample a set $S$ of size $m$. That is, each element $e \in S$ must be unique ...
2
votes
0answers
139 views

Optimal sampling strategy for EFA, CFA and SEM

I'm wondering what should be the optimal sampling strategy for my dissertation research. I have four data sources (two open source software projects meta-repositories and two global startup ...
1
vote
0answers
18 views

Sampling from an arbitrary distribution with unknown CDF

I have a continuous distribution whose PDF I know the expression for but whose CDF is difficult to compute analytically. I understand that if I know the CDF value, then I can use inverse transform ...
0
votes
0answers
24 views

Analysis of forest inventory data - non-random samples

I apologise, this isn't a single question. It is more of a general problem on which I am working and am seeking guidance for how to proceed. I have been provided with an inventory dataset of plot ...
1
vote
0answers
32 views

What is a proper scale to do the social network analysis?

Problem My problem is to profile the individual user (i.e. mine individual user's interest, location, and many more). What we have as input are the network structure (e.g. linked-in network) ...
1
vote
1answer
35 views

Effect of difference operator on sampling

Suppose I have a dataset $D$ and let it be split into 2 smaller datasets $D_1$ and $D_2$ such that $D = D_1 + D_2 $. Thus we can also say that $D_1=D - D_2$. Will we get a simple random sample of ...
1
vote
1answer
70 views

Rejection sampling from a normal distribution

I am running a Monte-Carlo simulation and I sample from various normal distributions. I was just wondering, is there a way by which I can increase the probability of selecting a point from the tails ...
3
votes
1answer
39 views

Cross Validation in Unbalanced Datasets

Is there a specific way of sampling which maintains the ratio of samples in an unbiased set? e.g., lets say I want to do k-fold cross-validation on my training set And my training set is very ...
6
votes
1answer
109 views

Expected standard deviation for a sample from a uniform distribution?

I've been trying to find information on the sampling distribution of the standard deviation for uniform distributions and have been having a heck of a time figuring out the expected value for the ...
1
vote
0answers
26 views

How can I sample from a multinomial distribution with a fixed expected number of remaining categories?

I have a population of entities associated with different categories, say blue 50000 red 300 green 80 yellow 10 pink 6 orange 3 white 2 The ...
1
vote
0answers
13 views

Why use the t-distribution for confidence interval for difference of means for unpaired samples when the population variance is unknown?

I am trying to understand how to derive the confidence interval for unpaired samples when the population variance is unknown and we assume that they are equal: $$({\bar X} - {\bar Y}) \pm ...
2
votes
1answer
80 views

How does the proof of Rejection Sampling make sense?

I am taking a course on Monte Carlo methods and we learned the Rejection Sampling (or Accept-Reject Sampling) method in the last lecture. There are a lot of resources on the web which shows the proof ...
0
votes
1answer
19 views

sampling question

I have a question about inclusion criteria to obtain a representative sample of people with "sustained" or uncontrolled high BP. A colleague proposed that I look at health history data retrospectively ...
1
vote
1answer
33 views

Validating a Data Set.. Which Test should I use

I have a data set(size around 1000) which was collected by a third party. I want to verify the correctness of this survey data using random sampling (by telephoning). Is this a correct approach to the ...
0
votes
0answers
41 views

Importance Sampling to evaluate integral in R

I have asked the question here also. However, there might be something wrong with my theoretical understanding hence I'm asking here as it is more relevant. Kindly do not diss without looking first. ...
6
votes
2answers
174 views

MCMC methods - burning samples?

In MCMC methods, I keep reading about burn-in time or the number of samples to "burn". What is this exactly, and why is it ...
0
votes
1answer
45 views

How should I handle correlated samples

For simple random sampling, I know that the probability of each point being part of the sample should be equal. Also, any sample of size say $k$ should be equally likely. In the sampling procedure I ...
1
vote
0answers
10 views

What test is best for comparing diagnostic accuracy

I am using a group of observers who will report a certain type of X-ray test on a sample of 50 patients to look for fractures. 2 weeks later the same sample of patients will be reported again, this ...
0
votes
1answer
21 views

How should I split this data when testing for heteroskedasticity

I have a set of time series data and am looking to split into different time periods to test for heteroskedascity or not over different time frames. Intially, I planned to do it this way: Take the ...
0
votes
0answers
40 views

What sample size do I need?

I have a population of 709 and the ability to draw a truly random stratified sample. I understand how to do the stratified part once I determine the sample size. One calculation indicated that I need ...
3
votes
2answers
84 views

Interpreting the sample mean, $\bar y$

I have a hypothetical population that contains the values $2,3,4,5,6,7,8,9$. I have to draw a sample of size $4$ from that given population. So I have $\binom{8}{4}=70$ ways to draw the sample. I ...
1
vote
2answers
42 views

Sampling from low probability regions

I have a pdf that I need to sample from, but I'm having numerical issues and was hoping someone could provide some insight. My pdf is a product of gaussians $$ P(\theta |d ) = \prod_{i=1}^{N} C_{i} ...
0
votes
0answers
42 views

Importance Sampling MC - a couple of questions regarding PDF

I implore the good people to quickly glance through this thread over at StackOverflow to get a better idea of my question if the following isn't clear. I have these integrals to evaluate using ...
2
votes
0answers
63 views

Performance testing: detecting changes in distributions over time

I have an interesting application I’d like some advice on. A common task when working with stochastic systems in an engineering context is testing for regressions and improvements in performance. ...
0
votes
0answers
43 views

Heterogenous vs. homogeneous sample frame

1) I am interested to know why a homogeneous sample is required while recruiting subjects for a study, that will involve ANOVA and Regression Analysis. Is it not true that if the sample frame is ...
3
votes
1answer
46 views

Why is a symmetric distribution sufficient for the sample mean and variance to be uncorrelated?

While reading, I came across the puzzling statement that the sample mean and variance are uncorrelated only in symmetric distributions and there is strong correlation if the distribution is heavily ...
21
votes
4answers
2k views

Why do political polls have such large sample sizes?

When I watch the news I've noticed that the Gallup polls for things like presidential elections have [I assume random] sample sizes of well over 1,000. From what I remember from college statistics ...
1
vote
0answers
27 views

Mark and recapture with known upper bound

I was thinking this may be similar to a Mark and recapture problem where there is a known upper bound, hence the title. I am doing a proportion but sometimes the estimate is larger than the known ...