Creating samples from a well-specified population using a probabilistic method and/or producing random numbers from a specified distribution.

learn more… | top users | synonyms

1
vote
0answers
11 views

Uniform sampling of a set of weighted samples

Consider a two-stage sampling scheme: First, use weighted random selection from a list to obtain a set of N unique elements. Next, use uniform random selection to pick one of those elements. How can ...
0
votes
0answers
15 views

Facing Single-Class Training Set when Using Random Sampling

In a highly imbalanced binary classification (rare class < 10% of whole data), when I perform random sample selection (less than 15% of whole data to be selected for training) in a trial of 1000 ...
0
votes
0answers
6 views

Balanced sampling in R [migrated]

Currently I'm using "cube" function for balanced sampling in R. It works fine on moderate amount of data. However, if the entire population of 10,000,000+ is used, R hangs. Is there any alternative ...
6
votes
1answer
104 views

Gibbs sampling for Ising model

Homework question: Consider the 1-d Ising model. Let $x = (x_1,...x_d)$. $x_i$ is either -1 or +1 $\pi(x) \propto e^{\sum_{i=1}^{39}x_ix_{i+1}}$ Design a gibbs sampling algorithm to generate ...
1
vote
3answers
97 views

Sampling data to have specific mean and standard deviation

I have a data that I want to sample such the resultant distribution of values should have specified mean and standard deviation. I can think of rejection sampling to achieve this however that seems to ...
4
votes
0answers
12 views

Is it necessary to sample a raster for spatial regression?

I'm looking to model land cover change using a variety of environmental predictors (e.g. elevation, rainfall, etc.) stored as raster layers. In most similar studies I've found in the literature the ...
1
vote
1answer
46 views

Can a machine learning algorithm be evaluated based on a random sample?

I am trying to evaluate how well (or bad) a semi-supervised algorithm is performing on a given dataset. The algorithms assigns one of 10 labels to each data point. The dataset is huge, and it's not ...
1
vote
1answer
24 views

Probability of Two Samples Containing Overlapping Data

Quick question about sampling a data population that contains non-overlapping data: if I take two samples of 50 from a population of 100,000 unique data points, how would I calculate the probability ...
4
votes
1answer
114 views

Why is the sampling distribution of normal distributed variable automatically also normal distributed

I am currently reading about the standard error. I know about central limit theorem, but I don't understand why, if my variable is normally distributed in the population, the sampling distribution ...
1
vote
0answers
12 views

The effect of oversampling on the positive predictive value

I need to calculate the positive predictive value for a validation set for a rare event. The problem is that the validation set was oversampled for the rare event. The event occurs in 5 percent of the ...
0
votes
1answer
27 views

How to determine probability of score when selecting from pool of values?

The question seems simple but I just can't solve it. I have twelve test scores and three are to be picked at random to determine the overall test grade. How can I calculate the probability of my ...
0
votes
1answer
19 views

Updating set of probabilities for sampling with features importance

I'm currently working on some algorithm and I'm kinda out of idea for a problem I'm trying to tacle. Basically I'm trying to subsample the features of a dataset. I want to subsample that given this ...
1
vote
1answer
29 views

Addressing Non-response in a Convenience Sample

I am studying customer satisfaction in a large hierarchical organization. I plan to administer a voluntary survey to customers across the organization, and need to address non-response in my analysis. ...
0
votes
0answers
11 views

Choosing an appropriate proposal distribution for metropolis hastings

So far the only constraint I've found for sampling from some target distribution $\pi(x)$ is that the proposal distribution must include the support of $\pi(x)$. That's very vague. What makes a ...
3
votes
1answer
68 views

How to find the probability that a random sample of 10 points out of 52 points of data will have a mean close to the total mean?

I have a data set that consists of 52 weeks of data. I want to find the probability that any 10 weeks of data within the set of 52 weeks is close to the mean of the total 52 weeks. Close meaning ...
1
vote
2answers
27 views

Distribution of max of samples with replacement

Suppose you have a set of numbers $\{1,2,...,m\}$ where $m \ge 5$ . Now you randomly choose five of those elements with replacement, $\text{a}_1$ ... $\text{a}_5$. What is the distribution of ...
3
votes
1answer
27 views

Defining PSU in “Sampled with Replacement” Cluster Samples

I am trying to decide on an approach to estimate design effect for a multi stage cluster survey. The clusters were selected with probability proportional to size sampling WITH replacement. The ...
0
votes
0answers
48 views

Help with 'alternate' proof that $E[x_i] = \bar{X}$

Notation $x_i$ - value of the ith observation in the sample $X_j$ - value of the jth observation in the population $\bar{x}$ is the sample mean $\bar{X}$ is the population mean This follows ...
0
votes
1answer
39 views

How to show that the $E[x_i] = \bar{X}$ [duplicate]

I know that $E[x_i] = \sum X_iP_i= \bar{X}$ But I can't quite figure out the probability in the middle step. I can't find any material online to help clarify this. Any suggestions ?
1
vote
2answers
89 views

Help with proof that the expected value of $x_i$ is $\bar{X}$

I'm having a little trouble with the proof that the expected value of $x_i$ is $ \bar{X} $. What I have is $E[x_i]=\sum_{j=1}^{N}X_j Pr(x_i=X_j) $ Then, $Pr(x_i=X_j) = 1/N $ This is the bit I ...
3
votes
2answers
42 views

Comparing Bernoulli means across subpopulations in which the number of observed successes may be zero

I've got a binary characteristic and a population $S$ with size $n$ and $P[X] = p$ such that $p$ may be small and $n$ is extremely large. Within this population are subpopulations of various sizes ...
0
votes
0answers
13 views

Subsampling two datasets so that the new sets have similar joint prob. distribution

I want to subsample two equivalent (in terms of features/columns) data sets in a way that the new subsampled data sets have the same joint probability distribution. To explain it better on an example: ...
1
vote
1answer
31 views

How do I remove emotional bias in rating survey questions?

I want to do a survey about the quality of a slot game. At the end of a game session (when player cashes out), there will be a question on screen: "Please rate the game on a scale of 1 to 5" Player ...
0
votes
1answer
26 views

Why is Sampling Importance Resampling (SIR) better than Importance Sampling (IS)?

From what I understand, SIR is a mechanism for sampling from a distribution $p$ that works as follows: Approximate a target distribution $p$ using an importance sample $S$ from a proposal ...
1
vote
1answer
38 views

Calculating mean and variance with logarithmic sample weights

I have run into a problem that must be pretty simple, but I keep getting snagged somewhere. I have an algorithm that returns a sample and the logarithm of the sample weight (which get themselves ...
3
votes
1answer
54 views

Is my understanding of this “confidence interval” solution correct?

I was faced with a problem of automatically detecting regions of continuity in a vector. I have a lot of these vectors, hundreds. One example of such vector is here. Basically looking at the vector, ...
3
votes
0answers
19 views

Equivalence of simple random sampling with- and without- replacement

We know that in simple random sampling without replacement the probability of drawing n units from a population of N units(where ordering is considered) is 1/(N)n or 1/(N n) (where ordering is not ...
1
vote
0answers
13 views

Gap in testing random forest using samples with different construction

I'm using a random forest to predict a true/false classification. I have roughly 20,000 registers per month over a year. If I leave out of my train set a random 20% of the data, I get ~40% KS. If ...
1
vote
0answers
11 views

The relations between sampling and optimization

Assume that we have $n$ training data $x_1, ... x_n$ , generated by a probability model $P(x;\theta)$. We want to estimate the parameters $\theta$ of the model based on the observations. In ...
2
votes
1answer
30 views

How many times, in expectation, am I fetching a random sample from a predefined discrete probability distribution of elements?

As part of a simulation that I'm working on, I have a probability distribution over $n$ elements, from which I have to sample a set $S$ of size $m$. That is, each element $e \in S$ must be unique ...
2
votes
0answers
142 views

Optimal sampling strategy for EFA, CFA and SEM

I'm wondering what should be the optimal sampling strategy for my dissertation research. I have four data sources (two open source software projects meta-repositories and two global startup ...
1
vote
0answers
21 views

Sampling from an arbitrary distribution with unknown CDF

I have a continuous distribution whose PDF I know the expression for but whose CDF is difficult to compute analytically. I understand that if I know the CDF value, then I can use inverse transform ...
0
votes
0answers
25 views

Analysis of forest inventory data - non-random samples

I apologise, this isn't a single question. It is more of a general problem on which I am working and am seeking guidance for how to proceed. I have been provided with an inventory dataset of plot ...
1
vote
0answers
32 views

What is a proper scale to do the social network analysis?

Problem My problem is to profile the individual user (i.e. mine individual user's interest, location, and many more). What we have as input are the network structure (e.g. linked-in network) ...
1
vote
1answer
35 views

Effect of difference operator on sampling

Suppose I have a dataset $D$ and let it be split into 2 smaller datasets $D_1$ and $D_2$ such that $D = D_1 + D_2 $. Thus we can also say that $D_1=D - D_2$. Will we get a simple random sample of ...
1
vote
1answer
71 views

Rejection sampling from a normal distribution

I am running a Monte-Carlo simulation and I sample from various normal distributions. I was just wondering, is there a way by which I can increase the probability of selecting a point from the tails ...
3
votes
1answer
46 views

Cross Validation in Unbalanced Datasets

Is there a specific way of sampling which maintains the ratio of samples in an unbiased set? e.g., lets say I want to do k-fold cross-validation on my training set And my training set is very ...
6
votes
1answer
116 views

Expected standard deviation for a sample from a uniform distribution?

I've been trying to find information on the sampling distribution of the standard deviation for uniform distributions and have been having a heck of a time figuring out the expected value for the ...
1
vote
0answers
27 views

How can I sample from a multinomial distribution with a fixed expected number of remaining categories?

I have a population of entities associated with different categories, say blue 50000 red 300 green 80 yellow 10 pink 6 orange 3 white 2 The ...
1
vote
0answers
15 views

Why use the t-distribution for confidence interval for difference of means for unpaired samples when the population variance is unknown?

I am trying to understand how to derive the confidence interval for unpaired samples when the population variance is unknown and we assume that they are equal: $$({\bar X} - {\bar Y}) \pm ...
2
votes
1answer
82 views

How does the proof of Rejection Sampling make sense?

I am taking a course on Monte Carlo methods and we learned the Rejection Sampling (or Accept-Reject Sampling) method in the last lecture. There are a lot of resources on the web which shows the proof ...
0
votes
1answer
19 views

sampling question

I have a question about inclusion criteria to obtain a representative sample of people with "sustained" or uncontrolled high BP. A colleague proposed that I look at health history data retrospectively ...
1
vote
1answer
33 views

Validating a Data Set.. Which Test should I use

I have a data set(size around 1000) which was collected by a third party. I want to verify the correctness of this survey data using random sampling (by telephoning). Is this a correct approach to the ...
0
votes
0answers
41 views

Importance Sampling to evaluate integral in R

I have asked the question here also. However, there might be something wrong with my theoretical understanding hence I'm asking here as it is more relevant. Kindly do not diss without looking first. ...
6
votes
2answers
176 views

MCMC methods - burning samples?

In MCMC methods, I keep reading about burn-in time or the number of samples to "burn". What is this exactly, and why is it ...
0
votes
1answer
45 views

How should I handle correlated samples

For simple random sampling, I know that the probability of each point being part of the sample should be equal. Also, any sample of size say $k$ should be equally likely. In the sampling procedure I ...
1
vote
0answers
10 views

What test is best for comparing diagnostic accuracy

I am using a group of observers who will report a certain type of X-ray test on a sample of 50 patients to look for fractures. 2 weeks later the same sample of patients will be reported again, this ...
0
votes
1answer
21 views

How should I split this data when testing for heteroskedasticity

I have a set of time series data and am looking to split into different time periods to test for heteroskedascity or not over different time frames. Intially, I planned to do it this way: Take the ...
0
votes
0answers
41 views

What sample size do I need?

I have a population of 709 and the ability to draw a truly random stratified sample. I understand how to do the stratified part once I determine the sample size. One calculation indicated that I need ...
3
votes
2answers
85 views

Interpreting the sample mean, $\bar y$

I have a hypothetical population that contains the values $2,3,4,5,6,7,8,9$. I have to draw a sample of size $4$ from that given population. So I have $\binom{8}{4}=70$ ways to draw the sample. I ...