Creating samples from a well-specified population using a probabilistic method and/or producing random numbers from a specified distribution.

learn more… | top users | synonyms

0
votes
0answers
6 views

sequential importance sampling step by step

I'm experiencing some problems when learning particle filtering (without resampling). I've found examples where the bootstrap filter has been used but none using the sequential importance sampling. ...
2
votes
0answers
10 views

Sampling For Correlation

Suppose we have population P that is split into large number of close size sub-populations (e.g. country is split to zips). For each sub-population a small sample was drawn and average of a variable X ...
0
votes
0answers
6 views

Is it possible to calculate the covariance btwn data and a subset of the data?

I have a regular and complete time-series vector and have created a subset of this vector based on a particular sampling algorithm (say every 10th value). To calculate the error variance of this ...
2
votes
2answers
17 views

Probability selection of elements from a set

If we have a set $S= \{s_1,s_2,..,s_n\}$ and each element in the set $S$ has an assigned probability $P_n$. Then a selection process is applied to the set $S$ such that, each element $S_n \in$ $S$ is ...
7
votes
4answers
369 views

How to sample when you don't know the distribution

I'm fairly new to statistics (a handful of beginner-level Uni courses) and was wondering about sampling from unknown distributions. Specifically, if you have no idea about the underlying distribution, ...
0
votes
1answer
18 views

Algorithm for sampling points according to weights

In my current problem, I need to sample points in proportion to the weights assigned to them and an original probability density function. Unfortunately, the weights aren't known ahead of time and ...
2
votes
0answers
15 views

Instagram representative sample

I am doing a survey to study both sellers and consumers behaviour on Instagram in Kuwait. My question is how to decide on the sample size? What sampling design should I use since there is no sampling ...
3
votes
2answers
54 views

Estimator for $E[X]^2$

I'm trying to understand the theory of estimators. As I understand it now, if you have an r.v. $X$ and take $n$ i.i.d. samples then an estimator for $E[X^{2}]$ would be $\overline{X^{2}}$ since ...
-1
votes
0answers
28 views

Statistical significance to a population

I have a question in regards to statistically significant sample sizes to a population. I am working with data in excel and wanting to put the formula for this into the workbook as well but have run ...
0
votes
1answer
34 views

sum of correlated random sample

Suppose I have 1000 draws each of two random variables X and Y. If I wanted to sample the sum of these variables, I would simply calculate 1000 samples, i.e. $$ S_{i}=X_{i}+Y_{i}, i=1,2,…,1000 $$ ...
0
votes
1answer
33 views

How to ReSample the Training DataSet

I have a training dataset which differs significantly on several parameters (t-test/f-test) with the test set (for which the labels are not known to me.). I am thinking of re-sampling from the ...
0
votes
0answers
9 views

Estimating cutoff point for sampling using bootstrapping

I'd like to determine a cutoff point for plot sampling. So far, I've sampled intensively and collected fecal pellet data for 13 forested stands. Each stand has 50 plots and the mean pellets per ...
1
vote
1answer
15 views

Obtaining a sample with an given sample (resulting) covariance matrix

Often, we are interested in generating data from a density $ f(x \vert \boldsymbol\theta) $, with data $x$ given some parameter vector $\boldsymbol\theta$. This results in a sample, from which we may ...
1
vote
0answers
21 views

Sampling from a distribution with a margin of error

A survey of a population is taken using sampling. It is determined that 70% prefer option A and 30% prefer option B with a margin of error being 5%. Normally when simulating the process with the ...
1
vote
0answers
13 views

Cluster Sampling problem

I'm using Sharon Lohr's book to self-study, and there is no solution to this problem. I just wanted to make sure I'm computing the values correctly. Question: The new candy Green Globules is being ...
0
votes
0answers
8 views

R: Looking for package to sample from Potts model with unbounded neighbourhood

I am looking for a package that samples from a simplified potts model: $$ p(z) = C(\beta)^{-1} exp\left\{ \beta \sum_{i \sim j} f(z_{i}, z_{j}) \right\} $$ where $z=(z_{i}; i \in V)$ with $V=\{1, 2, ...
2
votes
1answer
21 views

Distribution of Sample Means Compared to Population Mean

Assumptions I have a population (n=5000) and I know everything about it (all point values, mean, standard deviation, etc.) From this, I will sample 1000 items. I can calculate the mean of the sample ...
2
votes
0answers
15 views

Is there an efficient algorithm for sampling from the negative hypergeometric distribution?

I'm writing a small statistics library currently. One of the algorithms I'm implementing has two variants: one that samples the hypergeometric distribution and one that samples the negative ...
0
votes
0answers
12 views

Three stage cluster sampling

I was just curious if anyone knew where to find information on three-stage cluster sampling. There is a significant amount of information on two-stage cluster sampling, but I am unable to find ...
1
vote
1answer
28 views

Metropolis-Hastings sample reusal

I have a posterior distribution from which I calculate some statistics using sampling, for example I calculate expectation. So I draw 1000 samples using Metropolis-Hastings and then I calculate their ...
0
votes
0answers
23 views

How to calculate sample size for poll / ranking

Let's suppose there are Y options to choose from. I'd like to run an experiment with a sample of n participants (from a larger population), each will choose 1 option from the Y alternative according ...
2
votes
1answer
30 views

A non-i.i.d observations in a Bayesian inference problem

Suppose we have a simple Bayesian network which has two variables $x$ and $y$, $x$ is the parent of $y$. We sample $M$ $x$s independently based on $P(x)$, named $x_1,\ldots,x_M$, and for every $x_i$, ...
0
votes
0answers
169 views

Calculate at .05 level of significance

A research organization claims that the monthly wages of industrial workers in district X exceed those in district Y by more than Rs 150. Two different samples drawn independently from the two ...
1
vote
0answers
18 views

Estimating necessary sample size

I am pretty new to statistics and I'd like to get pointers on the correct way to do this. I have a dataset in which I'm interested in the 50th and 90th percentile. I'd like to take a sample of that ...
2
votes
1answer
25 views

Sampling under assumption of log normal distributed data with sample mean and standard deviation

I have the sample mean and the sample standard deviation of income calculated from individual tax data of all citizens in country (let's call this data X). I do not have access to this tax income ...
2
votes
1answer
26 views

Dirichlet Process Hyperparameter Estimation with Sampling

I have a dirichlet process for which I need to learn the concentration (strength) hyperparameter (with gamma prior on it). One way of doing is via maximizing the Likelihood. Another way of doing this ...
1
vote
3answers
69 views

smart sampling techniques in r

I have a large data set of about 1.8 million rows with 80 variables. I would like to find a good technique (code or package) in R that can reduce the amount of training data without damaging the ...
1
vote
1answer
26 views

which statistical sampling technique to employ to have a representative sample

There is kwashiorkor outbreak in my community Abeng (a city in Africa). The community is in sections A to F and it heterogeneous in nature.
2
votes
1answer
35 views

Empirical Distributions

I am working with a small data set that is clearly non-Gaussian. This data is bound within a fairly narrow range. I have been asked to estimate the quantiles of the population that this data is from. ...
5
votes
1answer
205 views

Why don't we use the unbiased sample variance to calculate the standard error?

The standard error is an approximation of the standard deviation of the sampling distribution of the sample means. The real standard deviation of the sampling distribution, $\sigma _{\bar x}$ is: ...
0
votes
1answer
27 views

Reservoir sampling with a computationally expensive weight function

I have a large dataset, and I want to obtain a small sample of it of size K, weighted by a function f(x) which is expensive to compute (I'm ok computing it O(K) times, but not too much more). Suppose ...
1
vote
0answers
8 views

How to calculate MoE for subsamples

I have demographic and income data for 3 million people from the American Community Survey (courtesy of IPUMS), and my goal is calculate the median income for every permutation of age group, gender, ...
0
votes
1answer
55 views

Simple linear regression and sampling

I have a small dataset (60 elements) for which I fit a simple linear regression model, and obtain a small coefficient of determination ($R^2 = 3\%$). I'm a beginner in statistics so I'm trying to ...
1
vote
1answer
19 views

Two-stage cluster sampling problem

I'm having difficulty understanding the following problem. It seems like it is a two stage cluster sampling problem, but then the defective red lightbulbs make the question even more difficult. I'd ...
0
votes
0answers
4 views

How to test data migration quality on very large sample set

I am migrating 2M individual bio-specimens from one IT application to another (sample location data, 5 levels). While the data migration tools are "validated", I want to create a statistical model to ...
1
vote
0answers
104 views

Ensuring exploratory study's validity with pseudo-simple random sampling

The context of my questions is as follows. I'm performing a cross-sectional secondary research study, involving open source software (OSS) projects. I collect data (information about the projects) ...
3
votes
2answers
109 views

Sampling from a product of two Multivariate Gaussians

I have a multivariate Gaussian defined as follows: $$ p(x) = \omega(x)\gamma(x) $$ where $\omega$ and $\gamma$ both are multivariate Gaussians and from which I can sample very efficiently given due to ...
0
votes
0answers
6 views

Data Collection Design

Would like some critique on data selection for testing. I am planning some indoor imagery localization experiment. I have two kinds of data (i) Image coming to me at 25HZ (ii) telemetry coming to me ...
0
votes
0answers
12 views

Sequential weighted sampling

I need to figure out the total path (A to Z) followed by an agent through a squared-element grid. Each grid element has a probability density function $\Delta$ assigned to it that represents the ...
0
votes
2answers
63 views

Stratified or multi-stage sampling?

I just started volunteering with a small organization that wants to estimate the average and total number environmental trouble spots in a large US city. They have a clear definition of an ...
0
votes
0answers
15 views

Summary statistics & CIs for sample variances from multiple populations

Suppose $X_i$ are $n$ random variables that are all distributed normally, but with differing means and standard deviations: $X_i \sim N(\mu_i, \sigma_i)$. Now suppose $y_{ij}$ is the $j$-th random ...
1
vote
1answer
24 views

Sampling frame of a voluntary survey

If one is conducting a survey in which all the respondents are volunteers, would there be a true sampling frame? I see three possibilities: We could view this as a situation where there is no set ...
1
vote
0answers
21 views

How to model the sampling distribution of the sample sum

I'm stuck on a stats problem and am wondering if someone can point out the error in my logic. Imagine you are planning for a camping trip for 50 people. Each person consumes 2.0 lb of food per day on ...
1
vote
1answer
38 views

How to sample from “biased” binomial distribution (ideally in python/numpy)

I need to sample a variable from a distribution that's like a binomial distribution except with a "bias", I'm not sure what it may be called: $p(X=k)$ is proportional to $k.B(n,p)(k)$ where $B(n,p)$ ...
3
votes
1answer
26 views

Estimate a proportion using priority sampling (I just made that up)

I have this idea in my head that is either bunk or has a name I don't know. (I'm not naive enough to think I'm breaking new ground here!) Here's my scenario: I would like to know the proportion of a ...
1
vote
2answers
110 views

How to choose the training, cross-validation, and test set sizes for small sample-size data?

Assume I have a small sample size, e.g. N=100, and two classes. How should I choose the training, cross-validation, and test set sizes for machine learning? I would intuitively pick Training set ...
1
vote
0answers
44 views

Has anyone publicly shared an implementation of RUSBoost in R?

There's no package available on CRAN, so I was hoping someone in the community had written their own function/package. I see it's been done in MATLAB, so I may just have to start with that and write ...
0
votes
0answers
13 views

How to calculate survey sample with more than one outcome variables

I'm working on a proposal and that needs survey design. The basic question is like this: The study seeks to examine factors (e.g., income) associated with the use of health checkup services of workers ...
-1
votes
2answers
44 views

How do you draw a random sample of unique IDs in a large dataset?

Rookie here -- I have a a large data set with about 75,000 observations, and 2000 unique IDs. Therefore, each unique ID has about 37 observations. I'm trying to draw a random sample of unique IDs, say ...