Creating samples from a well-specified population using a probabilistic method and/or producing random numbers from a specified distribution.

learn more… | top users | synonyms

2
votes
1answer
13 views

Sampling under assumption of log normal distributed data with sample mean and standard deviation

I have the sample mean and the sample standard deviation of income calculated from individual tax data of all citizens in country (let's call this data X). I do not have access to this tax income ...
2
votes
1answer
16 views

Dirichlet Process Hyperparameter Estimation with Sampling

I have a dirichlet process for which I need to learn the concentration (strength) hyperparameter (with gamma prior on it). One way of doing is via maximizing the Likelihood. Another way of doing this ...
0
votes
3answers
57 views

smart sampling techniques in r

I have a large data set of about 1.8 million rows with 80 variables. I would like to find a good technique (code or package) in R that can reduce the amount of training data without damaging the ...
1
vote
1answer
23 views

which statistical sampling technique to employ to have a representative sample

There is kwashiorkor outbreak in my community Abeng (a city in Africa). The community is in sections A to F and it heterogeneous in nature.
1
vote
0answers
23 views

Empirical Distributions

I am working with a small data set that is clearly non-Gaussian. This data is bound within a fairly narrow range. I have been asked to estimate the quantiles of the population that this data is from. ...
5
votes
1answer
195 views

Why don't we use the unbiased sample variance to calculate the standard error?

The standard error is an approximation of the standard deviation of the sampling distribution of the sample means. The real standard deviation of the sampling distribution, $\sigma _{\bar x}$ is: ...
0
votes
0answers
24 views

Reservoir sampling with a computationally expensive weight function

I have a large dataset, and I want to obtain a small sample of it of size K, weighted by a function f(x) which is expensive to compute (I'm ok computing it O(K) times, but not too much more). Suppose ...
1
vote
0answers
7 views

How to calculate MoE for subsamples

I have demographic and income data for 3 million people from the American Community Survey (courtesy of IPUMS), and my goal is calculate the median income for every permutation of age group, gender, ...
0
votes
1answer
47 views

Simple linear regression and sampling

I have a small dataset (60 elements) for which I fit a simple linear regression model, and obtain a small coefficient of determination ($R^2 = 3\%$). I'm a beginner in statistics so I'm trying to ...
1
vote
1answer
13 views

Two-stage cluster sampling problem

I'm having difficulty understanding the following problem. It seems like it is a two stage cluster sampling problem, but then the defective red lightbulbs make the question even more difficult. I'd ...
0
votes
0answers
3 views

How to test data migration quality on very large sample set

I am migrating 2M individual bio-specimens from one IT application to another (sample location data, 5 levels). While the data migration tools are "validated", I want to create a statistical model to ...
-1
votes
0answers
29 views

tutorial on sampling methods and MC

I'm looking for good tutorials that cover the various sampling methods: simple sampling, MCMC, Gibbs Sampling, and Metropolis Hastings Algorithm. I barely know what is an MCMC. I would like to learn ...
1
vote
0answers
97 views

Ensuring exploratory study's validity with pseudo-simple random sampling

The context of my questions is as follows. I'm performing a cross-sectional secondary research study, involving open source software (OSS) projects. I collect data (information about the projects) ...
3
votes
2answers
105 views

Sampling from a product of two Multivariate Gaussians

I have a multivariate Gaussian defined as follows: $$ p(x) = \omega(x)\gamma(x) $$ where $\omega$ and $\gamma$ both are multivariate Gaussians and from which I can sample very efficiently given due to ...
0
votes
0answers
6 views

Data Collection Design

Would like some critique on data selection for testing. I am planning some indoor imagery localization experiment. I have two kinds of data (i) Image coming to me at 25HZ (ii) telemetry coming to me ...
0
votes
0answers
12 views

Sequential weighted sampling

I need to figure out the total path (A to Z) followed by an agent through a squared-element grid. Each grid element has a probability density function $\Delta$ assigned to it that represents the ...
0
votes
2answers
55 views

Stratified or multi-stage sampling?

I just started volunteering with a small organization that wants to estimate the average and total number environmental trouble spots in a large US city. They have a clear definition of an ...
0
votes
0answers
12 views

Summary statistics & CIs for sample variances from multiple populations

Suppose $X_i$ are $n$ random variables that are all distributed normally, but with differing means and standard deviations: $X_i \sim N(\mu_i, \sigma_i)$. Now suppose $y_{ij}$ is the $j$-th random ...
1
vote
1answer
22 views

Sampling frame of a voluntary survey

If one is conducting a survey in which all the respondents are volunteers, would there be a true sampling frame? I see three possibilities: We could view this as a situation where there is no set ...
1
vote
0answers
21 views

How to model the sampling distribution of the sample sum

I'm stuck on a stats problem and am wondering if someone can point out the error in my logic. Imagine you are planning for a camping trip for 50 people. Each person consumes 2.0 lb of food per day on ...
1
vote
1answer
35 views

How to sample from “biased” binomial distribution (ideally in python/numpy)

I need to sample a variable from a distribution that's like a binomial distribution except with a "bias", I'm not sure what it may be called: $p(X=k)$ is proportional to $k.B(n,p)(k)$ where $B(n,p)$ ...
3
votes
1answer
23 views

Estimate a proportion using priority sampling (I just made that up)

I have this idea in my head that is either bunk or has a name I don't know. (I'm not naive enough to think I'm breaking new ground here!) Here's my scenario: I would like to know the proportion of a ...
1
vote
2answers
66 views

How to choose the training, cross-validation, and test set sizes for small sample-size data?

Assume I have a small sample size, e.g. N=100, and two classes. How should I choose the training, cross-validation, and test set sizes for machine learning? I would intuitively pick Training set ...
1
vote
0answers
38 views

Has anyone publicly shared an implementation of RUSBoost in R?

There's no package available on CRAN, so I was hoping someone in the community had written their own function/package. I see it's been done in MATLAB, so I may just have to start with that and write ...
0
votes
0answers
13 views

How to calculate survey sample with more than one outcome variables

I'm working on a proposal and that needs survey design. The basic question is like this: The study seeks to examine factors (e.g., income) associated with the use of health checkup services of workers ...
-1
votes
2answers
41 views

How do you draw a random sample of unique IDs in a large dataset?

Rookie here -- I have a a large data set with about 75,000 observations, and 2000 unique IDs. Therefore, each unique ID has about 37 observations. I'm trying to draw a random sample of unique IDs, say ...
0
votes
1answer
65 views

Is there an equivalent to LHS for a discrete input space?

I was wondering if there is a method that is equivalent to Latin Hypercube Sampling when the input space you are trying to sample from are a finite discrete set of possible values. For example, if I ...
4
votes
2answers
61 views

Do optimization techniques map to sampling techniques?

From any generic sampling algorithm, one can derive an optimization algorithm. Indeed, to maximize an arbitrary function $f: \textbf{x} \rightarrow f(\textbf{x})$, it suffices to draw samples from $g ...
0
votes
0answers
7 views

Missing values on clinical scoring system in SPSS

I have a SPSS database with 747 patients with sickness symptoms and lab results as variables. With those symptoms and results I have generate two scoring systems. I sum the scores of all the variables ...
1
vote
0answers
28 views

ANOVA's and reaction time data

I've been reading a lot on the effect of perceptual load on selective attention but the analysis confuses me somewhat. Some of these studies flag 20-40ms differences between conditions as significant ...
0
votes
0answers
17 views

sampling plan for various number records

I have a set of data. For example, 10,000 observations. Each set has the same format. Each set has 3 to 10,000 records. The question is, what kind of sampling method is reasonable to do with this set ...
0
votes
0answers
8 views

unbiased sampling of expecation over maximization operator

My problem setting is as follows, I have a set of M random variable X = \{ {X_1},{X_2},...{X_M}\} where each variable X_i is estimated via stochastic sequence ...
2
votes
1answer
35 views

How to interpret if my sample statistic is way out in the tail of the bootstrap distribution

I use bootstrapping to generate the distribution / histogram of my sample statistic and find out that the value of my real sample statistic is way out in the tail. What does this mean? Does it mean ...
0
votes
1answer
29 views

Multiple measurements of a single datapoint: what to do with variability?

I am interested in a general question about research design, but I will explain it on a specific example. Suppose I want to determine if one group of people has higher heart rate than the other ...
0
votes
0answers
8 views

sample generative model from a chain/tree

I have a tree of states and I would like to sample from this tree based on pure birth process; however, I don't know how exactly I can do this; so far I have done this; I simplified my problem; the ...
3
votes
0answers
28 views

Sampling methods and parallelization

A couple of years ago I learned about recent work in parallelizing slice sampling methods. More recently, I have read great things about NUTS and Hamiltonian Monte Carlo methods (HMC) in general (e.g. ...
1
vote
0answers
22 views

How are deterministic/logical nodes represented mathematically in directed graphical models?

Various software for performing inference on graphical models support logical(a.k.a deterministic) nodes. PyMc, Winbugs, Smile are the ones I'm aware of. Based on the different inference methods ...
2
votes
1answer
42 views

Is Slice Sampling a special case of Gibbs Sampling?

I read on this thread the following: If you can use both the gibbs sampler and slice sampling to sample from a posterior I would use the Gibbs sampler as the slice sampler seems unnecessary to ...
1
vote
1answer
17 views

choosing one pair randomly from repeated pairs

I have two vectors, in my case all states in the US, and another vector, which indicates the states that it borders. States in vector 1 repeat for each state it borders. For instance, there would be 3 ...
2
votes
1answer
43 views

Rejection-Sampling of Exponential Distribution

Consider the following question. Consider the generation of random numbers following an exponential distribution with some known mean. Give three reasons why the rejection-acceptance method would ...
0
votes
1answer
44 views

Does Normal distribution theory originate in the psychometric literature, especially reliability theory?

The basic statistical literature does not talk about the exact background of the normal distribution. Is the basis of this assumption in psychometry or it has an origin in pure statistics i.e. ...
1
vote
1answer
115 views

Gibbs Sampling Detecting Change point in time series

I was reading through this one page paper on using Gibbs sampling for detecting a change point in a time series like data. While I understand the part where the $\lambda$ and $\phi$ are chosen from a ...
3
votes
4answers
274 views

How exactly does null hypothesis testing work

I'm trying to gain a better understanding of how null hypothesis testing works. I have 3 questions related to the code below: Am I right in saying the probability of each t value in ...
1
vote
1answer
75 views

Sampling from a lognormal distribution

Suppose we are given $\mu$ and $\sigma$ for a lognormal distribution with random variable $X$. $\mu$ is the mean of the variable's logarithm and $\sigma$ is the standard deviation of the variable's ...
0
votes
0answers
19 views

Finding the fewest needed samples for a regression method

I have a regression method that I have applied with success to tens of thousands of well-observed objects. I'm looking to estimate how well it works on poorly observed objects. I selected 50 objects ...
1
vote
0answers
68 views

How to Improve the Design of a Survey of Multiple Rare Variables Using Rare Variable Location Information

I am attempting to update the design of an annual survey which measures the percent-deficient of 26 different asset inventories. Most of these assets don't occur on every sampling unit and some of ...
0
votes
0answers
21 views

Learning theory for search data?

Has learning theory ever been applied in practice for search log data? If so, what are some findings about generalization/learnability from this data? I'm interested in generalization about an ...
3
votes
1answer
30 views

How to sample from the distribution of a Gaussian scale parameter

I would like to be able to sample the standard deviation of a multidimensional Gaussian distribution of dimension $n$; that is, given some $\phi$, I would like to sample $P(\sigma | \phi) \propto ...
2
votes
0answers
35 views

Gibbs sample from AR(1) of exogenous input

I am trying to fit a model where there is a sequence of exogenous "shocks", $X_1, X_2, ..., X_T$, and a AR(1) of these shocks explain $Y_1, Y_2, ..., Y_T$. Specifically, Data (known): $X_1, X_2, ...