Creating samples from a well-specified population using a probabilistic method and/or producing random numbers from a specified distribution.
2
votes
1answer
89 views
standard error of known population values
i tried to get a clearer understanding of the standard error by constructing a hypothetical population consisting of the following values: 1, 3, 5, 7. i calculated the sample means of all samples with ...
1
vote
0answers
30 views
Metropolis sampling of constraint surface
I have a four variable $(x_1, x_2, v_1, v_2)$ state space for a system of ODEs. I would like to build a random sample of the initial conditions for these ODEs --- in other words, I wish to build a ...
0
votes
0answers
105 views
Bootstrapping Methodology
I have this bootstrap setting:
Given $n$ vector-valued statistics (each statistic is a vector) of dimension $r \times 1$, I generated $1000$ bootstrap samples and then generated an estimate of each ...
1
vote
1answer
29 views
How many measurements (10 million molecules each time) are needed to analyze 15 million unique molecues in a 20 million pool?
This is my homework: There are 20 million DNA molecules in a library for high throughput DNA sequencing, each sequencing run can generate 10 million reads (i.e., analyze 10 millions of DNA molecules), ...
2
votes
0answers
37 views
Fitting models to empirical data: do fits for random samples converge on the true fit?
I'm trying to fit a mathematical/theoretical model to empirical data, but the dataset is impractically big to fit all at once. (Specifically: I'm fitting a power law model using the methods of ...
1
vote
3answers
83 views
Downsampling excessively sampled curves
I'm looking for a quick (event if not so accurate) way to deal with over-sampled curves using R. Consider the following example in which x contains 1000 values in ...
3
votes
1answer
50 views
Mixing User Data For Cross-Validation
I have data from 12 users, and want to perform cross-validation. Is it necessary that I create my training and test data from different users, or can all of the data be randomized, and then split into ...
6
votes
1answer
221 views
How to generate a Bernoulli variate with bias $a/\mathbb{E}[X]$ given a sampler of $X$ and uniform variates?
Given:
A loaded "die" with unknown probabilities generating a discrete, positive random variable $X$ taking on values in $\mathcal{X}$.
A real number $a$, such that $0 \leq a \leq \mathbb{E}[X]$.
...
4
votes
2answers
73 views
Definition of a sample: can it include the same object twice?
Wikipedia defines a sample as:
a subset of a population.
While exploring the reason why we divide by $(n-1)$ instead of $n$ when calculating standard deviation (discussed in this question), I ...
2
votes
0answers
90 views
Gibbs sampling from full conditionals
I have the following joint density:
$p(x_1,x_2,y_1,y_2) \propto \exp\left(−\left(x_1^2+x_2^2+c_1(y_2-y_1)^2+c_2(y_2-y_1)^4\right)\right)$
Can I use Gibbs sampling to sample from that? How can I get ...
2
votes
0answers
42 views
Best method for estimating divergence with unequal sample sizes?
Background
I have several datasets of word frequencies where some datasets have much more data than others: from 500 samples to 3000 samples. I also have large reference corpora with millions of ...
13
votes
4answers
396 views
How to generate a non-integer amount of consecutive Bernoulli successes?
Given:
A coin with unknown bias $p$ (Head).
A strictly positive real $a > 0$.
Problem:
Generate a random Bernoulli variate with bias $p^{a}$.
Does anyone know how to do this? For instance, ...
4
votes
1answer
56 views
Is a chisquare on a (nearly) complete population data necessary?
I have data from a survey that was (attempted to to be) administered to all children of particular grades in a certain state. I am getting it after a cleaning step by the survey designers that removed ...
1
vote
1answer
66 views
Pointers on how to perform a Monte Carlo Analysis
Im looking for some pointers on how to perform a Monte Carlo simulation. Say im conducting a survey of how late flights are for a certain airline. I amass 1000 records, and plot a cumulative sum of ...
2
votes
0answers
28 views
seeking statistical measures to use when evaluating a facial photo's “quality”
I have a large set of facial photos and a facial feature finder which helps me locate faces with a general idea of the location of the eyes, nose, mouth, and a curve outlining the face. These facial ...
0
votes
1answer
48 views
How can you find the probability that an element is part of a sampled set?
(I'm a bit of a newbie, so pardon me if I use incorrect terminology.)
Let's say I have a set of scalar values V that I sampled from a set S. I want to test whether a given value X could be a member ...
1
vote
1answer
75 views
Sampling distribution of means
Im working through a textbook atm and Ive read that the mean of "the sampling distribution of means" is the same as the "populations" mean.
To construct the sampling distribution of means, i need to ...
3
votes
1answer
232 views
Sampling with or without replacement?
I don't know a lot about sampling methods.
I have a large population of size 2,000,000. I used one of those sample size calculators. It says that I need sample size of approximately 10,000.
I am ...
0
votes
0answers
44 views
Variable length word sampling problem
I have a unusual sampling problem.
I have a continuously growing file with millions of words in it. Each word can have 1 to 8 characters in them. A word can have duplicate characters in them. The ...
6
votes
1answer
131 views
Suggested books on spatial statistics
What are some of the best books for studying
i) variability of univariate and multivariate variables (real, count data) across a spatial domain.
ii) sampling a univariate or multivariate variable ...
1
vote
2answers
142 views
Under what circumstances is an N < 30 acceptable?
I am doing some research in which I want to expose subjects to an "unusual" event and record their response. I am recording the subjects' responses via a sensitive brain instrument, so I've been told ...
5
votes
1answer
193 views
Normal distributed random numbers in time - expected behaviour - askew distributions?
I have a 3D array of numbers, initialised with normal distributed values. The mean of the distribution is $0$, the standard deviation in the order of $10^{-5}$ to $10^{-8}$, but would be constant for ...
1
vote
0answers
20 views
Determine likelihood that a list of N items contains no negative results after examining X items
I have a large (N) list of results from a process that is either positive or negative. After examining X of them and receiving only positive results, how certain can I be that the list only contains ...
1
vote
0answers
210 views
Interpreting numerical value of standard error of the mean
I am unclear about how to interpret the value of the Standard Error of the Mean (SEM) directly. For example, when a mean is reported as 5.00 + 0.50SEM, how do you directly relate the 0.50 to 5.00?
To ...
1
vote
0answers
24 views
Probability of being included in a Ranked Set Sample
I am doing a research work which needs a probability calculation. I'll be grateful if the community cross validates the formula that I have derived using the idea of binary strings and following a ...
1
vote
0answers
61 views
Choosing which variable to sample to get a better model
Apologies if this question is long-winded and vague, but I'm really not familiar with the field and I'm having a hard time finding references.
We have a random variable $X$ which we assume is ...
3
votes
0answers
21 views
Distribution of number of times units are chosen for a simple random sample
Simple random sampling with replacement.
Population size = $ N $
Sample size = $ n $
I am interested in learning about the distribution of the random variables
$ X_0, X_1, ..., X_n $, where $ X_i ...
1
vote
0answers
53 views
Sampling Stationary Vector Autoregression coefficients while Gibbs Sampling
I have been estimating a Bayesian Vector Autoregression using Gibbs Sampling. When constructing the posterior predictive distribution, I have noticed that when the simulated coefficients from the MCMC ...
1
vote
1answer
82 views
Computing conditional expectation of ordered normal random variables
There are $m$ normally distributed, independent random variables $N_1, \ldots, N_m$ with distinct means $\mu_1, \ldots \mu_m$ and standard deviations $\sigma_1, \ldots, \sigma_m$. Then, we observe a ...
4
votes
2answers
219 views
In logistic regression, does the lack of significance of the parameter estimates in a test sample indicate overfitting?
I am trying to build a logistic regression model where I have a dependent variable $y$ and independent variables $x_1$, $x_2$... $x_n$. $y$ can take only two values - 0 or 1.
My original modelling ...
1
vote
1answer
54 views
When is blocked Metropolis sampling more efficient?
Consider the problem of sampling from $p(\mathbf{x}, \mathbf{y})$ using the Metropolis or Metropolis-Hastings (MH) algorithm. I can either propose samples for $p(\mathbf{x}, \mathbf{y})$ directly, or ...
2
votes
3answers
2k views
How does the sampling distribution of sample means approximate the population mean?
I am trying to learn statistics because I find that it is so prevalent that it prohibits me from learning somethings if I don't understand it properly. I am having trouble understanding this notion of ...
2
votes
2answers
90 views
Minimizing variance of an estimator under sampling cost penalty
I have an estimator $t$, whose variance depends on the dimension of
my sample $x_{1:n}$:
$$
\text{Var}(t(x_{1:n})) = f(n).
$$
Suppose that the form of $f(n)$ is known. I would like to determine
what ...
1
vote
0answers
50 views
Estimating population size and a proportion
I am interested in the following sampling problem, which I will try to describe by a motivating example.
Suppose we want to estimate how many people in a certain area, has blue eyes, how many have ...
1
vote
3answers
292 views
Pre- post- questionnaire (t-test)
I came across this forum as I am trying to get a convincing answer. Apologies for speaking 'plain English' as I am not a statistical genius!
Basically I am implementing a pre- and post- questionnaire ...
2
votes
0answers
46 views
Estimating mass proportions from fractional samples
Say you are on a beach that is composed of a mixture of different colored sands (for simplicity, black and white) and you wish to estimate how much there is of each type. You have a bucket that you ...
2
votes
1answer
184 views
Applying log-transformation when comparing two populations
I have a survey data containing two samples drawing from two populations (or the same population but at different time series). Based on these two samples, I would like to estimate the total and its ...
2
votes
1answer
229 views
Sample size vs. number of samples
This question came up in an archaeology class, and it seems like it should be something very basic.
Suppose I want to count the number of marbles in a large bucket of sediment. I only have the ...
5
votes
2answers
161 views
Distribution of ratio of sample means from two independent normal variables?
The Question
We have a sample of size $N$ with mean $\bar{x}$ and SD $\bar{\sigma_x}$ from a random variable $X \sim \mathcal{N} (\mu, \sigma^2)$
We have a sample of size $M$ with mean $\bar{y}$ and ...
4
votes
1answer
62 views
Get a representative vector from a large set, and compare it with samples
We're a small team of programmers and we're trying to solve a little problem, but we think we need some advices from professional mathematicians.
We want to know if a picture of a card is an Identity ...
3
votes
1answer
49 views
Sources for sampling in case-control studies
In my previous job as a statistics teacher, I was involved in studies mostly as an advisor on a general level. Not that much hands-on experience from doing the actual sampling or data cleaning, ...
2
votes
0answers
38 views
How to draw a balanced random sample given prediction confidence and model performance?
INTRO
I have trained a SVM model based on 300 training cases in order to build a filter that should help me extract a bigger and evenly balanced training set, that is to be validated by human judges.
...
2
votes
0answers
67 views
Random sampling for estimating mutual information - Time complexity and sampling error?
I have a dataset and I want to compute the mutual information (MI) for a selected set of variables. The dataset is large enough so that computation of the MI may take undesirably long time. Can I just ...
1
vote
0answers
23 views
Mass to area to mass/min
Given an collection of mass values (in $g$) which are samples collected over a minute in a moving-window fashion (thus their sum represents the mass per minute observed in the system), and a formula ...
0
votes
0answers
32 views
Condition in conditioned latin hypercube sampling
I am trying to use conditioned latin hypercube sampling in R (clhs package, see here). Unfortunately, I don't really understand how the cost parameter can be used. ...
3
votes
2answers
295 views
Generating samples from gibbs sampling
I am quite new to sampling. I am doing Gibbs sampling for a Bayesian network. I am aware about the algorithm for the Gibbs sampling but one thing I am not able to understand.
For example let's ...
0
votes
0answers
75 views
Alternative sampling method instead of fractional factorial design
I am trying to do my design of experiment for numerical simulations. Here, i have (eg.) 10 parameters which can be 0 or 1 ...
1
vote
0answers
41 views
Given two biased samplers, create a less biased sampler
Suppose you have two biased sampling algorithms, and you don't know the bias of the algorithms. Can you combine the algorithms somehow and reduce the bias?
Partly my motivation is discretizing ...
1
vote
3answers
93 views
What is the requirement on the instrumental density in importance sampling?
From Rubinstein's Simulation Monte Carlo Method
I was wondering why $g$ is required to dominate $Hf$?
From Wikipedia
The basic idea of importance sampling is to change the probability $P$ so ...
4
votes
0answers
80 views
Regarding the sampling procedure in Adaboost algorithm
The AdaBoost algorithm states that it is to train a classifier based on the training data according to a weight vector.
Assume the size of training data is N, the weight vector is of dimension N as ...
