All Questions
1,933 questions
83
votes
3
answers
105k
views
How is the minimum of a set of IID random variables distributed?
If $X_1, ..., X_n$ are independent identically-distributed random variables, what can be said about the distribution of $\min(X_1, ..., X_n)$ in general?
- 941
78
votes
7
answers
91k
views
Rule of thumb for number of bootstrap samples
I wonder if someone knows any general rules of thumb regarding the number of bootstrap samples one should use, based on characteristics of the data (number of observations, etc.) and/or the variables ...
- 1,201
73
votes
4
answers
191k
views
How do you calculate the probability density function of the maximum of a sample of IID uniform random variables?
Given the random variable
$$Y = \max(X_1, X_2, \ldots, X_n)$$
where $X_i$ are IID uniform variables, how do I calculate the PDF of $Y$?
- 871
69
votes
9
answers
8k
views
Taleb and the Black Swan
Taleb's book "The Black Swan" was a New York Times best seller when it came out several years ago. The book is now in its second edition. After meeting with statisticians at a JSM (an annual ...
- 43.2k
51
votes
1
answer
25k
views
What is the difference between Metropolis-Hastings, Gibbs, Importance, and Rejection sampling?
I have been trying to learn MCMC methods and have come across Metropolis-Hastings, Gibbs, Importance, and Rejection sampling. While some of these differences are obvious, i.e., how Gibbs is a special ...
- 2,425
51
votes
2
answers
24k
views
Can somebody explain to me NUTS in english?
My understanding of the algorithm is the following:
No U-Turn Sampler (NUTS) is a Hamiltonian Monte Carlo Method. This means that it is not a Markov Chain method and thus, this algorithm avoids the ...
- 641
49
votes
5
answers
41k
views
K-fold vs. Monte Carlo cross-validation
I am trying to learn various cross validation methods, primarily with intention to apply to supervised multivariate analysis techniques. Two I have come across are K-fold and Monte Carlo cross-...
- 633
46
votes
9
answers
21k
views
Approximate $e$ using Monte Carlo Simulation
I've been looking at Monte Carlo simulation recently, and have been using it to approximate constants such as $\pi$ (circle inside a rectangle, proportionate area).
However, I'm unable to think of a ...
42
votes
1
answer
9k
views
How to determine significant principal components using bootstrapping or Monte Carlo approach?
I am interested in determining the number of significant patterns coming out of a Principal Component Analysis (PCA) or Empirical Orthogonal Function (EOF) Analysis. I am particularly interested in ...
- 3,742
39
votes
7
answers
9k
views
Are all simulation methods some form of Monte Carlo?
Is there a simulation method that is not Monte Carlo? All simulation methods involve substituting random numbers into the function to find a range of values for the function. So are all simulation ...
- 6,635
34
votes
6
answers
11k
views
Generating random numbers manually
How can I manually generate a random number from a given distribution, as for instance, 10 realisations from the standard normal distribution?
- 480
32
votes
3
answers
17k
views
Extreme Value Theory - Show: Normal to Gumbel
The Maximum of $X_1,\dots,X_n. \sim$ i.i.d. Standardnormals converges to the Standard Gumbel Distribution according to Extreme Value Theory.
How can we show that?
We have
$$P(\max X_i \leq x) = P(...
- 1,271
31
votes
3
answers
37k
views
Would a Random Forest with multiple outputs be possible/practical?
Random Forests (RFs) is a competitive data modeling/mining method.
An RF model has one output -- the output/prediction variable.
The naive approach to modeling multiple outputs with RFs would be
to ...
- 878
30
votes
5
answers
3k
views
What are examples of statistical experiments that allow the calculation of the golden ratio?
There are some very simple experiences that can be done by a kid at home, whose result allows one to statistically approach famous numbers such as $\pi$ or $e$.
An example where $\pi$ shows up is ...
- 1,733
29
votes
5
answers
4k
views
Why use Monte Carlo method instead of a simple grid?
when integrating a function or in complex simulations, I have seen the Monte Carlo method is widely used. I'm asking myself why one doesn't generate a grid of points to integrate a function instead of ...
- 4,321
27
votes
2
answers
14k
views
What is importance sampling?
I'm trying to learn reinforcement learning and this topic is really confusing to me. I have taken an introduction to statistics, but I just couldn't understand this topic intuitively.
- 461
27
votes
5
answers
7k
views
Why is the term "Monte Carlo simulation" used instead of "Random simulation"? [duplicate]
I always read/hear "Monte Carlo" simulations. I have done "Monte Carlo" simulations before to calculate the odds in certain gambling games as part of my job and it was nothing more than basically ...
- 383
26
votes
6
answers
53k
views
Can mean plus one standard deviation exceed maximum value?
I have mean 74.10 and standard deviation 33.44 for a sample that has minimum 0 and maximum 94.33.
My professor asks me how can mean plus one standard deviation exceed the maximum.
I showed her ...
- 261
26
votes
2
answers
25k
views
When are Monte Carlo methods preferred over temporal difference ones?
I've been doing a lot of research about Reinforcement Learning lately. I followed Sutton & Barto's Reinforcement Learning: An Introduction for most of this.
I know what Markov Decision Processes ...
- 363
25
votes
2
answers
2k
views
Which distribution has its maximum uniformly distributed?
Let's consider $Y_n$ the max of $n$ iid samples $X_i$ of the same distribution:
$Y_n = max(X_1, X_2, ..., X_n)$
Do we know some common distributions for $X$ such that $Y$ is uniformly distributed $U(a,...
- 403
25
votes
2
answers
1k
views
Fitting custom distributions by MLE
My question relates to fitting custom distributions in R but I feel it has enough of a probability element to remain on CV.
I have an interesting set of data which has the following characteristics:
...
- 2,911
24
votes
2
answers
11k
views
Distribution of the maximum of two correlated normal variables
Say I have two standard normal random variables $X_1$ and $X_2$ that are jointly
normal with correlation coefficient $r$.
What is the distribution function of $\max(X_1, X_2)$?
- 2,295
24
votes
5
answers
5k
views
Why use extreme value theory?
I'm coming from Civil Engineering, in which we use Extreme Value Theory, like GEV distribution to predict the value of certain events, like The biggest wind speed, i.e the value that 98.5% of the wind ...
- 1,285
24
votes
4
answers
8k
views
Can Machine Learning or Deep Learning algorithms be utilised to "improve" the sampling process of a MCMC technique?
Based on the little knowledge that I have on MCMC (Markov chain Monte Carlo) methods, I understand that sampling is a crucial part of the aforementioned technique. The most commonly used sampling ...
- 584
24
votes
6
answers
48k
views
Why doesn't k-means give the global minimum?
I read that the k-means algorithm only converges to a local minimum and not to a global minimum. Why is this? I can logically think of how initialization could affect the final clustering and there is ...
- 589
24
votes
2
answers
10k
views
What are some techniques for sampling two correlated random variables?
What are some techniques for sampling two correlated random variables:
if their probability
distributions are parameterized
(e.g., log-normal)
if they have non-parametric
distributions.
The data are ...
- 617
23
votes
3
answers
3k
views
Distribution of the largest fragment of a broken stick (spacings)
Let a stick of length 1 be broken in $k+1$ fragments uniformly at random. What is the distribution of the length of the longest fragment?
More formally, let $(U_1, \ldots U_k)$ be IID $U(0,1)$, and ...
- 14.9k
21
votes
5
answers
2k
views
Let X,Y be 2 r.v. with infinite expectations, are there possibilities where min(X,Y) have finite expectation?
If it is impossible, what is the proof?
- 542
21
votes
4
answers
8k
views
Posterior distribution and MCMC [duplicate]
I have read something like 6 articles on Markov Chain Monte carlo methods, there are a couple of basic points I can't seem to wrap my head around.
How can you "draw samples from the posterior ...
- 681
21
votes
4
answers
22k
views
Bootstrap vs Monte Carlo, error estimation
I'm reading the article Error propagation by the Monte Carlo method in geochemical calculations, Anderson (1976) and there's something I don't quite understand.
Consider some measured data $\{A\pm\...
- 4,362
21
votes
1
answer
10k
views
Why do temporal difference (TD) methods have lower variance than Monte Carlo methods?
In many reinforcement learning papers, it is stated that for estimating the value function, one of the advantages of using temporal difference methods over the Monte Carlo methods is that they have a ...
- 330
21
votes
1
answer
494
views
How can we simulate from a geometric mixture?
If $f_1,\ldots,f_k$ are known densities from which I can simulate, i.e., for which an algorithm is available. and if the product $$\prod_{i=1}^k f_i(x)^{\alpha_i}\qquad \alpha_1,\ldots,\alpha_k>0$$ ...
- 108k
21
votes
2
answers
2k
views
How can we bound the probability that a random variable is maximal?
$\newcommand{\P}{\mathbb{P}}$Suppose we have $N$ independent random variables $X_1$, $\ldots$, $X_n$ with finite means $\mu_1 \leq \ldots \leq \mu_N$ and variances $\sigma_1^2$, $\ldots$, $\sigma_N^2$....
- 738
20
votes
1
answer
12k
views
MCMC on a bounded parameter space?
I am trying to apply MCMC on a problem, but my priors(in my case they are $\alpha\in[0,1],\beta\in[0,1]$)) are restricted to an area? Can I use normal MCMC and ignore the samples that fall outside of ...
- 1,615
20
votes
2
answers
8k
views
Why does thinning work in Bayesian inference?
In Bayesian inference, one needs to determine the posterior distribution of the parameters from the prior distribution and the likelihood of the data. As this computation might not be possible ...
- 285
19
votes
2
answers
12k
views
What is the variance of the maximum of a sample?
I'm looking for bounds on the variance of the maximum of a set of random variables. In other words, I'm looking for closed-form formulas for $B$, such that
$$
\mbox{Var}(\max_i X_i) \leq B \enspace,
$$...
- 273
18
votes
3
answers
11k
views
Generating random points uniformly on a disk [duplicate]
I have to randomly generate 1000 points over a unit disk such that are uniformly distributed on this disk. Now, for that, I select a radius $r$ and angular orientation $\alpha$ such that the radius $r$...
- 357
18
votes
1
answer
6k
views
Sampling from marginal distribution using conditional distribution?
I want to sample from a univariate density $f_X$ but I only know the relationship:
$$f_X(x) = \int f_{X\vert Y}(x\vert y)f_Y(y) dy.$$
I want to avoid the use of MCMC (directly on the integral ...
- 418
17
votes
3
answers
2k
views
Generate points efficiently between unit circle and unit square
I'd like generate samples from the blue region defined here:
The naive solution is to use rejection sampling in the unit square, but this provides only a $1-\pi/4$ (~21.4%) efficiency.
Is there ...
- 12.3k
17
votes
2
answers
1k
views
Why does this algorithm generate a standard normal distribution?
I have this algorithm which I encountered:
(1) Generate $U_1$, $U_2$ independently from Uniform(0,1)
(2) Set $Y_1 = -\log{U_1}, Y_2 = -\log{U_2}$. If $Y_2 > \frac{(1-Y_1)^2}{2}$, accept $(Y_1, Y_2)$...
- 173
17
votes
1
answer
2k
views
Metropolis-Hastings integration - why isn't my strategy working?
Assume I have a function $g(x)$ that I want to integrate
$$ \int_{-\infty}^\infty g(x) dx.$$
Of course assuming $g(x)$ goes to zero at the endpoints, no blowups, nice function. One way that I've been ...
- 1,247
17
votes
1
answer
2k
views
Hamiltonian Monte Carlo: how to make sense of the Metropolis-Hasting proposal?
I am trying to understand the inner working of Hamiltonian Monte Carlo (HMC), but can't fully understand the part when we replace the deterministic time-integration with a Metropolis-Hasting proposal. ...
- 799
17
votes
1
answer
4k
views
Scrambling and correlation in low discrepancy sequences (Halton/Sobol)
I am currently working on a project where I generate random values using low discrepancy / quasi-random point sets, such as Halton and Sobol point sets. These are essentially $d$-dimensional vectors ...
- 5,145
16
votes
2
answers
2k
views
What is the connection between Markov chain and Markov chain monte carlo
I am trying to understand Markov chains using SAS. I understand that a Markov process is one where the future state depends only on the current state and not on the past state and there is a ...
- 6,635
15
votes
3
answers
23k
views
How to create a toy survival (time to event) data with right censoring
I wish to create a toy survival (time to event) data which is right censored and follows some distribution with proportional hazards and constant baseline hazard.
I created the data as follows, but I ...
- 459
15
votes
2
answers
21k
views
What is the distribution for the maximum (minimum) of two independent normal random variables?
Specifically, suppose $X$ and $Y$ are normal random variables (independent but not necessarily identically distributed). Given any particular $a$, is there a nice formula for $P(\max(X,Y)\leq x)$ or ...
- 355
15
votes
2
answers
2k
views
What are some important uses of random number generation in computational statistics?
How and why are random number generators (RNGs) important in computational statistics?
I understand that randomness is important when choosing samples for many statistical tests to avoid bias towards ...
15
votes
1
answer
10k
views
Finding local extrema of a density function using splines
I am trying to find the local maxima for a probability density function (found using R's density method). I cannot do a simple "look around neighbors" method (where ...
- 373
14
votes
5
answers
10k
views
Is Matlab/octave or R better suited for monte carlo simulation?
I started to do Monte Carlo in R as a hobby, but eventually a financial analyst advised to migrate to Matlab.
I'm an experienced software developer.
but a Monte Carlo beginner.
I want to construct ...
- 681
14
votes
4
answers
8k
views
Bootstrap, Monte Carlo
I have been set the following question as part of homework:
Design and implement a simulation study to examine the performance of the bootstrap for obtaining 95% confidence intervals on the mean of a ...
- 141