Questions tagged [monte-carlo]
Using (pseudo-)random numbers and the Law of Large Numbers to simulate the random behavior of a real system.
1,280
questions
65
votes
7
answers
71k
views
Rule of thumb for number of bootstrap samples
I wonder if someone knows any general rules of thumb regarding the number of bootstrap samples one should use, based on characteristics of the data (number of observations, etc.) and/or the variables ...
- 1,051
47
votes
1
answer
22k
views
What is the difference between Metropolis-Hastings, Gibbs, Importance, and Rejection sampling?
I have been trying to learn MCMC methods and have come across Metropolis-Hastings, Gibbs, Importance, and Rejection sampling. While some of these differences are obvious, i.e., how Gibbs is a special ...
- 2,285
46
votes
9
answers
18k
views
Approximate $e$ using Monte Carlo Simulation
I've been looking at Monte Carlo simulation recently, and have been using it to approximate constants such as $\pi$ (circle inside a rectangle, proportionate area).
However, I'm unable to think of a ...
46
votes
5
answers
38k
views
K-fold vs. Monte Carlo cross-validation
I am trying to learn various cross validation methods, primarily with intention to apply to supervised multivariate analysis techniques. Two I have come across are K-fold and Monte Carlo cross-...
- 603
46
votes
2
answers
16k
views
Can somebody explain to me NUTS in english?
My understanding of the algorithm is the following:
No U-Turn Sampler (NUTS) is a Hamiltonian Monte Carlo Method. This means that it is not a Markov Chain method and thus, this algorithm avoids the ...
- 591
42
votes
1
answer
9k
views
How to determine significant principal components using bootstrapping or Monte Carlo approach?
I am interested in determining the number of significant patterns coming out of a Principal Component Analysis (PCA) or Empirical Orthogonal Function (EOF) Analysis. I am particularly interested in ...
- 3,682
37
votes
7
answers
8k
views
Are all simulation methods some form of Monte Carlo?
Is there a simulation method that is not Monte Carlo? All simulation methods involve substituting random numbers into the function to find a range of values for the function. So are all simulation ...
- 6,285
32
votes
5
answers
10k
views
Generating random numbers manually
How can I manually generate a random number from a given distribution, as for instance, 10 realisations from the standard normal distribution?
- 460
30
votes
5
answers
2k
views
What are examples of statistical experiments that allow the calculation of the golden ratio?
There are some very simple experiences that can be done by a kid at home, whose result allows one to statistically approach famous numbers such as $\pi$ or $e$.
An example where $\pi$ shows up is ...
- 1,648
28
votes
5
answers
4k
views
Why use Monte Carlo method instead of a simple grid?
when integrating a function or in complex simulations, I have seen the Monte Carlo method is widely used. I'm asking myself why one doesn't generate a grid of points to integrate a function instead of ...
- 4,261
28
votes
3
answers
32k
views
Would a Random Forest with multiple outputs be possible/practical?
Random Forests (RFs) is a competitive data modeling/mining method.
An RF model has one output -- the output/prediction variable.
The naive approach to modeling multiple outputs with RFs would be
to ...
- 848
27
votes
2
answers
14k
views
What is importance sampling?
I'm trying to learn reinforcement learning and this topic is really confusing to me. I have taken an introduction to statistics, but I just couldn't understand this topic intuitively.
- 461
26
votes
5
answers
6k
views
Why is the term "Monte Carlo simulation" used instead of "Random simulation"? [duplicate]
I always read/hear "Monte Carlo" simulations. I have done "Monte Carlo" simulations before to calculate the odds in certain gambling games as part of my job and it was nothing more than basically ...
- 373
22
votes
4
answers
7k
views
Can Machine Learning or Deep Learning algorithms be utilised to "improve" the sampling process of a MCMC technique?
Based on the little knowledge that I have on MCMC (Markov chain Monte Carlo) methods, I understand that sampling is a crucial part of the aforementioned technique. The most commonly used sampling ...
- 564
22
votes
2
answers
8k
views
What are some techniques for sampling two correlated random variables?
What are some techniques for sampling two correlated random variables:
if their probability
distributions are parameterized
(e.g., log-normal)
if they have non-parametric
distributions.
The data are ...
- 597
21
votes
2
answers
20k
views
When are Monte Carlo methods preferred over temporal difference ones?
I've been doing a lot of research about Reinforcement Learning lately. I followed Sutton & Barto's Reinforcement Learning: An Introduction for most of this.
I know what Markov Decision Processes ...
- 313
21
votes
1
answer
461
views
How can we simulate from a geometric mixture?
If $f_1,\ldots,f_k$ are known densities from which I can simulate, i.e., for which an algorithm is available. and if the product $$\prod_{i=1}^k f_i(x)^{\alpha_i}\qquad \alpha_1,\ldots,\alpha_k>0$$ ...
- 99.6k
20
votes
4
answers
18k
views
Bootstrap vs Monte Carlo, error estimation
I'm reading the article Error propagation by the Monte Carlo method in geochemical calculations, Anderson (1976) and there's something I don't quite understand.
Consider some measured data $\{A\pm\...
- 3,420
20
votes
1
answer
11k
views
MCMC on a bounded parameter space?
I am trying to apply MCMC on a problem, but my priors(in my case they are $\alpha\in[0,1],\beta\in[0,1]$)) are restricted to an area? Can I use normal MCMC and ignore the samples that fall outside of ...
- 1,605
19
votes
4
answers
7k
views
Posterior distribution and MCMC [duplicate]
I have read something like 6 articles on Markov Chain Monte carlo methods, there are a couple of basic points I can't seem to wrap my head around.
How can you "draw samples from the posterior ...
- 661
19
votes
1
answer
7k
views
Why do temporal difference (TD) methods have lower variance than Monte Carlo methods?
In many reinforcement learning papers, it is stated that for estimating the value function, one of the advantages of using temporal difference methods over the Monte Carlo methods is that they have a ...
- 310
17
votes
3
answers
2k
views
Generate points efficiently between unit circle and unit square
I'd like generate samples from the blue region defined here:
The naive solution is to use rejection sampling in the unit square, but this provides only a $1-\pi/4$ (~21.4%) efficiency.
Is there ...
- 11.9k
17
votes
2
answers
6k
views
Why does thinning work in Bayesian inference?
In Bayesian inference, one needs to determine the posterior distribution of the parameters from the prior distribution and the likelihood of the data. As this computation might not be possible ...
- 255
17
votes
1
answer
2k
views
Metropolis-Hastings integration - why isn't my strategy working?
Assume I have a function $g(x)$ that I want to integrate
$$ \int_{-\infty}^\infty g(x) dx.$$
Of course assuming $g(x)$ goes to zero at the endpoints, no blowups, nice function. One way that I've been ...
- 1,227
17
votes
1
answer
5k
views
Sampling from marginal distribution using conditional distribution?
I want to sample from a univariate density $f_X$ but I only know the relationship:
$$f_X(x) = \int f_{X\vert Y}(x\vert y)f_Y(y) dy.$$
I want to avoid the use of MCMC (directly on the integral ...
- 408
17
votes
1
answer
4k
views
Scrambling and correlation in low discrepancy sequences (Halton/Sobol)
I am currently working on a project where I generate random values using low discrepancy / quasi-random point sets, such as Halton and Sobol point sets. These are essentially $d$-dimensional vectors ...
- 4,685
16
votes
3
answers
7k
views
Generating random points uniformly on a disk [duplicate]
I have to randomly generate 1000 points over a unit disk such that are uniformly distributed on this disk. Now, for that, I select a radius $r$ and angular orientation $\alpha$ such that the radius $r$...
- 327
16
votes
2
answers
2k
views
What is the connection between Markov chain and Markov chain monte carlo
I am trying to understand Markov chains using SAS. I understand that a Markov process is one where the future state depends only on the current state and not on the past state and there is a ...
- 6,285
16
votes
2
answers
1k
views
Why does this algorithm generate a standard normal distribution?
I have this algorithm which I encountered:
(1) Generate $U_1$, $U_2$ independently from Uniform(0,1)
(2) Set $Y_1 = -\log{U_1}, Y_2 = -\log{U_2}$. If $Y_2 > \frac{(1-Y_1)^2}{2}$, accept $(Y_1, Y_2)$...
- 163
16
votes
1
answer
2k
views
Hamiltonian Monte Carlo: how to make sense of the Metropolis-Hasting proposal?
I am trying to understand the inner working of Hamiltonian Monte Carlo (HMC), but can't fully understand the part when we replace the deterministic time-integration with a Metropolis-Hasting proposal. ...
- 769
15
votes
3
answers
21k
views
How to create a toy survival (time to event) data with right censoring
I wish to create a toy survival (time to event) data which is right censored and follows some distribution with proportional hazards and constant baseline hazard.
I created the data as follows, but I ...
- 449
15
votes
2
answers
2k
views
What are some important uses of random number generation in computational statistics?
How and why are random number generators (RNGs) important in computational statistics?
I understand that randomness is important when choosing samples for many statistical tests to avoid bias towards ...
14
votes
5
answers
10k
views
Is Matlab/octave or R better suited for monte carlo simulation?
I started to do Monte Carlo in R as a hobby, but eventually a financial analyst advised to migrate to Matlab.
I'm an experienced software developer.
but a Monte Carlo beginner.
I want to construct ...
- 681
14
votes
2
answers
1k
views
Unbiased estimator of exponential of measure of a set?
Suppose we have a (measurable and suitably well-behaved) set $S\subseteq B\subset\mathbb R^n$, where $B$ is compact. Moreover, suppose we can draw samples from the uniform distribution over $B$ wrt ...
- 799
14
votes
4
answers
8k
views
Best way to seed N independent random number generators from 1 value
In my program I need to run N separate threads each with their own RNG which is used to sample a large dataset. I need to be able to seed this entire process with a single value so I can reproduce ...
- 243
14
votes
1
answer
3k
views
Why use the parametric bootstrap?
I am currently trying to get my head around some things concerning parametric bootstrap. Most things are probably trivial but I still think I may have missed something.
Suppose I want to get ...
- 484
13
votes
6
answers
4k
views
How should one approch Project Euler problem 213 ("Flea Circus")?
I would like to solve Project Euler 213 but don't know where to start because I'm a layperson in the field of Statistics, notice that an accurate answer is required so the Monte Carlo method won't ...
- 233
13
votes
1
answer
4k
views
Monte carlo simulation in R
I am trying to solve the following exercise but I actually have no clue on how to start doing this. I've found some code in my book that looks like it but it's a completely different exercise and I ...
- 133
13
votes
3
answers
19k
views
Simulation study: how to choose the number of iterations?
I would like to generate data with "Model 1" and fit them with "Model 2". The underlying idea is to investigate robustness properties of "Model 2". I am particularly interested in the coverage rate of ...
- 2,089
13
votes
2
answers
11k
views
What is the difference between the Monte Carlo (MC) and Monte Carlo Markov Chain (MCMC) method?
The goal of both methods seems to be to derive an estimate of a posterior/target distribution. If a process model exists which links some input parameters (which are themselves uncertain and can be ...
- 133
13
votes
2
answers
2k
views
Results on Monte Carlo estimates produced by importance sampling
I have been working on importance sampling fairly closely for the past year and have a few open-ended questions that I was hoping to get some help with.
My practical experience with importance ...
- 4,685
13
votes
2
answers
1k
views
For Hamiltonian Monte Carlo, why does negating the momentum variables result in a symmetric proposal?
I have been going through Radford Neal's excellent HMC book chapter in detail. However, there is one detail that I'm really obsessing with now, and I'm not sure if I'm thinking about it right. When ...
13
votes
3
answers
2k
views
Hamiltonian/Hybrid MCMC 'mass matrix' terminology
I am trying to implement HMC with a non-diagonal mass matrix, but am getting tripped up by some of the terminology.
According to BDA3 and Neal's review, the kinetic energy term (that I guess is ...
- 2,761
13
votes
2
answers
529
views
What should I know about designing a good Hybrid/Hamiltonian Monte Carlo algorithm?
I am designing a Hybrid Monte Carlo sampling algorithm for PyMC, and I am trying to make it as fuss free and general as possible, so I am looking for good advice on designing an HMC algorithm. I have ...
12
votes
5
answers
1k
views
Estimate the Euler–Mascheroni constant ($\gamma$) by Monte Carlo simulations
The Euler–Mascheroni constant is defined simply as the limiting difference between harmonic series and the natural logarithm.
$$\gamma =\lim_{n\to \infty}\left(\sum _{k=1}^{n}{\frac {1}{k}}-\ln n\...
- 233
12
votes
2
answers
3k
views
Why we don't use weighted arithmetic mean instead of harmonic mean?
I wonder what is an intrinsic value of using harmonic mean (for instance to calculate F-measures), as opposed to weighted arithmetic mean in combining precision and recall? I am thinking that weighted ...
- 131
12
votes
4
answers
7k
views
Bootstrap, Monte Carlo
I have been set the following question as part of homework:
Design and implement a simulation study to examine the performance of the bootstrap for obtaining 95% confidence intervals on the mean of a ...
- 121
12
votes
1
answer
19k
views
Coverage probabilities of the basic bootstrap confidence Interval
I have the following question for a course I'm working on:
Conduct a Monte Carlo study to estimate the coverage probabilities of
the standard normal bootstrap confidence interval and the basic ...
- 799
12
votes
3
answers
2k
views
How to program a Monte Carlo simulation of Bertrand's box paradox?
The following problem has been posted on Mensa International Facebook Page:
$\quad\quad\quad\quad\quad\quad\quad\quad$
The post itself received 1000+ comments but I won't go into details about the ...
12
votes
2
answers
3k
views
MCMC sampling of decision tree space vs. random forest
A random forest is a collection of decision trees formed by randomly selecting only certain features to build each tree with (and sometimes bagging the training data). Apparently they learn and ...
- 2,362