All Questions
1,933 questions
11
votes
1
answer
18k
views
Rules to apply Monte Carlo simulation of p-values for chi-squared test
I'd like to understand the use of Monte Carlo simulation in the chisq.test() function in R.
I have a qualitative variable which has 128 levels / classes. My sample ...
- 351
11
votes
1
answer
14k
views
Required number of simulations for Monte Carlo analysis
My question is about the required number of simulations for Monte Carlo analysis method.
As far as I see the required number of simulations for any allowed percentage error $E$ (e.g., 5) is
$$
n = \...
- 111
11
votes
1
answer
236
views
Distribution of $\min_{j\ge 1}(X_1+X_2+\cdots+X_j)/j$ when $X_i$'s are i.i.d $\text{Exp}(1)$
Suppose $(X_n)_{n\ge 1}$ is a sequence of independent Exponential random variables with mean $1$. I am trying to find the distribution of $\min_{j\ge 1}(X_1+X_2+\cdots+X_j)/j$.
Simulation suggests the ...
- 11.6k
11
votes
1
answer
1k
views
Is there a Monte Carlo/MCMC sampler implemented which can deal with isolated local maxima of posterior distribution?
I'm currently using a bayesian approach to estimate parameters for a model consisting of several ODEs. As I have 15 parameters to estimate, my sampling space is 15-dimensional and my searched for ...
- 656
11
votes
2
answers
547
views
Tail bounds on Euclidean norm for uniform distribution on $\{-n,-(n-1),...,n-1,n\}^d$
What are known upper bounds on how often the Euclidean norm of a uniformly chosen
element of $\:\{-n,~-(n-1),~...,~n-1,~n\}^d\:$ will be larger than a given threshold?
I'm mainly interested in bounds ...
- 111
11
votes
1
answer
2k
views
Rao-Blackwellization of sequential Monte Carlo filters
In the seminal paper "Rao-Blackwellised Particle Filtering for Dynamic Bayesian Networks" by A. Doucet et. al. a sequential monte carlo filter (particle filter) is proposed, which makes use of a ...
- 291
11
votes
1
answer
3k
views
Expected value of softmax transformation of Gaussian random vector
Let $\mathbf w_1,\mathbf w_2,\ldots,\mathbf w_n \in \mathbb R^p$ and $\mathbf v \in \mathbb R^n$ be fixed vectors, and $\mathbf x \sim \mathcal N_p(\boldsymbol{\mu}, \mathbf{\Sigma})$ be an $p$-...
- 538
10
votes
5
answers
1k
views
Expectation of random sum of non-random numbers
I have a continuous random variable $\tau$ and I want to evaluate
$$
E\left(\sum_{i=1}^{\lfloor \tau \rfloor} Y_i\right),
$$
where $Y_i$ are known, non-random, and $\lfloor . \rfloor$ is the floor ...
- 860
10
votes
2
answers
2k
views
Is Markov chain based sampling the "best" for Monte Carlo sampling? Are there alternative schemes available?
Markov Chain Monte Carlo is a method based on Markov chains that allows us to obtain samples (in a Monte Carlo setting) from non-standard distributions from which we cannot draw samples directly.
My ...
- 453
10
votes
1
answer
642
views
Distribution of argmax of beta-distributed random variables
Let $x_i \sim \text{Beta}(\alpha_i, \beta_i)$ for $i \in I$. Let $j = \operatorname*{argmax}_{i \in I} x_i$ (ties broken arbitrarily). What is the distribution of $j$ in terms of $\alpha$ and $\beta$? ...
- 1,033
10
votes
2
answers
624
views
Why is E(θ / (1 - θ)) different than E(θ) / (1 - E(θ))?
I've encountered a problem question:
The probability of success for a random variable follows a Beta(5, 3) distribution.
The posterior mean is θ = 0.625.
The odds of success is defined as θ / (1 - θ)....
- 235
10
votes
4
answers
6k
views
When to use Gradient descent vs Monte Carlo as a numerical optimization technique
When a set of equations cannot be solved analytically, then we can use a gradient descent algorithm. But it seems that there is also the method of Monte Carlo simulation that can be used to solve ...
- 6,635
10
votes
3
answers
937
views
Misunderstanding of Monte Carlo Pi Estimation
I am fairly sure that I understand the how Monte Carlo integration works but I am not understanding the formulation of how it is used to estimate Pi. I am going by the procedure outlined in the 5th ...
- 1,895
10
votes
5
answers
6k
views
Generate random multivariate values from empirical data
I'm working on a Monte Carlo function for valuing several assets with partially correlated returns. Currently, I just generate a covariance matrix and feed to the the ...
- 627
10
votes
2
answers
2k
views
Differences between Sampler, MonteCarlo, Metropolis-Hasting method, MCMC method and Fisher formalism
1) I make confusions about what we call a "sampler". From what I understand, a sampler allows to generate a distribution of points that follows a known PDF (probability distribution function)...
user226073
10
votes
1
answer
4k
views
What's the difference between Bayesian Optimization (Gaussian Processes) and Simulated Annealing in practice
Both processes seem to be used to estimate the maximum value of an unknown function, and both obviously have different ways of doing so.
But in practice is either method essentially interchangeable? ...
- 459
10
votes
3
answers
433
views
If $Z_i =\min \{k_i, X_i\}$, $X_i \sim U[a_i, b_i]$, what is the distribution of $\sum_iZ_i$?
Assume the following set up:
Let $Z_i = \min\{k_i, X_i\}, i=1,...,n$. Also $X_i \sim U[a_i, b_i], \; a_i, b_i >0$. Moreover $k_i = ca_i + (1-c)b_i,\;\; 0<c<1$ i.e. $k_i$ is a convex ...
- 60.8k
10
votes
1
answer
637
views
Does Monte Carlo == apply a random process?
I never had a formal statistics course but due to my line of research I'm constantly coming across articles which apply several statistical concepts.
Often I'll see a description of a Monte Carlo ...
- 4,362
10
votes
2
answers
1k
views
Reconciling Langevin MC methods as one-step HMC versus as diffusion or brownian motion
I have a basic understanding of Hamiltonian monte carlo and why it works. I've read that Langevin MC is basically a special case of HMC when you only step the dynamics forward a single timestep before ...
- 26.5k
10
votes
2
answers
575
views
Improving the minimum estimator
Suppose that I have $n$ positive parameters to estimate $\mu_1,\mu_2,...,\mu_n$ and their corresponding $n$ unbiased estimates produced by the estimators $\hat{\mu_1},\hat{\mu_2},...,\hat{\mu_n}$, i.e....
- 4,504
10
votes
1
answer
539
views
Expected value of maximum ratio of n iid normal variables
Suppose $X_1,...,X_n$ are iid from $N(\mu,\sigma^2)$ and let $X_{(i)}$ denote the $i$'th smallest element from $X_1,...,X_n$.
How would one be able to upper bound the expected maximum of the ratio ...
- 103
10
votes
2
answers
340
views
Exact Sampling from Improper Mixtures
Suppose I want to sample from a continuous distribution $p(x)$. If I have an expression of $p$ in the form
$$p(x) = \sum_{i=1}^\infty a_i f_i(x)$$
where $a_i \geqslant 0, \sum_i a_i= 1$, and $f_i$ ...
- 1,356
10
votes
2
answers
11k
views
How to find when a graph reaches a peak and plateaus?
This may sound very basic, but I have this problem:
I've got a queue of data with a window size of 300. New data is added at one end, old values are removed from the other end.
I expect the queue ...
- 203
10
votes
1
answer
363
views
How to optimally spread draws when calculating multiple expectations
Suppose we want to calculate some expectation:
$$E_YE_{X|Y}[f(X,Y)]$$
Suppose we want to approximate this using Monte Carlo simulation.
$$E_YE_{X|Y}[f(X,Y)] \approx \frac1{RS}\sum_{r=1}^R\sum_{s=1}^...
- 1,011
10
votes
1
answer
949
views
Show estimate converges to percentile through order statistics
Let $X_1, X_2, \ldots, X_{3n}$ be a sequence of iid random variables sampled from an alpha stable distribution, with parameters $\alpha = 1.5, \; \beta = 0, \; c = 1.0, \; \mu = 1.0$.
Now consider ...
- 123
10
votes
1
answer
2k
views
Extreme Value Theory: Lognormal GEV parameters
Lognormal distribution belongs to the Gumbel maximum domain of attraction, where:
$F^{logN}(x; \mu,\sigma)=\Phi\left(\frac{\ln x - \mu}{\sigma}\right)$,
$F^{Gum}(x;\mu,\beta) = e^{-\exp\left({-\frac{...
- 1,271
10
votes
1
answer
667
views
Monte Carlo Integration for non-square integrable functions
I hope this is the right place to ask, if not feel free to move it to a more appropriate forum.
I've been wondering for quite a while now how to treat non-square integrable functions with Monte Carlo ...
- 203
10
votes
1
answer
3k
views
How is TD(1) of TD(lambda) equivalent to Monte Carlo?
In Sutton and Barto's book about RL they say that the TD($\lambda$) algorithm is equivalent to Monte Carlo when $\lambda = 1$. I don't see how that is the case.
They define the lambda return as:
$$...
- 307
10
votes
1
answer
3k
views
Maximum likelihood estimator for minimum of exponential distributions
I am stuck on how to solve this problem.
So, we have two sequences of random variables, $X_i$ and $Y_i$ for $i=1,...,n$. Now, $X$ and $Y$ are independent exponential distributions with parameters $\...
- 1,903
9
votes
3
answers
2k
views
Monte Carlo simulations for arbitrary functions
I'm familiar with MC methods for approximating PDF integrals. But in this question, I'm curious how we might adapt these methods for other problems. For example evaluating $\int_{0}^{1} x^2 dx$ . I ...
- 3,520
9
votes
4
answers
855
views
Why is the average of the highest value from 100 draws from a normal distribution different from the 98th percentile of the normal distribution?
Why is the average of the highest value from 100 draws from a normal distribution different from the 98% percentile of the normal distribution? It seems that by definition that they should be the ...
- 19k
9
votes
3
answers
10k
views
What is the expected MINIMUM value drawn from a uniform distribution between 0 and 1 after n trials?
Assume you draw a uniformly distributed random number between 0 and 1 n times. How would one go about calculating the expected minimum number drawn after n trials?
In addition, how would one go ...
- 597
9
votes
1
answer
5k
views
Monte Carlo estimation of probabilities
I would appreciate some advice on how to use Monte Carlo for estimating probabilities.
Generally speaking the problem I have involves running an experiment and counting the frequency of output (which ...
- 359
9
votes
1
answer
6k
views
MAP estimation as regularisation of MLE
Going through the Wikipedia article on Maximum a posteriori estimation, it got confusing after reading this:
It is closely related to the method of maximum likelihood (ML) estimation, but employs ...
- 1,059
9
votes
2
answers
2k
views
Solving a simple integral equation by random sampling
Let $f$ be a nonnegative function. I am interested in finding $z \in [0,1]$ such that
$$ \int_0^{z} f(x)\,dx = \frac{1}{2}\int_0^1 f(x)\,dx$$ The caveat: all I can do is sample $f$ at points in $[0,1]$...
- 433
9
votes
3
answers
2k
views
Extreme value theory for count data
I am aware of extreme value theory for continuous distributions. I need to fit an extreme value distribution to the maximum observation of number of events on a day, per month. This seems to be the ...
- 205
9
votes
1
answer
970
views
Why is the log derivative estimator considered of large variance?
It's mentioned in the paper Variational Bayesian Inference with Stochastic Search that, the variance of the following approximation may be very large, but I didn't quite understand why this is so.
It ...
- 16.8k
9
votes
1
answer
3k
views
The code variable in the nlm() function
In R there is a function nlm() which carries out a minimization of a function f using the Newton-Raphson algorithm. In particular, that function outputs the value of the variable code defined as ...
- 22.4k
9
votes
1
answer
9k
views
Expected value of minimum order statistic from a normal sample
UPDATE Jan 25th 2014: the mistake is now corrected. Please ignore the calculated values of the Expected Value in the image uploaded - they are wrong- I don't delete the image because it has generated ...
- 60.8k
9
votes
2
answers
2k
views
Sampling from bivariate distribution with known density using MCMC
I tried to simulate from a bivariate density $p(x,y)$ using Metropolis algorithms in R and had no luck. The density can be expressed as $p(y|x)p(x)$, where $p(x)$ is Singh-Maddala distribution
$p(x)...
- 35.4k
9
votes
1
answer
442
views
Intuition about the coupon collector problem approaching a Gumbel distribution
The coupon collector's problem
Let there be $n$ different types of coupons and we try to collect all of the types.
We do this by independent random draws of coupons in which each type of coupon has an ...
- 86.5k
9
votes
2
answers
189
views
What is the most powerful result about the maximum of i.i.d. Gaussians? The most used in practice?
Given $X_1, \ldots, X_n, \ldots \sim \mathscr{N}(0,1)$ i.i.d., consider the random variables
$$ Z_n := \max_{1 \le i \le n} X_i\,. $$
Question: What is the most "important" result about ...
- 6,479
9
votes
1
answer
2k
views
MCMC and data augmentation
I have been looking at an MCMC data augmentation question; the general form of the question is as follows:
Suppose data gathered on a process suggests $X_{i} \sim \text{Pois}(\lambda)$ and a prior ...
- 1,521
9
votes
2
answers
932
views
What is the distribution of maximum of a pair of iid draws, where the minimum is an order statistic of other minima?
Consider $n\cdot m$ independent draws from cdf $F(x)$, which is defined over 0-1, where $n$ and $m$ are integers. Arbitrarily group the draws into $n$ groups with m values in each group. Look at the ...
- 1,049
9
votes
1
answer
2k
views
Approximating the mathematical expectation of the argmax of a Gaussian random vector
Let $X = \left( {{X_1},...,{X_n}} \right) \sim \mathcal{N}\left( {{\mathbf{\mu }},{\mathbf{\Sigma }}} \right)$ be a Gaussian random vector and $I = \mathop {\arg \max }\limits_{i = 1,n} {X_i}$.
$I$ ...
user193726
9
votes
1
answer
2k
views
Is Monte Carlo uncertainty estimation equivalent to analytical error propagation?
If I have a deterministic, analytic model, $y=f(x)$, I can analytically calculate the uncertainty in $y$ from a known uncertainty in $x$, $\sigma$. Or I can do a Monte Carlo integration: sample from ...
- 5,541
9
votes
1
answer
2k
views
How to get multivariate credible interval estimate(s) / highest density regions (HDR) after MCMC
I'm estimating 15 parameters of my model using a Bayesian approach and a Markov Chain Monte Carlo (MCMC) method. My data after running a MCMC chain of 100000 samples is therefore a 100000×15 table of ...
- 656
9
votes
0
answers
163
views
In sports modelling, are hot simulations better or cold simulations?
I'm thinking here largely of the context in which someone has an Elo rating model for a particular sport.
To calculate things such as how often the team makes the Finals series, or wins the ...
8
votes
5
answers
2k
views
When sampling a population for surveys we can often limit our sample size to hundreds, but when doing a Monte Carlo simulation we need way more. Why?
I’m a bit of a stats-noob, so I am not sure I will manage to formulate this question properly, but let me do my best.
I‘m trying to develop an intuition for sample sizes and when they are sufficient ...
- 183
8
votes
4
answers
1k
views
Linearity of maximum function in expectation
I was solving an exercise for a probability theory course and stumbled upon the following problem.
Given a continuous random variable $X$, and $\max(a,b) = a$ if $a > b$ and $b$ otherwise, is
$$
E[\...
- 193