Questions tagged [numerical-integration]
A class of algorithms to approximate definite integrals.
172
questions
1
vote
0
answers
21
views
How to compute the expected value of a function of a random variable given its log-density function? [closed]
Given a log-density function $\mathcal{L}f_{X}(x)$ of an 1d continuous random variable $X \in \mathcal{L}^{\infty}$ and an 1d polynomial function $h: \mathcal{I}(X) \to \mathbb{R}$, the expected value ...
3
votes
0
answers
84
views
Can Fisher Information Matrix be calculated numerically through finite differentiation?
In GLMs (generalized linear models), the negative of the Fisher information matrix takes the form of a cross product between covariates $\mathbf{X}$ and a diagonal matrix:
$$
\mathbf{X}^T \mathbf{D} \...
- 587
1
vote
0
answers
84
views
$E(XY)$ for a truncated bivariate normal
If $(X, Y)$ follows a bivariate Gaussian distribution with mean ${\bf \mu}$ and covariance ${\bf \Sigma}$ with truncation bounds $(a_x, b_x, a_y, b_y)$, can we compute $E(XY)$ in closed form? If not, ...
- 7,812
0
votes
0
answers
46
views
Why use integrals when building exponential composite functions?
I recently read this paper, which describes a generalisation for statistical exponential decay models used in ecology. Essentially, the parameter $k$ of the exponential decay function $f(x) = ce^{-kx}$...
1
vote
0
answers
32
views
(Bayesian) Quadrature when the density can be factorised with respect to a directed graph
I am very new to (Bayesian) quadrature and am trying to understand if or how we can use additional information to approximate our definite integral.
Specifically, I am interested in the expected value....
- 927
3
votes
0
answers
121
views
Numerical moments of a multivariate Poisson Log-normal posterior
I have a log-density of the form:
$$P(\mathbf{x}) \propto \exp\left( - \mathbf{b}^{\top} e^{ \mathbf{x} } - \frac{1}{2}\mathbf{x}^{\top}A\mathbf{x} \right)$$
where $A$ is a symmetric positive definite ...
- 4,420
0
votes
0
answers
33
views
Numerical quadrature for Pareto distribution
I would like to numerically evaluate an integral of the following type, when evaluating $f(x)$ at any given point is numerically costly:
$$
\int_{x_m}^\infty x^{-\alpha}f(x) \, dx, \quad \alpha >1, ...
- 31
0
votes
0
answers
62
views
The meaning of probability density functions' product followed by an integration
Scipy's KDE object allows integration of a function multiplied by another KDE object. I assume that this is meant to be used for the estimation of distance between two distributions. As far as I ...
- 390
0
votes
0
answers
23
views
Numerical Optimization of Marginal Likelihood that Explodes
I have a model with a marginal likelihood of the following form:
$$\mathcal{L}(\theta_1, \theta_2, \theta_3|\{x_{i,j}\}_{i=1, j=1}^{N, M_i})=\prod_{i=1}^{N}\int_{0}^{1} f(p_i;\theta_1) \prod_{j=1}^{...
- 238
1
vote
1
answer
17
views
Algorithm to determine sufficient range for numerical integration of PDF to get quantiles
Given a probability density function (PDF), we want to use numerical integration to find a quantile at $\alpha \in (0,1)$. Are there standard algorithms for determining a sufficient and optimal range ...
- 1,132
2
votes
1
answer
56
views
Numerical Approximation of Integrals in GLMM
Consider a GLMM setting. Let $i=1,...,m$ denote cluster. Let $l_i(y_i|x_i,b_i;\beta)$ denote the log-likelihood (viewed as a function of the parameters) for each cluster $i$, which includes notably ...
- 217
3
votes
1
answer
89
views
Calculation of Posterior distribution numerically
For calculating posterior probabilities numerically, I did not understand that why is in the following codes they have divided by 0.001 in the denominator to calculate ...
- 309
0
votes
0
answers
16
views
Integrating splines over a set of cells
Let $f(s) = \sum_{i=1}^p \beta_i \phi_i(s)$ such that $f: \mathbb{R}^2 \mapsto \mathbb{R}_+$ and the $\phi_i$ are splines (B-splines or Thin-plate splines) and suppose we are interested in $W\subset ...
- 393
2
votes
0
answers
150
views
Maximum simulated likelihood estimation in R - binary logit example
Very often the likelihood function involves an integral which does not have a closed-form solution (e.g., random coefficients, latent variables). In these cases, we can employ monte-carlo simulation ...
1
vote
1
answer
53
views
Best way or rules of thumbs for evaluating 1d integrals from randomly sampled grid points
I have a 1D domain (let's say the interval $(0,1)$) on which I randomly sample $N$ points from the uniform distribution. I have a function $f\colon (0,1) \to \mathbb{R}$ which is integrable.
What is ...
- 111
2
votes
1
answer
49
views
MLE for parametric binomial model
I have a model in which $p_i=f(\theta,Z_i)$, where $Z_i$ are iid latent variables distributed with CDF $F_\theta$, and $d_i\sim B(n_i,p_i)$, where $B$ is the binomial distribution. The likelihood ...
- 211
0
votes
0
answers
17
views
When to stop MCMC within collapsed Gibbs?
I am setting up a Hierarchical model whose target distribution is $p(\theta,w|y)$, $\theta$ being a reduced set of high-level parameters, $w$ being a data augmentation of very high dimension, and $y$ ...
- 11
0
votes
0
answers
50
views
How to numerically get expectation of a non-linear function of a normally distributed random variable
I'm trying to calculate the expectation of the following numerically:
$$\mathbb{E}[V(\theta)]$$ where $\theta\sim N(\mu,\sigma^2)$ and $V(\theta)$ is strictly increasing.
I'm struggling to understand ...
- 203
1
vote
1
answer
43
views
How to use control variate method to estimate $\theta = \int_{1}^{\infty}\frac{x^2}{\sqrt{2\pi}}e^{-x^2/2}dx$
This is the target I want to integrate with Monte Carlo with control variate method:
$$\theta = \int_{1}^{\infty}\frac{x^2}{\sqrt{2\pi}}e^{-x^2/2}dx$$
I have checked with Wolfram that it is 0.400626, ...
- 113
4
votes
1
answer
220
views
Unbiased estimator of an integral raised to a power
I'd like to estimate an integral of the following form using Monte Carlo method:
$$ \int_{t_1}^{t_2} g(t) \left[ \int_{- \infty}^{\infty} f(t, u) du \right] ^\gamma dt$$
In case of $\gamma$ being a ...
3
votes
3
answers
669
views
How to use self-normalized importance sampling method to estimate $\int_{1}^{\infty} \frac{x^2}{\sqrt{2\pi}}e^{\frac{-x^2}{2}}dx$?
I want to use self-normalized importance sampling methods to estimate $$\int_{1}^{\infty} \frac{x^2}{\sqrt{2\pi}}e^{\frac{-x^2}{2}} \,dx$$ I choose exponential distribution with rate $\lambda=1$ as ...
- 33
2
votes
1
answer
40
views
Numerical integration over a Random Forest Regression Model
I am trying to compare the accuracy of a polynomial model and a Random forest regression model in predicting a variable Y but also the integral of this variable ,
With the polynomial model, it is ...
- 21
0
votes
1
answer
92
views
Estimating the integral $\int_{0}^\infty x^4 e^{-2x}\,dx$
We have been given a random variable having a Gamma distribution as shown below:
Using the accept-reject algorithm, we are supposed to sample from the Gamma distribution using exponential as the ...
5
votes
1
answer
285
views
How to Find PDF of Transformed Random Variables Numerically?
Is there a way to compute and plot the transformed random variable in Python or R?
Let's say if $X_1$ and $X_2$ are independent and identically distributed random variable with PDF defined over non-...
- 53
1
vote
1
answer
84
views
Are discrete mixtures Gauss quadrature-like integral approximations?
I noticed that the formula for Gauss (or Newton-Cotes) quadrature looks very similar to the formula for the PDF of a general mixture distribution.
Let $p_{comp}(x)$ be the PDF of a compound ...
- 310
1
vote
3
answers
213
views
Why not just run a Markov chain to get stationary probabilities?
I'm reading Performance Modeling and Design of Computer Systems which contains some analysis of Markov chains. In particular, it emphasises various analytical methods for finding the stationary ...
- 814
2
votes
1
answer
2k
views
Estimating the cumulative probability of a bivariate normal distribution
I have a quick question regarding working out the probabilities of a bivariate normal distribution. To my knowledge, there is no nice closed-form for a cumulative distribution function for the ...
- 23
2
votes
1
answer
141
views
Bayesian Quadrature to find expectation of unkown function w.r.t. known pdf
I am interested in estimating the integral $\int f(x) P(x) dx$, where $f(x)$ is an expensive function and $P(x)$ is has an analytic form. I would like to evaluate this with as few evaluations of $f(x)$...
- 157
4
votes
1
answer
123
views
Approximate Posterior Predictive Quantiles with Numerical Methods
I have a posterior function which is easy to approximate using numerical methods (the posterior has only 2 parameters, and is approximately Gaussian because of the large sample). However, I need to ...
- 1,065
1
vote
0
answers
100
views
Conditional expectation of a normal variable given a lower bound [duplicate]
I saw the identity below but I'm not sure how to derive it.
$$E[X\mid X>K] = \mu + \sigma \frac{\phi(z)}{\Phi(-z)} \text{ where } z = \frac{K-\mu} \sigma$$
I'm stuck at the following step:
$$E[X\...
3
votes
1
answer
355
views
Gompertz-Makeham Model Life Expectancy using Numerical Integration and Analytical Solution of Castellares et al. (2020)
I am trying to calculate life expectancies for the Gompertz-Makeham model, but can't replicate the results of the paper which gives the formulas. I use R to ...
4
votes
2
answers
613
views
How to calculate Quasi-Monte Carlo integration error when sampling with Sobol's sequence?
My understanding is that QMC integration using random sampling will converge with $O(\frac{1}{\sqrt{n}})$, while using Sobol's sampling will converge with $O(\frac{(\log{n})^d}{n})$. However I'm ...
- 173
12
votes
5
answers
1k
views
Estimate the Euler–Mascheroni constant ($\gamma$) by Monte Carlo simulations
The Euler–Mascheroni constant is defined simply as the limiting difference between harmonic series and the natural logarithm.
$$\gamma =\lim_{n\to \infty}\left(\sum _{k=1}^{n}{\frac {1}{k}}-\ln n\...
- 285
1
vote
0
answers
25
views
What is the expectation of a random variable satisfying some conditions?
How to find the expectation E[X.I(Y<x,X<x)], where X and Y are independent random variables with respective cumulative distribution functions F(.) and G(.) respectively. x is a positive value. ...
- 79
12
votes
2
answers
2k
views
Plain English explanation of Ito's integral?
I'm looking for a plain English explanation of Ito's integral. I don't need an exhaustive proof, derivation, etc. Just a simple ~this is effectively what it does and why it's better than a Riemann sum ...
- 3,020
3
votes
1
answer
96
views
Numerically integrate gaussian pdf
Suppose I want to numerically integrate the function $g(\mathbf{x}) = \exp\bigg(-\frac12 \mathbf{x}^\mathsf{T}\mathbf{\Lambda} \mathbf{x}\,\bigg)$ to obtain the normalization constant
$$\int_{\mathcal{...
- 1,357
3
votes
1
answer
912
views
Kullback–Leibler divergence between multivariate t and the multivariate normal?
I want to calculate the Kullback–Leibler divergence between a multivariate $t$ distribution and a multivariate normal distribution, for different values of the degrees of freedom $\nu$.
However, this ...
- 31
2
votes
0
answers
136
views
Numerical integration of the exact distribution of the product of two random gaussian variables
I am trying to implement this function in R, which is equation (2) in this paper by Cui et al. (2016), which is the exact distribution of the product of two random gaussian variables.
...
- 668
1
vote
0
answers
19
views
How do mixed models bypass the overflow inside the integration?
In a mixel model, when you try to maximize the likelihood, you have to integrate over the random effects.
This means that you cannot take logs, since what would normally be a product of individual ...
- 11
1
vote
1
answer
98
views
Computing the probability $P(\exists X\in\{X_1,\ldots, X_N\}:X>\text{max}\{Y_1,\ldots, Y_M\})$
I am trying to somewhat generalize a question, which has been asked in one way or another a several times here on StackExchange**. However, I have not managed to find an answer to the below problem.
...
- 11
2
votes
1
answer
820
views
How to calculate confidence intervals for a generic (non-normal) pdf
In particular, this is a question about distributions with some skew and/or where the mean, median and mode might all be different.
For any pdf, it's possible to find multiple different domains within ...
- 2,422
0
votes
1
answer
968
views
Different estimates for mixed effects Logistic regression and pwrssUpdate Error message with binomial glmer
Hi this is my first post here. I reported an issue on Github and got valuable help from Dr. Bolker. However, I did some other analyses on the same data and got confused about some of the results. I ...
0
votes
0
answers
53
views
Discrete correlation function (sample cross-covariance)
The continuous correlation function for the random variable $A(t)$ at a instant of time $t$ is given by
\begin{equation}
C_{AA}(\tau) =\frac{1}{T} \int_{0}^T d\bar{t} A(\bar{t})A(\bar{t}+\tau)
\end{...
- 101
-3
votes
1
answer
105
views
Positive or negatively bounded CDFs [closed]
If $X\in\mathbb{R}^n$ is a continuous random variable whose cumulative distribution function is ordinarily
$$F_X(x) = \int_{-\infty}^{\infty} f_X(x) dx $$
what is the meaning of
$$F_X(x) = \int_{0}^{\...
- 3,927
7
votes
1
answer
316
views
Expressing a marginal probability using copulas
Please correct me if I am wrong and kindly provide me with the correct notations. I have two questions:
We know that for the variables $(X,Y,Z)\in \mathbb{R}^3$, the marginal joint density $f(x,y)$ ...
- 1,206
1
vote
0
answers
45
views
How to calculate Integral error from MCMC
I've recently started using Markov Chain Monte Carlo to calculate integrals, but I can't seem to find how to calculate the error for such an integral. In the standard case of importance sampling, the ...
- 248
1
vote
1
answer
452
views
Computational Complexity of Multivariate Normal CDF
I'm going to post this here also as per user suggestions since I feel like the root cause of the issue is more maths related than code related.
I'm working on a multivariate cross-entropy minimization ...
- 111
1
vote
1
answer
157
views
Finding confidence interval for unimodal function equivalent to and comparable with standard deviation of normal
I'm trying to characterise an arbitrary, unimodal distribution in a way that is a) easily understandable (to a physics audience) and b) comparable with a normal distribution.
My thinking goes like ...
- 13
3
votes
1
answer
256
views
Numerical integration with measurement errors: what scheme to use?
I have an unknown 'smooth' function that is sampled at some $n$ equally spaced locations $x_i$ between $0$ and $1$, with some measurement values $f(x_i)=y_i$, but each has some Gaussian uncertainty ($(...
- 151
1
vote
1
answer
343
views
Calculating variance of MC estimator of an integral
Let
$$
f(x) = \sqrt{1-x^2}, \quad I = \int \limits_0^1 f(x) dx.
$$
We consider the following estimator of I:
$$
\hat{I}_n = \frac{1}{n} \sum \limits_{i=1}^n f(X_i),
$$
where $X_1, X_2, ...$ are iid $...
- 325