Questions tagged [approximation]

Approximations to distributions, functions, or other mathematical objects. To approximate something means to find some representation of it which is simpler in some respect, but not exact.

Filter by
Sorted by
Tagged with
18
votes
3answers
5k views

Evaluate definite interval of normal distribution

I know that an easy to handle formula for the CDF of a normal distribution is somewhat missing, due to the complicated error function in it. However, I wonder if there is a a nice formula for $N(c_{-}...
38
votes
4answers
18k views

Approximate order statistics for normal random variables

Are there well known formulas for the order statistics of certain random distributions? Particularly the first and last order statistics of a normal random variable, but a more general answer would ...
23
votes
2answers
4k views

Difference of two i.i.d. lognormal random variables

Let $X_1$ and $X_2$ be 2 i.i.d. r.v.'s where $\log(X_1),\log(X_2) \sim N(\mu,\sigma)$. I'd like to know the distribution for $X_1 - X_2$. The best I can do is to take the Taylor series of both and ...
11
votes
1answer
191 views

Should degrees of freedom corrections be used for inference on GLM parameters?

This question is inspired by Martijn's answer here. Suppose we fit a GLM for a one parameter family like a binomial or Poisson model and that it is a full likelihood procedure (as opposed to say, ...
13
votes
3answers
4k views

How to compute the probability associated with absurdly large Z-scores?

Software packages for network motif detection can return enormously high Z-scores (the highest I've seen is 600,000+, but Z-scores of more than 100 are quite common). I plan to show that these Z-...
8
votes
3answers
2k views

Does the universal approximation theorem for neural networks hold for any activation function?

Does the universal approximation theorem for neural networks hold for any activation function (sigmoid, ReLU, Softmax, etc...) or is it limited to sigmoid functions? Update: As shimao points out in ...
5
votes
1answer
553 views

Bound for weighted sum of Poisson random variables

Suppose I have some independent Poisson-distributed random variables $X_1 \ldots X_N$ with parameters $\lambda_1 \ldots \lambda_N$. These can be thought of as processes where each arrival/event ...
16
votes
1answer
3k views

How does a random kitchen sink work?

Last year at NIPS 2017 Ali Rahimi and Ben Recht won the test of time award for their paper "Random Features for Large-Scale Kernel Machines" where they introduced random features, later codified as ...
3
votes
2answers
898 views

Function Approximation vs. Regression

Some background before I state the questions: I have a $d$-dimensional random vector $X=(X_1,\ldots,X_n)$ and a function $f:\mathbb{R}^d\rightarrow\mathbb{R}$. Ultimately my goal is to understand $f$ ...
12
votes
1answer
702 views

Approximate distribution of product of N normal i.i.d.? Special case μ≈0

Given $N\geq30$ i.i.d. $X_n\approx\mathcal{N}(\mu_X,\sigma_X^2)$, and $\mu_X \approx 0$, looking for: accurate closed form distribution approximation of $Y_N=\prod\limits_{1}^{N}{X_n}$ asymptotic (...
9
votes
2answers
844 views

Universal approximation theorem for convolutional networks

The universal approximation theorem is a quite famous result for neural networks, basically stating that under some assumptions, a function can be uniformly approximated by a neural network withing ...
8
votes
4answers
5k views

normal approximation to the binomial distribution: why np>5?

Nearly every text book which discusses the normal approximation to the binomial distribution mentions the rule of thumb that the approximation can be used if $np\geq5$ and $n(1-p)\geq 5$. Some books ...
6
votes
3answers
325 views

Probability for finding a double-as-likely event

Repeating an experiment with $n$ possible outcomes $t$ times independently, where all but one outcomes have probability $\frac{1}{n+1}$ and the other outcome has the double probability $\frac{2}{n+1}$,...
5
votes
3answers
1k views

regression with constraints

I have some domain knowledge I want to use in a regression problem. Problem statement The dependent variable $y$ is continuous. The independent variables are $x_1$ and $x_2$. Variable $x_1$ is ...
0
votes
0answers
88 views

Approximate distribution of product of N normal i.i.d.? General case [duplicate]

Given $N\geq30$ i.i.d. $X_n\approx\mathcal{N}(\mu_X,\sigma_X^2)$, and NO assumptions about $\mu_X$ and $\sigma_X$, looking for: accurate closed form distribution approximation of $Y_N=\prod\limits_{...
12
votes
3answers
17k views

Normal approximation to the Poisson distribution

Here in Wikipedia it says: For sufficiently large values of $λ$, (say $λ>1000$), the normal distribution with mean $λ$ and variance $λ$ (standard deviation $\sqrt{\lambda}$), is an excellent ...
6
votes
1answer
552 views

Half-normal probability plot

To construct the half-normal probability plot, plot the absolute values in a certain statistical diagnostic (residual, leverage, Cook distance and others) versus $z_i$ where: $\displaystyle z_{i} = \...
5
votes
1answer
835 views

How should sampling ratios to estimate quantiles change with population size?

I want to cut my data of size N into k equal-sized bins. But I am happy with roughly equal-sized bins, with some $\varepsilon$ error. As precise quantiles of the data are computationally costly (...
4
votes
1answer
157 views

In exactly what sense do MCMC draws approximate the target?

Background We want to sample from some intractable density $\pi(\theta)$. Using an MCMC algorithm, we generate a sample of draws $\{\theta_i\}_{i=1}^N$ from a Markov chain that has $\pi(\theta)$ as ...
3
votes
1answer
223 views

natural log approximation

I've got an equation that contains x^p - 1 x is any positive number, such as 2, and p is a small positive number close to 0 such as 0.001 For some reason (that ...
1
vote
2answers
2k views

Using continuity correction for normal approximation or not?

Below is a question on a recent actuarial exam, Exam 3L of the CAS. I didn't know whether or not to use the continuity correction when using the normal approximation to do hypothesis testing ...
10
votes
1answer
1k views

Do Gaussian process (regression) have the universal approximation property?

Can any continuous function on [a, b], where a and b are real numbers, be approximated or arbitrarily close to the function (in some norm) by Gaussian Processes (Regression)?
12
votes
1answer
2k views

One sided Chebyshev inequality for higher moment

Is there an analogue to the higher moment Chebyshev's inequalities in the one sided case? The Chebyshev-Cantelli inequality only seem to work for the variance, whereas Chebyshevs' inequality can ...
10
votes
2answers
4k views

When do Taylor series approximations to expectations of (entire) functions converge?

Take an expectation of the form $E(f(X))$ for some univariate random variable $X$ and an entire function $f(\cdot)$ (i.e., the interval of convergence is the whole real line) I have a moment ...
10
votes
1answer
8k views

What is the normal approximation of the multinomial distribution?

If there are multiple possible approximations, I'm looking for the most basic one.
20
votes
1answer
3k views

Error in normal approximation to a uniform sum distribution

One naive method for approximating a normal distribution is to add together perhaps $100$ IID random variables uniformly distributed on $[0,1]$, then recenter and rescale, relying on the Central Limit ...
8
votes
1answer
2k views

How to understand the geometric intuition of the inner workings of neural networks?

I've been studying the theory behind ANNs lately and I wanted to understand the 'magic' behind their capability of non-linear multi-class classification. This led me to this website which does a good ...
7
votes
1answer
1k views

Distribution of the Levenshtein distance between two random strings

The Levenshtein or edit distance between two strings is the minimum number of edits (adding a letter, removing a letter or changing a letter) required to transform one into the other. Assume that we ...
19
votes
5answers
2k views

Why bother with low rank approximations?

If you have a matrix with n rows and m columns, you can use SVD or other methods to calculate a low-rank approximation of the given matrix. However, the low rank approximation will still have n rows ...
6
votes
2answers
373 views

Bayes Factor approximation

A brute force method to approximate the Bayes Factor (the ratio of the denominators (normalizing constants) in the Bayes formula) is to do the following for the two models of interest: repeat ...
6
votes
1answer
1k views

Approximating the distribution of a linear combination of beta-distributed independent random variables

This question is related with these other two questions in Cross Validated, which has been already answered: Approximate the distribution of the sum of ind. Beta r.v Central limit theorem when the ...
8
votes
1answer
274 views

Analytically solving sampling with or without replacement after Poisson/Negative binomial

Short version I am trying to analytically solve/approximate the composite likelihood that results from independent Poisson draws and further sampling with or without replacement (I don't really care ...
7
votes
5answers
2k views

Approximation of logarithm of standard normal CDF for x<0

Does anyone know of an approximation for the logarithm of the standard normal CDF for x<0? I need to implement an algorithm that very quickly calculates it. The straightforward way, of course, is ...
7
votes
2answers
4k views

Fast approximation to inverse Beta CDF

I am looking for a fast approximation to the inverse CDF of the Beta distribution. The approximation need not be precise, but more stress is on simplicity (I'm thinking Taylor expansion of the first 1 ...
1
vote
1answer
705 views

Approximating the expected value and variance of the function of a (continuous univariate) random variable

Let $X$ be a univariate continuous random variable (r.v.). Let $g$ be a smooth real function defined on the sample space of $X$. I have been told that the following approximations are true: $$ \...
1
vote
1answer
74 views

Approximate distribution of product of N normal i.i.d.? Special case μ>10σ, σ>0

Given $N\geq30$ i.i.d. $X_n\approx\mathcal{N}(\mu_X,\sigma_X^2)$, and $|\mu_X|\geq10\sigma_X$, $\sigma > 0$, looking for: accurate closed form distribution approximation of $Y_N=\prod\limits_{1}^{...
4
votes
1answer
119 views

Estimate probability of event using moments of a distribution or a Taylor expansion involving the moments

Let's say we have four moments $(\mu_1, \mu_2, \mu_3, \mu_4)$ of a probabilty distribution of a random variable $X$ and the goal is to get the probability $\rm{P}(X \leq t)$ for a certain value of $t$....
3
votes
2answers
5k views

Normal approximation to the binomial distribution

I am having trouble getting to the bottom of this concept for two types of questions (hw is already passed, but I have a test this week and would like to do better). Hopefully someone can help me get ...
3
votes
2answers
3k views

How many times do I have to roll a dice to get six six times in a row? [duplicate]

I wonder if there's any exact way to find out these two things about six-sided dice: How many throws would be necessary to get six times the same number (let's say six) in a row? What's the ...
2
votes
0answers
582 views

maximum gap between order statistics of normally distributed random variables [closed]

I am currently working on a not-that-easy problem involving order statistics. As I am unsure as to how I could solve it, I thought it might already possess a solution. So here I am, my questions is: ...
1
vote
0answers
117 views

help with expectation propagation: updating factors

I have been trying unsuccessfully for a while to understand and setup an inference problem and use EP with it. I have tried to extract a simple case to highlight he issue. I hope someone will be kind ...
6
votes
3answers
992 views

Approximation of Cauchy distribution

I have a ratio of two random, (dependent or independent) normally distributed variables. Knowing that the resulting Cauchy-distribution does not produce any moments. May I ask: Is there an ...
4
votes
1answer
410 views

Arbitrary function approximation in one dimension

Suppose we have some arbitrary function $f: X \mapsto Y, X \in \mathbb{R}, Y \in [0, 1]$. It may be smooth but it may not. I am looking for some way to approximate this function given samples drawn ...
4
votes
0answers
264 views

Expectation of the softmax transform for Gaussian multivariate variables

Prelims In the article Sequential updating of conditional probabilities on directed graphical structures by Spiegelhalter and Lauritzen they give an approximation to the expectation of a logistic ...
3
votes
1answer
172 views

Approximate distribution of normal squared

I am studying for a test, one section of which will cover the delta method. This problem came from that section: Let $X\sim N(\mu,n^{-1})$. Find an approximate distribution of $X^2$. (It also asks ...
3
votes
0answers
97 views

Results on continuity corrections

For me, the first thing that comes to mind when I come across the term continuity correction is that when $X\sim\mathrm{Bin}(n,p)$, one approximates $\Pr(X\le x)=\Pr(X<x)$ by $\Pr(Y<x+1/2)$ ...
3
votes
1answer
1k views

code Welch's t-test in C++ using approximations for student's t values

I need to code a Welch's t-test between two populations in C++ without using external libraries like, for example, boost. I know that given my two populations of size $N_1$ and $N_2$, I can calculate ...
2
votes
1answer
222 views

How to approximate Bayes Factor?

I am searching for a computationally simple way to approximate a Bayes Factor. Currently, I'm using an approach which seems pretty logical to me but I would still be interested to know if this is ...
1
vote
2answers
69 views

Calculate the tendency of a set of samples

I develop an application in which i constantly get samples of heart pulse. I defined an interval of t seconds. In each t seconds I have n samples. In every interval, I want to calculate the ...
1
vote
1answer
5k views

Why are 5+ expected frequencies needed in Pearson's chi-square [duplicate]

I've been told in my current textbook that for approximation of the exact distribution by means of $X^2$ or $G^2$ the expected frequencies have to be $\ge 5$. The book says that the approximation ...