We’re rewarding the question askers & reputations are being recalculated! Read more.

Questions tagged [approximation]

Approximations to distributions, functions, or other mathematical objects. To approximate something means to find some representation of it which is simpler in some respect, but not exact.

Filter by
Sorted by
Tagged with
1
vote
0answers
79 views

Comparing nonlinear models using the Chi-squared asymptotic approximation

I am attempting to identify the best model fit for a nonlinear mixed effects model using the asymptotic approximation to the Chi-squared test. When calculating the appropriate number of degrees of ...
10
votes
2answers
4k views

When do Taylor series approximations to expectations of (entire) functions converge?

Take an expectation of the form $E(f(X))$ for some univariate random variable $X$ and an entire function $f(\cdot)$ (i.e., the interval of convergence is the whole real line) I have a moment ...
4
votes
2answers
498 views

Low rank approximation of binary valued matrix

How does one get the low rank approximation of binary matrix? Is the low rank approximation also a binary matrix? Note - Here binary matrix just means that any entry of the matrix can either be 0 or ...
2
votes
0answers
248 views

Classification and regression tree (CART) on large data set

I am trying to approximate a multivariate function $y = f(x_1, ...x_n)$, which I have reason to believe will be well approximated by a classification and regression tree. Some of the variables are ...
6
votes
1answer
1k views

Approximating the distribution of a linear combination of beta-distributed independent random variables

This question is related with these other two questions in Cross Validated, which has been already answered: Approximate the distribution of the sum of ind. Beta r.v Central limit theorem when the ...
7
votes
1answer
838 views

Berry-Esseen bound for binomial distribution

From the Berry-Essen theorem I can deduce $$\sup_{x\in\mathbb R}\left|P\left(\frac{B(p,n)-np}{\sqrt{npq}} \le x\right) - \Phi(x)\right| \le \frac{C(p^2+q^2)}{\sqrt{npq}}$$ with $C \le 0.4748$. My ...
4
votes
1answer
823 views

Number of Gaussian mixture components needed to approximate any distribution

I remember reading an actual proven number of components, that can approximate any distribution. Somehow I think it was 18. Can someone point me to a book/article stating something of the sort? Might'...
2
votes
1answer
216 views

Dvoretzky–Kiefer–Wolfowitz inequality hold for discrete distributions?

I am wondering whether Dvoretzky–Kiefer–Wolfowitz inequality holds for discrete distributions? Any comments or references would be greatly appreciated.
3
votes
0answers
141 views

bound on expectation of a two-variable function under an independent distribution

Consider a probability distribution $P(x)$, a set observed samples $S = \{x_1,\cdots, x_n\}$ where $x_i \stackrel{iid}{\sim} P(x)$ for $i \leq n$, and a symmetric function $h(x,y)$. How can one ...
23
votes
2answers
4k views

Difference of two i.i.d. lognormal random variables

Let $X_1$ and $X_2$ be 2 i.i.d. r.v.'s where $\log(X_1),\log(X_2) \sim N(\mu,\sigma)$. I'd like to know the distribution for $X_1 - X_2$. The best I can do is to take the Taylor series of both and ...
4
votes
1answer
413 views

Arbitrary function approximation in one dimension

Suppose we have some arbitrary function $f: X \mapsto Y, X \in \mathbb{R}, Y \in [0, 1]$. It may be smooth but it may not. I am looking for some way to approximate this function given samples drawn ...
0
votes
0answers
116 views

Cox PH Modelling: How important are ties to the end result?

I am working on create a survival model using Cox PH, but am concerned with the limitation of the continuous time requirement. In my dataset there are often ties. For example, out of 35000 records, ...
1
vote
0answers
1k views

wald test and score test, normal or chi square?

I learnt from section 10.3 of statistical inference that both Wald test statistic $\frac{W_n-\theta_0}{S_n}\approx\frac{W_n-\theta_0}{\sqrt{\hat I_n(W_n)}}$ and score test statistic $\frac{S(\theta_0)}...
4
votes
2answers
927 views

How to extract the function being approximated by a neural network?

We all know that in general, a neural network takes in a set of training examples having the form $\{x, f(x)\}$ and it aims to approximate the function $f$ thereby "classifying" $x$ to its correct ...
1
vote
1answer
74 views

Approximate distribution of product of N normal i.i.d.? Special case μ>10σ, σ>0

Given $N\geq30$ i.i.d. $X_n\approx\mathcal{N}(\mu_X,\sigma_X^2)$, and $|\mu_X|\geq10\sigma_X$, $\sigma > 0$, looking for: accurate closed form distribution approximation of $Y_N=\prod\limits_{1}^{...
0
votes
0answers
88 views

Approximate distribution of product of N normal i.i.d.? General case [duplicate]

Given $N\geq30$ i.i.d. $X_n\approx\mathcal{N}(\mu_X,\sigma_X^2)$, and NO assumptions about $\mu_X$ and $\sigma_X$, looking for: accurate closed form distribution approximation of $Y_N=\prod\limits_{...
12
votes
1answer
706 views

Approximate distribution of product of N normal i.i.d.? Special case μ≈0

Given $N\geq30$ i.i.d. $X_n\approx\mathcal{N}(\mu_X,\sigma_X^2)$, and $\mu_X \approx 0$, looking for: accurate closed form distribution approximation of $Y_N=\prod\limits_{1}^{N}{X_n}$ asymptotic (...
1
vote
1answer
157 views

Universal approximation of probability distribution with latent variable model

I want to show that under certain circumstances this form can approximate any probability distribution. For that, I came up with the following argument. Consider a directed graphical model of the ...
7
votes
1answer
1k views

Distribution of the Levenshtein distance between two random strings

The Levenshtein or edit distance between two strings is the minimum number of edits (adding a letter, removing a letter or changing a letter) required to transform one into the other. Assume that we ...
2
votes
0answers
63 views

Accuracy of approximation to sample median

Suppose we sample $n$ real numbers uniformly on $[a, b]$. We wish to find the sample median, but we will approximate it as follows: randomly choose a subset of the sample of size $k$ with uniform ...
2
votes
1answer
122 views

Statistically Approximating Clicks From Limited Data

Assume a business started in January 2014. I have the following daily data (from June 2014 to December 2014): 1. Number of people who joined the website; 2. Number of people who left the website; ...
3
votes
1answer
91 views

On using one distribution to approximate another

Every basic text about different statistical distributions make notions about when one distribution can be approximated using another one. For example, we are told that the binomial distribution with ...
2
votes
1answer
2k views

Is it possible to get confusion matrices from AUC?

When I have one confusion matrix for each cutoff level (from 0.00 to 0.99), I can compute AUC coefficient. It looks like: ...
1
vote
0answers
26 views

Are there any approximation for ratios of factorials (and other similar high-precision ratios with rounding issues?) [duplicate]

I was wondering if there is some mathematical approximation for calculating the ratios of factorials. I don't mean a shortcut to deriving the factorial results through a means like Stirling's Formula, ...
3
votes
2answers
3k views

How many times do I have to roll a dice to get six six times in a row? [duplicate]

I wonder if there's any exact way to find out these two things about six-sided dice: How many throws would be necessary to get six times the same number (let's say six) in a row? What's the ...
2
votes
0answers
83 views

linearization of an estiamtor

Suppose we have two variables $x$ and $y$ defined in some population, with all values of $x$ known. A Poisson sample is drawn, with corresponding inclusion probabilities $\pi_k$ that are proportional ...
0
votes
1answer
133 views

Backward message passing in variational Bayesian inference

I have come across in a research paper that, I do understand the logic. But the paper has't mentioned about the way of updating $\eta_{t}$. When I asked from the authors they said when we equate the ...
3
votes
2answers
901 views

Function Approximation vs. Regression

Some background before I state the questions: I have a $d$-dimensional random vector $X=(X_1,\ldots,X_n)$ and a function $f:\mathbb{R}^d\rightarrow\mathbb{R}$. Ultimately my goal is to understand $f$ ...
1
vote
0answers
242 views

Approximate Probability Distribution Function

I am trying to approximate a large discreet probability distribution function using a histogram with a small number of entries. I.e., create a piece-wise first-order polynomial approximation for a ...
0
votes
1answer
71 views

Reservoir sampling with a computationally expensive weight function

I have a large dataset, and I want to obtain a small sample of it of size K, weighted by a function f(x) which is expensive to compute (I'm ok computing it O(K) times, but not too much more). Suppose ...
3
votes
0answers
43 views

Error bounds when approximating densities

I was curious whether it is possible to obtain approximation error bounds on continuous densities from approximation error bounds of random variables. To make my question more precise: We consider ...
6
votes
1answer
680 views

Numbers too large for R. How to approximate probability mass function?

Social network data is frequently found in a two-mode form: people vs. events they attend, people vs. classes they attend, countries vs. treaties they sign, etc. A strategy for analyzing this data is ...
0
votes
1answer
268 views

Numerical approximation of percentiles from arbitrary pdf

Given an easily-computable probability density function $f(x)$, what algorithm can we use to numerically approximate percentiles? For instance, we might be looking for $x$ such that given $X \sim f(x)...
1
vote
1answer
42 views

Do I need to care about constants in Expectation Propagation

I am trying to approximate a certain factor in my graph. Following Tom Minka's tutorial what I have to do is as follows: $$ \prod_{i=1}^3 q_{w_i}(\pi_2)\approx \int p(\pi_2|w_1)q_{\pi_1}(w_1)\prod_{...
4
votes
2answers
129 views

Log approximation

Can somebody help me with this approximation: as $m$ gets really large, $\dfrac{r}{m}$ gets really small and hence $\log\left(1+\dfrac{r}{m}\right) \sim \dfrac{r}{m}$. I don't understand how did $\...
1
vote
0answers
103 views

How accurate does the kernel density estimate need to be to use it in the mean shift algorithm?

Mean shift is an iterative procedure for locating the maxima of a density function, given discrete data sampled from that function. It is useful for detecting the modes of this density. How accurate ...
7
votes
5answers
2k views

Approximation of logarithm of standard normal CDF for x<0

Does anyone know of an approximation for the logarithm of the standard normal CDF for x<0? I need to implement an algorithm that very quickly calculates it. The straightforward way, of course, is ...
0
votes
1answer
58 views

Simple approximation of tail surprisal of poisson distribution

I want to determine (in an algorithm) the approximate surprisal of getting an outcome "as extreme as $k$" from the $Poisson(\lambda)$ distribution. My original plan was to use $-log_2(1/2-|1/2-F_{...
0
votes
0answers
596 views

Skew normal approximation of Poisson distribution

What is the skew normal approximation to Poisson($\lambda$)? Am I doing this wrong?
3
votes
1answer
2k views

Sum of Dependent Poisson Random Variables

I am working with the distribution of the sum of two dependent random variables. In my problem, there are two unobserved events, X and Y, where X precedes Y and Y is a function of the outcome of X, ...
3
votes
2answers
109 views

Is $F(E[Y_n]) \approx E[F(Y_n)]$ a reasonable approximation?

Studying the asymptotic distribution of order statistics I came across this approximation: $$F \left( E \left[ Y_n^{\left(n \right)} \right] \right) \approx E \left[ F \left( (Y_n^{\left( n \right)} \...
4
votes
0answers
85 views

Likelihood Function for Complicated Transformations

Suppose that data X have a Normal distribution with some mean $\mu$ and some variance $\sigma^2$. However, you don't get to see X. Instead, you see $Y = g(X)$ where $g$ is a known function. Assume ...
2
votes
0answers
157 views

expectation of multivariate discrete distribution

The expectation of a function $f(x)$ over a probability distribution $p(x)$ where $x=(x_1,\dots,x_n)$ and $x_i \in \left\{1,\dots,K\right\}$ requires a summation over all possible $K^n$ combinations. ...
2
votes
0answers
186 views

deterministic sampling for multivariate discrete distributions

Unscented transform can approximate expectations well via deterministic sampling. if $f(x) = N(\mu,\Sigma)$ and $x\in \mathbb{R}^d$ then $\int f(x) h(x) \approx \frac{1}{2d}\sum_{i=1}^{2d} h(x_k)$ ...
0
votes
0answers
53 views

Approximations to Normality

I have heard of Wilson-Hilferty approximation of Chi-Square Distribution to Normal Distribution. 1) Can this approximation be used with regard to testing of parameters of Chi-Square Distribution? 2)...
2
votes
1answer
102 views

help with this gradient computation in Expectation Propagation

I am trying to use Expectation propagation (EP) for approximating a posterior distribution in the Gaussian family. In this case, it is done by finding the Gaussian distribution with the same first and ...
7
votes
2answers
4k views

When is a second hidden layer needed in feed forward neural networks?

I'm using a feed forward neural network to approximate a function with 24 inputs, and 3 outputs. Most of the literature suggests that a single layer neural network with a sufficient number of hidden ...
1
vote
0answers
515 views

computing KL divergence: M projections for arbitrary distributions

Background I have a generative model for a process that can be described as follows: $$ y = t(x, w) + e $$ where $x$ and $y$ observations of a set of random variables which are related by a non-...
2
votes
1answer
203 views

getting from Edgeworth expansions to Cornish Fisher Expansions

I am currently trying to work out how to get from the Edgeworth expansion to the Cornish-Fisher expansion. I use van-der-Vaarts "Asymptotics Statistics" and Hall's book on Edgeworth expansions and the ...
8
votes
2answers
2k views

variational bayes vs expectation propagation

Just wondering what are the advantages of using one over the other. I'm just looking for some general answers here. For starters: VB gives a guaranteed lower bound for the likelihood. EP is faster? ...