Questions tagged [approximation]
Approximations to distributions, functions, or other mathematical objects. To approximate something means to find some representation of it which is simpler in some respect, but not exact.
452
questions
0
votes
0
answers
10
views
approximating the density of the studentized range distribution
Is anyone aware of an approximation to the density function for the studentized range distribution https://en.wikipedia.org/wiki/Studentized_range_distribution ? I've found a fast approximation for ...
- 41
0
votes
0
answers
18
views
Non-stationary Random Fourier Features
Random Fourier Features (RFFs) were introduced by A. Rahimi and B. Recht in their 2007 publication Random Features for Large-Scale Kernel Machines. RFFs are based on Bochner's theorem, which applies ...
- 1
1
vote
0
answers
19
views
Restrictions on sample cumulants/moments for truncated Edgeworth expansion
I'm trying to approximate an unknown distribution by a truncated Edgeworth series, with cumulants/central moments estimated from a large sample.
I notice though that I am getting negative tail ...
- 171
0
votes
0
answers
17
views
Overflow when computing binomial distribution for large n [duplicate]
How do you compute a binomial probability distribution for large $n$? If I try the following, I get an integer overflow in any programming language:
...
- 111
2
votes
2
answers
84
views
How should I deduce the variance and expectation of the log of a variable?
I read this paper "voom: precision weights unlock linear model analysis tools for RNA-seq read counts", in the methods, the "Delta rule for log-cpm" section:
The RNA-seq data ...
- 53
3
votes
1
answer
39
views
Robustness of Posterior distribution wrt likelihood function
Suppose we have
$$
X_1, \ldots, X_n \mid \theta \, \mathop{\sim}^{iid} \, L(\cdot \mid \theta), \quad \theta \sim \pi
$$
By Bayes' theorem, the corresponding posterior distribution is
$$
\pi_n(\mathrm ...
- 530
1
vote
2
answers
70
views
How can I use the Central Limit Theorem to calculate the distribution of $\bar{X}$?
The central limit theorem says that $$
\frac{\bar{X}-\mu}{\frac{\sigma}{\sqrt{n}}} \stackrel{\mathcal{D}}{\rightarrow} N(0,1)
$$
What is the distribution of $\bar{X}$? I've seen it given as $\sum X \...
- 101
0
votes
1
answer
44
views
Extract the functional mapping between input and output from a machine learning model
A lot of ML models, such as neural networks, are Universal Function Approximators. But when evaluating ML models, we use usually metrics, such as MSE or accuracy, to assess the performance of a ML ...
0
votes
0
answers
17
views
Sample Average Approximation Algorithm- Confusion on samples to be used
Considering a Machine Learning scenario with some pre-available training samples S.
In the objective function, let's suppose we have expectation over some reference distribution P0 whose parameter (...
- 401
0
votes
0
answers
24
views
Probability bound on Maxima under random sampling
I have a set $S$ = {$e_1,e_2,..e_{400}$} of 400 elements and a non-linear function $f:2^{(S)}\to[0,1]$ that takes a subset of $S$ and returns a real number in $[0,1]$. I want to compute the subset for ...
- 121
1
vote
0
answers
24
views
Do Neural Networks tend to have Zero Mean Errors in each Output?
My NN (a few linear layers with ReLUs + batch normalization, no activation in the last layer) learns to approximate vector-valued labels $y_z$ from data $z\sim\rho_z$ in a supervised way, i.e. net$(z)=...
- 111
0
votes
0
answers
44
views
Applying Variational inference to a simple case
My a goal is understand how variational inference works and be able to derive it in a simple way.
To do so, I need help deriving variational inference for this simple example.
Given a set of labels $L$...
- 367
1
vote
1
answer
143
views
Algorithm for approximating linear-interpolated curve
Goal
Given a curve defined by a set of (x, y) coordinates with linear interpolation, we want to find the best approximation using a smaller set of points (w/ linear interpolation) that fall along a ...
- 11
1
vote
1
answer
89
views
Comparing Gibbs sampler and variational inference
I am learning about variational inference and Gibbs simpler.
I am in the process of deriving variational inference on my own. In this process, I need to make a comparison with Gibbs sampler.
I am ...
- 367
0
votes
0
answers
35
views
Bayesian Linear Regression: Is there significant difference between setting a gamma prior or flat prior over hyper parameters?
I am reading Bishop’s Pattern Recognition and Machine Learning 2006 and I am confused about this claim that defining conjugate gamma prior distributions over hyper parameters alpha and beta leads to ...
- 13
0
votes
0
answers
21
views
Continuous representation of categorical distributions via piecewise truncated normal distributions?
I'm looking for a continuous representation/approximation of a categorical distribution. The idea is if I have a factor corresponding to say, BMI ranges, that I can encode this across the interval $[0,...
- 65
0
votes
0
answers
52
views
Aproximate maximum of two multivariate Gaussians with multivariate Gaussian
Given two multivariate Gaussians $G_1(\mathbf{x}), G_2(\mathbf{x})$ (not PDFs!) with the same center at the coordinate origin and different covariance matrix $\mathbf{F}_1, \mathbf{F}_2$, where $\...
- 1
2
votes
2
answers
64
views
Using 1-Layer Fully-Connected Neural Network to Appoximate Exponential Functions
Consider a 1-layer fully-connected neural network (FCNN) given by
$$
f(x) = \sum_{i=1}^n v_i\sigma\!\left({w_i}^T x\right)
$$
where $x,w_i\in\mathbb{R}^d$, $v_i\in\mathbb{R}$, and $\sigma(y)=\max(y,0)$...
- 121
0
votes
0
answers
45
views
four-point forward-difference formula using Newton's form for first order derivative [closed]
We know that ${f'(x) \approx \frac{f(x+h)- f(x)}{h}}$. If we have three points ${x_0 = x-h}$, ${x_1 = x}$, ${x_2 = x + h}$, we can compute the 3-point centered-difference formula using the Newton's ...
- 217
0
votes
0
answers
29
views
Understanding the Purpose of LogSumExp
According to Wikipedia, the LogSumExp function is a "smooth approximation to the maximum function mainly used by machine learning algorithms." Furthermore,
The LSE function is often ...
- 113
0
votes
0
answers
49
views
Approximate Expected Value of Product of Two Random Variables
I'm looking to find an approximation of $\mathbb{E}\left[ XY \right]$ in terms of $\mathbb{E}\left[ X \right]^n$, $\mathbb{E}\left[ Y \right]^n$, $\mathbb{E}\left[ X^n \right]$, $\mathbb{E}\left[ Y^n \...
- 111
2
votes
2
answers
86
views
Binomial to Poisson Approximation
So, a little context. The image you see is from the GCE A-LEVEL syllabus where they have defined the conditions for approximating binomial to poisson.
But why did they have mention that the ...
- 23
0
votes
0
answers
8
views
Bivariate normal CFD approximation using characteristic function
The normal distribution CFD can be approximated using
$$F_X (x)=P[X≤x]=\frac{1}{2}-\frac{1}{π} \int^{\infty}_{0}\operatorname{Re}\left[\frac{e^{-iux}\phi_X (u)}{iu}\right]du$$
where the characteristic ...
- 11
0
votes
0
answers
21
views
How much error will there be if one models the multivariate hypergeometric distribution using a multinomial distribution?
See the title. I stumbled upon this answer which explains the approximation in more detail. One can approximate the multivariate hypergeometric distribution by using the multinomial distribution.
...
- 101
0
votes
0
answers
6
views
Why approximation and estimation errors (of generalisation error decomposition) are called as such?
Approximation error: The degree to which the underlying distribution (D) can be well approximated by the hypothesis class (H)
Estimation error: Provided H, the degree to which the underlying ...
0
votes
0
answers
25
views
Nonlinear regression with derivative dependence
I am trying to perform a functional approximation on some experimental data.
I have a set of measurements of a d-dimensional state $\mathrm x \in \mathbb{R}^d$, together with velocities $\dot{\mathrm ...
0
votes
0
answers
34
views
Best approximation for the size of a test
Let $X \sim \mathrm{Bernoulli}(\vartheta)$ for some unknown $\vartheta \in (0,1)$, and let $(X_1, …, X_n)$ be a moderately large IID sample for $X$.
Let $\vartheta_0 \in (0,1)$. I want to test $H_0 \...
- 101
4
votes
0
answers
123
views
Can GMM approximate any given probability density function?
I am currently studying on Bayesian models, and still new to probability theory.
I learned that Gaussian Mixture Model is used to represent the distribution of given population as a weighted sum of ...
- 63
0
votes
1
answer
49
views
How do you define a good approximation for a probability distributions?
We know a series of probability distribution approximations that are considered good as long as some condition holds. A few examples are:
Binomial can be approximated by Normal if $np(1-p) > 10$ ...
- 405
1
vote
0
answers
24
views
How to prove the mean of a sample approximates well to the mean of the population
Suppose I have a population whose distribution is definitely not normal but both the population and sample size will be large. Is there any way I can prove/ show that the mean of a large enough sample ...
0
votes
2
answers
63
views
Approximate distribution of test statistic for weighted sample mean
Let
$$ R_{i}(t) \sim \mathcal{N}(\mu_i, \sigma_i^2), $$
denote the one period return distribution for asset $i$, from which we observe the iid samples $\{R_i(t)\}_{t=1}^{n_i}$. The MLE sample mean and ...
0
votes
0
answers
22
views
Modifying Variational Inference to be robust to outliers?
Normally, for variational inference, you have some evidence data $Z$, you have some true distribution $P(X|Z)$, and you have a simpler parameterized distribution $Q(X|\theta)$, and you're trying to ...
- 401
0
votes
0
answers
48
views
How many components of a gaussian mixtures do I need to match moments up to the $r$-th order?
Suppose I have a ($k$-dimensional) random variable $X \sim D$ and I want to find a Gaussian Mixture $GM \sim \sum_{i=1}^C \pi_i \mathcal{N}(\mu_i, \Sigma_i)$ such that the moments of order $r'$, for $...
- 1
0
votes
0
answers
17
views
Asymptotic Distribution of Likelihood Ratio under Nonlinear Hypothesis
Suppose we are testing $\mathbf h(\boldsymbol\theta) = \mathbf 0$ versus $\mathbf h(\boldsymbol\theta) \neq \mathbf 0$ for a vector of parameters $\boldsymbol\theta \in \boldsymbol\Theta\subset \...
0
votes
0
answers
36
views
Best way to approximate head point having only face keypoints
I'm using the BlazeFace model from TensorFlow which only has this few keypoints:
I need those keypoints plus a head keypoint, like this one:
My question is, which would be the best way to ...
0
votes
0
answers
233
views
LSA and cosine similarity approximation with large matrix
This question is the non-r related version of this one
I have a tdm with 16k x 350k dimension, for which I am trying to get the document-document similarity (cosine type). With RSpectra I have found a ...
- 11
0
votes
0
answers
46
views
Distribution of the approximation error in Gaussian Process Regression (finite data setting)
I am learning about Gaussian Process Regression. I would like to have some references or results regarding the distribution of the error between a given function, and the posterior obtained in ...
3
votes
2
answers
484
views
Universal approximation of Gaussians
Can gaussian kernels reproduce non continuous L2 integrable functions? ( Do non continuous L2 integrable functions lie in the RKHS constructed by a Gaussian Kernel?)
Edit:
I think my question is being ...
- 75
1
vote
0
answers
225
views
Simplifying the Kullback-Leibler divergence for a sum of distributions
I want to find an approximation of a mixture of probability distributions that minimises the Kullback-Leibler divergence (KLD).
I need to verify my result, as it seems suspect.
We have a joint ...
- 81
1
vote
0
answers
41
views
Universal Approximation Capabilities of Mixture of Weibulls
Can a mixture of $N$ Weibull distributions approximate any continuous density with non-negative support, if $N$ is sufficiently large? (If so, a reference to the proof would be greatly appreciated).
(...
- 115
1
vote
0
answers
42
views
Is there a way to correct for degrees of freedom when using a generalized linear model with a Poisson distribution featuring random effects?
I am running a generalized linear mixed effect model with a Poisson distribution to analyse count data. The model has a random effect that takes into account multiple observation obtained by the same ...
- 11
0
votes
0
answers
29
views
Conservative coverage probability when an biased estimator is used for the variance
Suppose that $X_n\sim N(\mu, \sigma^2_n)$. Thus, to construct a 95% CI for $\mu$, we can use $X_n\pm 1.96 \sigma_n$. The coverage probability, $P(\mu\in [X_n\pm 1.96 \sigma_n])$, is equal to 95%.
...
- 820
2
votes
1
answer
123
views
Confidence interval for multiple samples of ratios of counts (in R)
Data and objective
I have count data from two groups, A and B, from across multiple samples. I want to estimate the average ratio of A to B across all samples, along with a confidence interval.
Issues
...
- 65
0
votes
0
answers
23
views
What is known, in principle, about the possibility of approximating the random discrepancy between a statistical estimate and its parameter?
The difference between the value of a statistical estimate and its parameter's value is almost never exactly $0$. For example, $r - \rho$, for a unique sample $r$, is likely to be some non-zero ...
- 194
1
vote
0
answers
41
views
A nondeterministic covariance-stationary process approximated by an ARMA process
We know that the Wold Decomposition Theorem says that any purely nondeterministic covariance-stationary process, $x = [x_t : t \in \mathbb{Z}]$, can be written as a linear combination of lagged values ...
- 1,027
7
votes
1
answer
192
views
Fast likelihood evaluation for Gaussian distribution with diagonal plus low rank covariance
Let's assume the likelihood
$$
y \sim\mathcal N_p(0, \Sigma + \Lambda\Lambda^\top)
$$
where $\Sigma$ is a diagonal $p \times p$ matrix and $\Lambda$ is a $p \times d$ matrix with $d \ll p$.
What is ...
- 530
3
votes
0
answers
94
views
Number of points a one hidden layer neural-network can interpolate
We am trying to understand the number of points that a neural network of a particular size can interpolate. I think this may be isomorphic to its degree of freedom? We are not interested in whether ...
- 495
1
vote
0
answers
78
views
Connection between mean field inference and mean field theory (physics)
In variational inference, the mean-field family of probability distributions is the set of distributions that factors over its terms (i.e. each component is independent of all others). This allows us ...
- 389
4
votes
1
answer
342
views
Is there a closed form approximation for the composition of the Gamma CDF with the inverse Normal CDF?
Given $k$, $\theta$ fixed shape and scale parameters for some Gamma distribution which has a CDF $F$. Let $G^{-1}$ be the inverse CDF of the standard Normal distribution. Consider the composition $H(x)...
- 43
0
votes
0
answers
84
views
Approximation of a polynomial via histogram
Note: I originally tried to pose this question generally, without discussing the specific type of stochastic process. I hope that this can still be an interesting question generally.
Assume that we ...
