We’re rewarding the question askers & reputations are being recalculated! Read more.

Questions tagged [approximation]

Approximations to distributions, functions, or other mathematical objects. To approximate something means to find some representation of it which is simpler in some respect, but not exact.

Filter by
Sorted by
Tagged with
1
vote
0answers
61 views

Optimization: Approximate function - Which points to evaluate next?

I am looking for a statistical method (and a link to a nice R package would be cool too!) which allows me to find which point to evaluate next for a given function. I have a non-stochastic function z ...
2
votes
1answer
406 views

Normal approximation to Poisson: With Continuity Correction the Approximation Seems Worse

This is Exercise 3 in Section 6.3 of Probability and Statistics, 4th edition, by DeGroot and Schervish: Suppose that the distribution of the number of defects on any given bolt of cloth is the ...
8
votes
1answer
274 views

Analytically solving sampling with or without replacement after Poisson/Negative binomial

Short version I am trying to analytically solve/approximate the composite likelihood that results from independent Poisson draws and further sampling with or without replacement (I don't really care ...
1
vote
2answers
65 views

Compression of 18000 curves

I have over $18000$ curves that I need to compress to save $\geq 50\%$ of space. Each curve is described by points $f(1), f(2), ..., f(96)$, each $f(x)$ is 8-bit long. The curves in compressed form ...
6
votes
1answer
203 views

Approximating the mathematical expectation of the argmax of a Gaussian random vector

Let $X = \left( {{X_1},...,{X_n}} \right) \sim \mathcal{N}\left( {{\mathbf{\mu }},{\mathbf{\Sigma }}} \right)$ be a Gaussian random vector and $I = \mathop {\arg \max }\limits_{i = 1,n} {X_i}$. $I$ ...
4
votes
1answer
141 views

CLT for random sums: Anscombe's Theorem vs. “classical” version

Given a compound Poisson distribution $$S(t):=\sum_{k=1}^{N(t)} X_{k}$$ with $N(t)\in\mathbb{N},\,t\geq0$ a Poisson process with rate $\lambda.$ $X_{k}$ are non-negative iid random variables such ...
-2
votes
2answers
931 views

Continuous approximation to binomial distribution

Consider an integer variable $k$ that follows a binomial distribution, $$\binom{N}{k}p^{k}\left(1-p\right)^{N-k}$$ with total draws $N$ and probability of success $p$. I am interested in the ...
0
votes
0answers
53 views

What's a good approximation to the count distribution of people for days of birth?

I have data on $n$ people's dates of birth and let's ignore the years and look only at the $k$ = 366 days of the year (including Feb 29). Assuming that dates of birth are uniformly and independently ...
3
votes
1answer
43 views

Difference vs log-difference: do they behave similarly?

Consider two positively-valued time series, $\{X(t),Y(t)>0|t\geq0\}$. Now consider two transformations: $$ U(t) = Y(t) - \beta X(t),\\ V(t) = \ln{[Y(t)]} - \ln{[\beta X(t)]}, $$ with $\beta>0$ ...
2
votes
1answer
125 views

Estimate correlation between data and data-fit model for variance reduction in Monte Carlo estimate

Say that I want to estimate the mean of a function $f$, $\mathbb{E}[f(X)]$, given some input distribution $x\sim P(x)$. I don't know anython about the form of $f$ except that it is smooth and ...
3
votes
1answer
100 views

Estimating function with Gaussian Procceses

I do not have strong math background, but I am trying to understand Gaussian Processes by example using the lecture Machine learning - Introduction to Gaussian processes by Nando de Freitas. Here is ...
5
votes
1answer
93 views

Evaluate approximation of PCA from randomized algorithms

I have been comparing different PCA implementations (some via explicit calculation of the covariance matrix, some with randomized/truncated SVD) in terms of speed, and now wanted to compare how good ...
0
votes
1answer
51 views

Contraction or Reduction of a Neural Network

Circuit size reduction is common practice in Theoretical Computer science. It is very common to approximate a circuit with a smaller circuit (Or a polynomial and so on). Are there any such techniques ...
2
votes
0answers
55 views

Better skill checks for RPGs - Conditional probability given 2 independent parameters

I am trying to find a better way (theoretically, not practically speaking) to do a skill check for a Skill Trial in RPGs. In several RPGs, a skill check consists of a Playing Character (PC) trying to ...
5
votes
1answer
152 views

How good an approximation is sampling with replacement to sampling without replacement?

I'm learning about probability with Feller's book and he states that, when the population size $n$ is big in comparison with the sample size $r$, then $n_r$, which is a shorthand for $\frac{ n!}{(n-r)!...
0
votes
0answers
12 views

How to approximate table-defined function using non-linear least squares

I read least squares method and haven`t found a good example of using non-linear least squares. Problen: I have an arbitrary values for x = 1, 2, 3, 4, 5, therefore, I have a table-defined function. ...
0
votes
0answers
32 views

Approximation of fractional function that has real-power numerator

I have the function $f(x)=\frac{(1+x)^k}{1+ax}$, where $x>0, 0<a<k<1$. The function has only one maximum at $x_0=\frac{a-k}{a(k-1)}$, increases on the left of $x_0$ and decreases on the ...
3
votes
0answers
26 views

Reducing a logistic model used for prediction

I'm developing a logistic regression used for prediction. I have pre-selected, based on prev. literature, 15 candidate predictors (fitting my ~200 events). Now, I want a reduced/more parsimonious ...
0
votes
1answer
37 views

Regularization for approximation in neural networks

In the case of approximation tasks using neural networks, should we standardize the data, as in the classification ?
3
votes
0answers
224 views

Testing a low rank estimator of a covariance matrix

I am exploring ways to reduce the noise of a covariance matrix estimator when the number of variables is greater than the number of observations, i.e. $n > t$. First, I tried using a low rank ...
1
vote
1answer
125 views

Find $\mathbb{E} \bigg[ \frac{\textbf{h}^{H} \textbf{y}\textbf{y}^{H} \textbf{h}}{ \| \textbf{y} \|^{4} } \bigg]$ with Mathematica? [duplicate]

Considering the following complex random vectors (Complex Gaussian random variables): \begin{align} \textbf{h} &= [h_{1}, h_{2}, \ldots, h_{M}]^{T}\ \ \sim \mathcal{CN}(\textbf{0}_{M},d\textbf{I}_{...
2
votes
0answers
55 views

What is this approximation called?

In Bayesian statistics, you have a likelihood and a prior, $f(x_1,\ldots,x_n \mid \theta)$ and $\pi(\theta)$ respectively, and you use these to obtain the posterior $\pi(\theta \mid x_1, \ldots, x_n) \...
1
vote
1answer
93 views

The distribution of the product of a multivariate normal and a lognormal distribution

If $$X=\left(\begin{array}{c} X_{1}\\ X_{2} \end{array}\right)\sim N\left[\left(\begin{array}{c} \mu_{X_{1}}\\ \mu_{X_{2}} \end{array}\right),\left(\begin{array}{cc} \sigma_{X_{1}}\\ \sigma_{X_{1}X_{2}...
2
votes
1answer
264 views

Expectation of the absolute value in a sequence of Bernoulli trials

On this tweet: Can I get some help in understanding the proposed solution by N. Taleb: It is not clear how he describes success, i.e. $n-x$ to come up with $\binom{n}{n-x}.$ It almost seems as ...
2
votes
1answer
185 views

Variational Inference of Univariate Gaussian mixtures

I am reading this paper. In the paper, they use an example of mixture of unit-variance univariate Gaussians with the following parameterization: \begin{align} \mu_k & \sim \mathcal{N}(0, \sigma^2)...
16
votes
1answer
3k views

How does a random kitchen sink work?

Last year at NIPS 2017 Ali Rahimi and Ben Recht won the test of time award for their paper "Random Features for Large-Scale Kernel Machines" where they introduced random features, later codified as ...
8
votes
3answers
2k views

Does the universal approximation theorem for neural networks hold for any activation function?

Does the universal approximation theorem for neural networks hold for any activation function (sigmoid, ReLU, Softmax, etc...) or is it limited to sigmoid functions? Update: As shimao points out in ...
1
vote
0answers
13 views

Reference - Moment approximation error vs. L2 error

The following is a standard, but I couldn't find a single textbook for it. Could you help me find a reference for this lemma? Given $f,f_N \in L^2 (\Omega) \cap L^1 (\Omega)$ for some probability ...
1
vote
1answer
47 views

Probability Distribution Approximation problem

My problem is 7.6 from A First Course in Probability and Statistics. The answer is provided in the book but not how to arrive at the solution. I thought I understood the chapter fairly well, but I can'...
4
votes
0answers
265 views

Expectation of the softmax transform for Gaussian multivariate variables

Prelims In the article Sequential updating of conditional probabilities on directed graphical structures by Spiegelhalter and Lauritzen they give an approximation to the expectation of a logistic ...
8
votes
2answers
373 views

What is the justification of using taylor approximations inside expectation operators?

I sometimes see people use taylor approximation as follows: $$E(e^x)\approx E(1+x)$$ I know that the taylor approximation works for $$e^x \approx 1+x$$ But it is not clear to me that we can do the ...
11
votes
1answer
191 views

Should degrees of freedom corrections be used for inference on GLM parameters?

This question is inspired by Martijn's answer here. Suppose we fit a GLM for a one parameter family like a binomial or Poisson model and that it is a full likelihood procedure (as opposed to say, ...
2
votes
2answers
266 views

Regression for function approximation

I have a program for heat exchanger calculations which uses correlations that are complex and highly non-linear. I need to come up with an approximation of the function using regression. The ...
1
vote
0answers
71 views

Numerical method to compress empirical probability distribution

I am trying to grapple with the following problem. I have an application that develops empirical distributions. In essence, I end up with a histogram of equally spaced $x$ values, with both a $max$...
7
votes
1answer
502 views

Is applying the CLT to the sum of random variables a good approximation?

I use $(\mu, \sigma^2)$ to mean a distribution with mean $\mu$ and variance $\sigma^2$, $\mathcal{N}$ added to mean the normal distribution. Let's suppose $X_1, \dots, X_n\overset{\text{iid}}{\sim}(\...
1
vote
0answers
196 views

Understanding a Taylor expansion for the bias of local polynomial regression

I'm trying to understand the proof of an expression for the asymptotic bias in local polynomial regression of degree $p\ge0$. Specifically, I'm distraught with equation $(3.59)$ on page 102 of this ...
4
votes
0answers
40 views

Expected value of a “logistic uniform” multivariate

Let $\mathbf{a}_1,\ldots,\mathbf{a}_n \in \mathbb R^d$ and $b_1,\ldots,b_n \in \mathbb R$ be fixed. For $\mathbf{x} \sim \mathcal U([0,1]^d)$ and $j \in \{1,\ldots,n\}$, consider the real variable ...
3
votes
0answers
294 views

Convergence of approximate Gibbs sampling

We have a bivariate random variable $(X,Y)$ for which sampling is challenging. If we were to know how to sample from the conditionals $(X|Y)$ and $(Y|X)$, we could get samples from the joint using ...
3
votes
1answer
176 views

Why is p(x|z) tractable but p(z|x) intractable?

In variational methods, given a set of latent variables $z$ corresponding to visible variables $x$, why is it that the probability distribution $p\left(x\middle|z\right)$ is tractable to compute, but $...
1
vote
1answer
55 views

Poisson Approximation to Binomial (How to proceed)

A charity issues a large number of certificates each costing $£10$ and each being repayable one year after issue. Of these certificates, $1$% are randomly selected to receive a prize of £10 such that ...
4
votes
1answer
133 views

Second order approximations of expected exponentiated sums of dependent Bernoulli random variables

I would like to approximate $\mathbb{E}[e^y]$ for $y=\sum_i^n c_i x_i$, and $x_i\sim \text{Bernoulli}(p_i)$ ($p_i$'s distinct) using $p_i$ and $\text{cov}(x_i,x_j)$. The $c_i\in \mathbb{R}$ are fixed, ...
1
vote
0answers
178 views

Approximating a 3d function based on 3d scattered points

Hello I have many 3d points (x, y, z). I have averaged every z with a matching (x, y) value and plotted the resulting means as a surface. The data looks pretty good: The green indicates where the ...
9
votes
2answers
848 views

Universal approximation theorem for convolutional networks

The universal approximation theorem is a quite famous result for neural networks, basically stating that under some assumptions, a function can be uniformly approximated by a neural network withing ...
1
vote
1answer
57 views

Approximating interactions in OLS

Suppose I have a simple linear model with two variables and their interactions: $y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_1 x_2 + \beta_4 x_1^2 + \beta_5 x_2^2 + \epsilon$ where the $\...
2
votes
0answers
81 views

Universal approximation capability of neural networks with random weights

There is a ton of literature (see, for example, a highly cited paper by Huang et al. (2006)) on neural networks with random weights (NNRWs), i.e. neural networks whose weights are random except for ...
2
votes
1answer
184 views

Estimate monotone function from noisy obsersations

Assume that $y=f(x)$ is an unknown monotonically increasing function of variable $x$. We have access to $N$ observations of this function given as tuples $(x_i,y_i)$, such that $y_i=f(x_i)$. The ...
1
vote
1answer
709 views

Approximating the expected value and variance of the function of a (continuous univariate) random variable

Let $X$ be a univariate continuous random variable (r.v.). Let $g$ be a smooth real function defined on the sample space of $X$. I have been told that the following approximations are true: $$ \...
1
vote
0answers
28 views

Correcting wild deviations in polynomial interpolation/regression

When using polynomials to do spline interpolation or least-squares regression on a set of points which are not evenly spaced, there is a tendency for the polynomial to deviate wildly in those regions ...
2
votes
2answers
371 views

Why isn't the Welch–Satterthwaite equation nonsensical?

As I understand it, the Welch–Satterthwaite equation says that if $s_i^2$ is the sample variance of group $i$ (with $n_i$ samples), then, assuming that the measurements in each group are iid's that ...
1
vote
2answers
1k views

Polynomial approximations of nonlinearities in neural networks

Imagine, that the only operations I have are scalar addition and scalar multiplication and I want to implement different nonlinearities for neural networks with them. The only option I see here is to ...