Questions tagged [approximation]

Approximations to distributions, functions, or other mathematical objects. To approximate something means to find some representation of it which is simpler in some respect, but not exact.

Filter by
Sorted by
Tagged with
12
votes
1answer
389 views

Should degrees of freedom corrections be used for inference on GLM parameters?

This question is inspired by Martijn's answer here. Suppose we fit a GLM for a one parameter family like a binomial or Poisson model and that it is a full likelihood procedure (as opposed to say, ...
2
votes
2answers
556 views

Regression for function approximation

I have a program for heat exchanger calculations which uses correlations that are complex and highly non-linear. I need to come up with an approximation of the function using regression. The ...
2
votes
0answers
79 views

Numerical method to compress empirical probability distribution

I am trying to grapple with the following problem. I have an application that develops empirical distributions. In essence, I end up with a histogram of equally spaced $x$ values, with both a $max$...
7
votes
1answer
832 views

Is applying the CLT to the sum of random variables a good approximation?

I use $(\mu, \sigma^2)$ to mean a distribution with mean $\mu$ and variance $\sigma^2$, $\mathcal{N}$ added to mean the normal distribution. Let's suppose $X_1, \dots, X_n\overset{\text{iid}}{\sim}(\...
1
vote
0answers
423 views

Understanding a Taylor expansion for the bias of local polynomial regression

I'm trying to understand the proof of an expression for the asymptotic bias in local polynomial regression of degree $p\ge0$. Specifically, I'm distraught with equation $(3.59)$ on page 102 of this ...
6
votes
1answer
76 views

Expected value of a "logistic uniform" multivariate

Let $\mathbf{a}_1,\ldots,\mathbf{a}_n \in \mathbb R^d$ and $b_1,\ldots,b_n \in \mathbb R$ be fixed. For $\mathbf{x} \sim \mathcal U([0,1]^d)$ and $j \in \{1,\ldots,n\}$, consider the real variable ...
4
votes
0answers
412 views

Convergence of approximate Gibbs sampling

We have a bivariate random variable $(X,Y)$ for which sampling is challenging. If we were to know how to sample from the conditionals $(X|Y)$ and $(Y|X)$, we could get samples from the joint using ...
3
votes
1answer
407 views

Why is p(x|z) tractable but p(z|x) intractable?

In variational methods, given a set of latent variables $z$ corresponding to visible variables $x$, why is it that the probability distribution $p\left(x\middle|z\right)$ is tractable to compute, but $...
2
votes
1answer
80 views

Poisson Approximation to Binomial (How to proceed)

A charity issues a large number of certificates each costing $£10$ and each being repayable one year after issue. Of these certificates, $1$% are randomly selected to receive a prize of £10 such that ...
5
votes
1answer
171 views

Second order approximations of expected exponentiated sums of dependent Bernoulli random variables

I would like to approximate $\mathbb{E}[e^y]$ for $y=\sum_i^n c_i x_i$, and $x_i\sim \text{Bernoulli}(p_i)$ ($p_i$'s distinct) using $p_i$ and $\text{cov}(x_i,x_j)$. The $c_i\in \mathbb{R}$ are fixed, ...
1
vote
0answers
317 views

Approximating a 3d function based on 3d scattered points

Hello I have many 3d points (x, y, z). I have averaged every z with a matching (x, y) value and plotted the resulting means as a surface. The data looks pretty good: The green indicates where the ...
17
votes
3answers
2k views

Universal approximation theorem for convolutional networks

The universal approximation theorem is a quite famous result for neural networks, basically stating that under some assumptions, a function can be uniformly approximated by a neural network within any ...
1
vote
1answer
68 views

Approximating interactions in OLS

Suppose I have a simple linear model with two variables and their interactions: $y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_1 x_2 + \beta_4 x_1^2 + \beta_5 x_2^2 + \epsilon$ where the $\...
2
votes
0answers
90 views

Universal approximation capability of neural networks with random weights

There is a ton of literature (see, for example, a highly cited paper by Huang et al. (2006)) on neural networks with random weights (NNRWs), i.e. neural networks whose weights are random except for ...
2
votes
1answer
341 views

Estimate monotone function from noisy obsersations

Assume that $y=f(x)$ is an unknown monotonically increasing function of variable $x$. We have access to $N$ observations of this function given as tuples $(x_i,y_i)$, such that $y_i=f(x_i)$. The ...
1
vote
1answer
2k views

Approximating the expected value and variance of the function of a (continuous univariate) random variable

Let $X$ be a univariate continuous random variable (r.v.). Let $g$ be a smooth real function defined on the sample space of $X$. I have been told that the following approximations are true: $$ \...
1
vote
0answers
35 views

Correcting wild deviations in polynomial interpolation/regression

When using polynomials to do spline interpolation or least-squares regression on a set of points which are not evenly spaced, there is a tendency for the polynomial to deviate wildly in those regions ...
2
votes
2answers
660 views

Why isn't the Welch–Satterthwaite equation nonsensical?

As I understand it, the Welch–Satterthwaite equation says that if $s_i^2$ is the sample variance of group $i$ (with $n_i$ samples), then, assuming that the measurements in each group are iid's that ...
3
votes
2answers
2k views

Polynomial approximations of nonlinearities in neural networks

Imagine, that the only operations I have are scalar addition and scalar multiplication and I want to implement different nonlinearities for neural networks with them. The only option I see here is to ...
2
votes
0answers
32 views

How well can an AR(p) process model any given stationary time series?

Are there any theorems which tell us how well AR(p) models are able to approximate any stationary finite time series? If so, what are the relevant results?
3
votes
2answers
440 views

Natural log approximation

I've got an equation that contains $$x^p - 1$$ $x$ is any positive number (such as 2) and $p$ is a small positive number close to 0 (such as 0.001). For some reason (that I may have known in High ...
1
vote
1answer
107 views

How to deal with increasing action space in TD learning using linear function approximation

I am working on an application of reinforcement learning in an environment in which the number of possible actions increasing throughout an episode. The first step has only 8 possible actions, which ...
2
votes
1answer
1k views

Lower Bound on the Total Variation Distance between two Binomials

Let $X= B(n,1/2)$, $Y=B(n,1/2 + \delta)$, for a small $\delta >0$ be two Binomial Distributions. Question 1. I am looking for a lower bound on the Total Variation Distance the two Binomials ...
2
votes
1answer
121 views

What are some of the common techniques for density estimation?

I'm trying to estimate the probability density function of a real random variable given its iid realizations. What are some of the standard techniques to do this? One method I have heard of is the ...
2
votes
3answers
133 views

How to calculate or approximate the integral $\int_{-\infty}^{\infty}\sigma(x)(1-\sigma(x))\mathcal{N}(x|0,1)dx$

$\sigma(x)$ is the sigmoid function, that is, $\sigma(x)=\frac{1}{1+e^{-x}}$. And $x$ is from a Normal distribution of 0 mean and 1 variance. Now I'd like to calculate the expectation of the ...
1
vote
1answer
85 views

Is approximate normality limited to the exponential family?

A GLM looks like $$g(\mu) = X\beta,\ \ \mu = EY_i$$ where $Y_i$ is an exponential family. It is commonly assumed for a decent sample size that $\beta$ is approximately normal with mean $\hat{\beta}$ ...
2
votes
0answers
71 views

Is vcov the information matrix /n from the theory?

If I fit a model m <- glm/gam/gamm/lme/whatever(y ~ x + z, family = some exponential family) and extract coef(m) vcov(m) ...
1
vote
1answer
41 views

Simultaneously finding least squares predictors of every feature based on all others

Given set of $d$ datapoints and $f$ features in a $X_{f,d}$ data matrix, I'm trying to fit a least squares predictor of every feature based on every other feature. In other words for every feature $i$...
2
votes
1answer
298 views

Kahn Pseudo Normal distribution

I read about a so-called "Kahn Pseudo Normal" distribution in a forum post. It says when $U$ is uniformly distributed over $[0,1]$, then $\log(1/U-1)$ is approximately normal distributed, that is ...
3
votes
1answer
199 views

A Bound based on Jensen's Inequality

Consider: $$X \sim \text{Gamma}(\alpha, \beta)$$ $$Y = \frac{1}{X+c}, \ c > 0$$ I am interested in $E(Y)$, which I'm pretty sure is intractable... $$E(Y) = \frac{\beta^\alpha}{\Gamma(\alpha)}\...
0
votes
0answers
434 views

Multivariable piece-wise linear approximation in R?

I am trying to run linear approximation on a function of 2 or 4 variables. The approx function in R is a very nice, optimized time saver, but it only works for 1 dimension functions I think. Anyone ...
0
votes
1answer
115 views

How to determine recurring use numbers for my app

A few months ago, I wrote an Android App. I am getting two key pieces of data from this app regarding its usage. The first metric, is number of downloads per day. The second metric, is basically, ...
0
votes
1answer
36 views

Algorithm to Fit Intuitively Important Data Points

It is hard to formalize what I am asking without knowing the answer, but in experimental mathematics it is very common to see a graph of points hinting at limiting behavior. The graph below makes me ...
5
votes
3answers
3k views

regression with constraints

I have some domain knowledge I want to use in a regression problem. Problem statement The dependent variable $y$ is continuous. The independent variables are $x_1$ and $x_2$. Variable $x_1$ is ...
0
votes
0answers
27 views

Are there barriers to sampling derivatives of $\log(p)$?

Suppose I have a probability distribution $p_\theta$ on $\Bbb R^n$ dependant on some parameters $\theta$. A natural problem is to evaluate the derivative of some expectation by the parameters: $$d_\...
3
votes
1answer
657 views

Finding the optimal number of hidden nodes and training epochs for function approximation [duplicate]

In an attempt to find the optimal number of hidden nodes and number of training epochs (to obtain optimal performance while not over-fitting) in a single-hidden layer neural network, I generated the ...
9
votes
4answers
3k views

What is the CDF of the sum of weighted Bernoulli random variables?

Let's say we have a random variable $Y$ defined as the sum of $N$ Bernoulli variables $X_i$, each with a different, success probability $p_i$ and a different (fixed) weight $w_i$. The weights are ...
3
votes
1answer
2k views

Function approximation using multilayer perceptron (neural network)

I've been asked to solve a problem for a project. I'm working on Python or R. I need to approximate a function with multiplayer perceptron (neural network). The function is: $y= 2\text{cos}(x)+4$ on ...
0
votes
0answers
56 views

Data approximation with unequal intervals

For some sampled function of real variable, defined on [a, b], how can [a, b] be split into subintervals of unequal width, so that for each interval one can say, "On this interval, function is ...
6
votes
1answer
772 views

Half-normal probability plot

To construct the half-normal probability plot, plot the absolute values in a certain statistical diagnostic (residual, leverage, Cook distance and others) versus $z_i$ where: $\displaystyle z_{i} = \...
1
vote
0answers
113 views

Estimate mean and variance of pdf from truncated taylor expansion of logarithm of pdf

In a maximum likelihood fit, one estimates the parameter with the mode of the likelihood $L$, and the variance of this estimator with the second derivative of $\log(L)$: $$ \bar\theta = \mathrm{Mode}[...
16
votes
1answer
2k views

Do Gaussian process (regression) have the universal approximation property?

Can any continuous function on [a, b], where a and b are real numbers, be approximated or arbitrarily close to the function (in some norm) by Gaussian Processes (Regression)?
2
votes
0answers
545 views

Approximating Uniform Distribution with Mixture of Gaussians

Let $T$ be a compact, connected, proper subset of $\mathbb{R}^3:\quad T \subset \mathbb{R}^3$. Further let $\left\{ \boldsymbol{\mu}_i \right\}_{i=1}^n$ be a given finite set of $n$ point in $T$: $$ \...
1
vote
1answer
526 views

Approximating a joint distribution from marginals of sums of variables

Suppose I have a set of random variables ${X_1, X_2, ..., X_n}$. For each of the variables, I have the marginal distribution. Furthermore, I have the marginal distribution for various sums of subsets ...
3
votes
1answer
40 views

Using clusters to estimate model variance

I am working with a blackbox prediction model which takes known inputs and outputs a single mean response. I know this model's residuals to be heteroskedastic, but also can assume the error term of ...
15
votes
5answers
2k views

Approximation error of confidence interval for the mean when $n \geq 30$

Let $\{X_i\}_{i=1}^n$ be a family of i.i.d. random variables taking values in $[0,1]$, having a mean $\mu$ and variance $\sigma^2$. A simple confidence interval for the mean, using $\sigma$ whenever ...
0
votes
1answer
83 views

Calculate probabilities from binomial or normal distribution

I have the following experiment: Two balls are let fall from a certain high (could be the same or different) Then for each of them, I get a random number from 0 to 1 with a uniform distribution and ...
4
votes
2answers
288 views

Claim that a distribution is almost Gaussian

I prove a some theorem under the assumption that some random variable X is Gaussian. Now in practice, in my experiments section I have real-world samples from ...
3
votes
2answers
505 views

Show that $1-\Phi(x)$ is approximately $\varphi(x)/x$ for large $x$ (standard-normal random variable) [duplicate]

Demonstrate that, for a standard normal random variable: $$1-\Phi(x) \approx \frac{\varphi(x)}x$$ for large values of $x$.
2
votes
0answers
29 views

When is it possible to estimate the non-linearity error when approximating data with a linear model?

The most common form of linear regression estimates the best values of $\vec{\beta}$ and $\sigma^2$ assuming that data is sampled from a model $y = \vec{\beta} \cdot \vec{x} + \vec{\epsilon}$ where $\...

1 2 3
4
5
9