Skip to main content

Questions tagged [taylor-series]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
10 views

Characterize conditions in which Taylor moment approximation is good

I am working with the Projected Gaussian, or Angular Gaussian distribution, which is given by $z = \frac{x}{||x||}$, where $x \sim \mathcal{N}(\mu, \Sigma)$. This is a distribution on the sphere in $\...
dherrera's user avatar
  • 1,268
0 votes
0 answers
14 views

Can I find the explicit feature map that generates exponent of a kernel?

Let's say I have a kernel $K$, and another kernel of the form : $$ K' = e^K $$ now I know how to prove K' is a kernel, I can do it using taylor expansion of $e^x$ around $0$, but let's say if I want ...
aroma's user avatar
  • 123
0 votes
0 answers
27 views

taylor approximation multivariate OLS coefficient

Say we have the following multivariate regression model: $ y = \beta_1 x_1 + \beta_2 x_2 + \varepsilon $ The OLS formula for the first coefficient looks like this $ \hat{\beta}_1 = \frac{Cov(\tilde{y}...
user9875321__'s user avatar
1 vote
0 answers
75 views

Taylor approximation for function of a random variable [closed]

There is a function $f$ whose domain is the space of CDFs on $\mathbb{R}_+$ and whose range is $[0,1]$, e.g. $f$ maps a CDF on to a real number. Further, $f$ is continuous, increasing with respect to ...
user's user avatar
  • 111
0 votes
1 answer
37 views

Please can someone explain the notation of this multivariate Taylor expansion?

Kamanzi-wa-Binyavanga, 2009, wrote the following paper, Calculating Cumulants of a Taylor Expansion of a Multivariate Function: What I am confused about, is how precise the notation. I understand ...
Nick Green's user avatar
4 votes
1 answer
87 views

Formulas or approximations for $\mathbb{E}\left( \frac{X}{\|X\|} \right)$, $X\sim N(\mu, Id)$?

This is a cross-posting of this math SE question. I want to compute or approximate the following expected value with some analytic expression: $\mathbb{E}\left( \frac{X}{||X||} \right)$ , where $X \in ...
dherrera's user avatar
  • 1,268
0 votes
0 answers
28 views

Expectations with respect to affine transformation of a log-normal distribution

Let $X$ be a log-normal distribution and consider $Y=aX+b$ for some $a,b>0$. I would like to know if one can compute $$\mathbb{E}[\log(Y)]$$ This would be very easy if it was $b=0$, since in this ...
Francesco Bilotta's user avatar
6 votes
2 answers
134 views

Combinatorial/probabilistic meaning/analogy for $x^n / n!$

The expression $x^n / n!$ appears in the infinite sum defining $e^x$ and similar terms in the sums defining $\cos(x)$, $\sin(x)$, etc. I would like to know if there is some combinatorial/probabilistic ...
Gilad's user avatar
  • 61
4 votes
2 answers
357 views

Computation of ratio with Dirichlet distribution

I would like to compute ratio of proportions coming from a Dirichlet distribution. My understanding is that each proportion should be treated as a random variable and therefore I should use Taylor ...
Umka's user avatar
  • 51
1 vote
0 answers
71 views

Please help me to understand the Taylor’s theorem when transiting from Gradient Boost to XGboost

I am reading this article, which explains how the algorithm replaces the actual loss function with so-called 2nd order Taylor expansion. I can understand til Step 4, and can't understand step 5. I ...
yts61's user avatar
  • 109
2 votes
0 answers
42 views

Does XGBoost use gradient descent? [duplicate]

XGBoost is said to be based on "gradient tree boosting" in the original paper. Reading the paper and the introduction on the official website, it seems to me that the algorithm does not use ...
bob_cart's user avatar
5 votes
1 answer
529 views

Taylor expansion in Hoeffding's Lemma proof

Hoeffding's Lemma proof uses Taylor expansion with this statement: From Taylor's theorem, for some $ 0\leq \theta \leq 1$ $ L(h) = L(0) + h L'(0) + \frac{1}{2} h^2 L''(h\theta) \leq \frac{1}{8}h^2 $ ...
Tavakoli's user avatar
  • 151
3 votes
1 answer
756 views

Delta method for estimating a ratio involving variance and mean

Let $X$ be a binomial RV with parameters $(n,p)$. I am interested in the ratio given by $\hspace{5cm}\boxed{R=\frac{var[f(X)]}{\mu[f(X)](1-\mu[f(X)])}}$ where $\mu[f(X)]$ denotes the mean of $f(X)$. ...
wanderer's user avatar
  • 214
2 votes
2 answers
146 views

Directly discarding big term in the proof of error propagation formula of variance from random variable $x$ to $f(x)$?

I read the error propagation formula scanario said that, the connection between the variance of a random variable $x$ and $f(x)$ is $\frac{var(f(x))}{|\partial_xf(x)|_{x=\bar{x}}|^2}=var(x)$. While I'...
narip's user avatar
  • 187
4 votes
0 answers
89 views

Expectation of x/√(x²+2px+1) under Normal distribution

I'm need to find (or at least approximate) as a function of $p$, the expectation under $x \sim Normal(0,1)$ of: $$f(x) = \frac{x}{\sqrt{x^2+2px+1}},\hspace{1em}\textrm{where}-1<p<1$$ Wolfram ...
Luke's user avatar
  • 41
3 votes
2 answers
2k views

2nd order Taylor expansion of KL divergence

I am having trouble understanding page 6 of this PDF: https://people.eecs.berkeley.edu/~pabbeel/cs287-fa09/lecture-notes/lecture20-2pp.pdf This is also a question in cs229-autumn2018 PS#3-3d. In ...
z.x.99's user avatar
  • 93
0 votes
0 answers
73 views

Textbook clarification on Taylor expansion of mgf (Casella, Berger)

In Statistical Inference by Casella, Berger (version 2), Section 2.6.1, we have the following expression for the Taylor expansion of the mgf: $$M_X(t)=\sum_{r=0}^\infty \frac{(-1)^r\mu_r't^r}{r!}$$ ...
Shivashriganesh Mahato's user avatar
1 vote
0 answers
173 views

Calculation of Variance from a 2 order Taylor expansion - Expecting a better estimation than with 1st order Taylor expansion

I tried to compute the variance of a squared ratio of 2 Gaussians random variables (not the same means and standard deviations between both). I generate the samples by Monte-Carlo method. I expect ...
user avatar
0 votes
1 answer
96 views

How to operationalize stratified, cluster survey sample “standard of the mean” in Excel

I have a question on the analysis of complex survey design. Basically, I am trying to replicate the SAS Surveymeans procedure in Excel in order to understand how to operationalize the equations to ...
tas_taba's user avatar
2 votes
0 answers
244 views

Taylor expansion of Gaussian process function with input noise

I am reading "Gaussian Process Training with Input Noise" by Andrew McHutchon and Carl Edward Rasmussen, where it is assumed that the inputs $x$ are noisy measurements of the actual latent ...
bs.jun's user avatar
  • 21
3 votes
1 answer
317 views

Do all moments of a random variable need to be well controlled for a valid 2nd order Taylor approximation, or is the third moment sufficient?

In this post, the accepted answer states that we need certain conditions before a second order Taylor series approximation is robust, due to the fact that the variance does not control higher moments. ...
sonicboom's user avatar
  • 940
1 vote
1 answer
334 views

taylor series expansion in laplace approximation of bayesian neural network prediction

In chapter 6 of the book Pattern recognition and machine learning, there is this part about prediction in Bayesian neural network using laplace approximation : why assuming small variance compared ...
yj_billie's user avatar
5 votes
1 answer
100 views

Does an explicit expression exist for the moments of the residuals in least squares regression?

Consider the linear regression model is $$ Y_i = \beta_0 + \beta_1 X_i + \varepsilon_i, $$ where $X$ is a random variable and the error has finite variance $\sigma^2$. When we solve this with least ...
Bertus101's user avatar
  • 785
3 votes
1 answer
45 views

Help me understand how the following likelihood function is derived

A week ago, I asked a question concerning the Taylor expansion of an arbitrary distribution function. As noted by a member of the forum, the question was vague and perhaps incorrect. I had asked this ...
Carl's user avatar
  • 1,216
9 votes
2 answers
300 views

Trying to approximate $E[f(X)]$ - Woflram Alpha gives $E[f(X)] \approx \frac{1}{\sqrt{3}}$ but I get $E[f(X)] \approx 0$?

Let $X \sim \mathcal{N}(\mu_X,\sigma_X^2) = \mathcal{N}(0,1)$. Let $f(x) = e^{-x^2}$. I want to approximate $E[f(X)]$. Wolfram Alpha gives \begin{align} E[f(X)] \approx \frac{1}{\sqrt{3}}. \end{align} ...
Bertus101's user avatar
  • 785
-1 votes
1 answer
38 views

Finding C for a PMF of a frequency distribution

N has probability mass function: $p_o = p_1 =0$ and $p_k = c/k!$ for $k=2,3,4,...$ I used exp series $\sum_{n=1}^{\infty} \frac{x^k}{k!} = e^x$ to get $ c\sum_{n=1}^{\infty} \frac{1}{k!}$ then $ce=1$ ...
confusedmathstudent's user avatar
0 votes
0 answers
365 views

Question about delta method and variance-stabilization

The delta method or variance-stabilizing transformation can be applied to make the variance be "nearly constant" (https://en.wikipedia.org/wiki/Variance-stabilizing_transformation). They use ...
Gregory's user avatar
  • 141
2 votes
0 answers
267 views

how Taylor series expansion is applied on the pixel array in python?

I wanted to compute Taylor series expansion of each pixel values from the pixel array. I am wondering how Taylor expansion is going to approximate each pixel values with certain approximation order. ...
Jerry07's user avatar
  • 121
4 votes
1 answer
1k views

How does an influence function-based estimator estimate a target functional for an unknown distribution?

How exactly does a "1-step" influence function-based estimator estimate a target functional (like average treatment effect) for an unknown distribution? As described in Aaron Fisher and Edward H. ...
RobertF's user avatar
  • 6,124
0 votes
0 answers
114 views

How do you approximate the variance for arcsin of a proportion?

I want to calculate the variance of $$\sqrt{n}\arcsin \sqrt{P}$$ and I believe I'm supposed to use the Taylor approximation where $$\sqrt{n}\arcsin \sqrt{P} = Z,$$ where $$Z = g(P).$$ I'm a bit ...
Sophia's user avatar
  • 21
5 votes
1 answer
2k views

What is the general second-order Taylor approximation to $\mathbb{V}(f(X))$?

If $X \sim \text{N}(0, \sigma^2)$ it is well-known that we have the second-order Taylor approximation: $$\mathbb{V}[f(X)] \approx f'(\mu)^2 \cdot \sigma^2 + \frac{f''(\mu)^2}{2} \cdot \sigma^4.$$ ...
Ben's user avatar
  • 127k
4 votes
1 answer
503 views

Probability distribution function expressed in terms of a divergent series

I'm interested in finding the CDF and PDF of $U_i$ defined as follows, $$U_i=\frac g{d^{\alpha}}$$ where $g$ is a gamma distributed random variable with shape $k$ and scale $\theta$, and $d$ is a ...
William's user avatar
  • 117
0 votes
1 answer
31 views

Taylor explansion of the feed-forward network error

The Optimal Brain Damage paper proposes to approximate the error objective function by a Taylor series. Is E a generic error function? And how is the formula (1) obtained? The paper: http://...
cerebrou's user avatar
  • 275
0 votes
0 answers
448 views

Quadratic Approximation of the binary logistic regression

I am using https://web.stanford.edu/~hastie/Papers/glmnet.pdf package to solve my optimization problem for the Binary Logistic Regression. On page 10 it is stated that the quadratic approximation of ...
Annalise Azzopardi's user avatar
1 vote
1 answer
1k views

Multivariate Taylor series for moments of a random variable

In the expectation propagation for the generative aspect model, Minka uses Taylor series for the parameter estimation of the topics $p(w\mid a)$ eq 31. I am a little confused in the last equation. He ...
c.uent's user avatar
  • 115
1 vote
0 answers
63 views

Taylor Series Power Function in R

I'm trying to recreate a compartmental model from Yu et al (2017). In this model there is a expanded taylor series power function that is used that I am not understanding how to code. where W(t) = 0, ...
Ash's user avatar
  • 111
1 vote
0 answers
75 views

The question of Taylor expansion of loss function in XGBoost [duplicate]

I am learning XGBoost from documentation, but there are a few questions in the derivation of it. In the part of ...
Bowen Peng's user avatar
1 vote
1 answer
889 views

xgboost tayler expansion detail [duplicate]

This is the objective function for Xgboost. I have no idea where $g_{i}$ and $h_{i}$ came from is some one explain how this two terms came form? or direct me to the related tutorial page then I ...
slowmonk's user avatar
  • 129
2 votes
1 answer
302 views

How to use derivatives of a function to better estimate its variance over the domain?

How to use derivatives of a function to better estimate its variance over the domain? I have a scalar smooth function $f(x)$ and a multivariate random variable $x$ with known distribution (e.g. ...
MInner's user avatar
  • 283
1 vote
1 answer
811 views

Iterated estimation of Taylor series

Say your data generating process is given by the function $y=f(x|\theta)$, where $y$ and $x$ represent variables (data) and $\theta$ represent parameter(s). For convergence reasons (e.g. $f(\cdot)$ is ...
luchonacho's user avatar
  • 2,750
3 votes
1 answer
2k views

Taylor expansion for random variables

Let $X_n$ and $Y_n$ be random variables such that $X_n-Y_n\overset{p}{\longrightarrow}0$ as $n\rightarrow\infty$. Let $f(.)$ be a differentiable function. Is the following correct? $$f(X_n) = f(Y_n) + ...
Hercules's user avatar
4 votes
0 answers
234 views

Taylor Series Expansion of Unconditional Expectation

We know that the best 1st order approximation of an unconditional expectation is the following- $$E(y|x)=(E(y)-\beta E(x))+\beta x$$ where $\beta=\frac{\operatorname{Cov}(y,x)}{\operatorname{Var}(x)}...
Amrit Prasad's user avatar
2 votes
0 answers
82 views

Is there any sort of quadratic SVD for dimensionality reduction?

X-Posted on math.stackexchange, apologies, though I thought this was equally relevant to both communities. I'm wondering if there exists any higher-order SVD for dimensionality reduction. Note that ...
user650261's user avatar
0 votes
0 answers
52 views

Exponential Kth Moment Derivation

I have essentially a mathematical question, relating to deriving the formula for the kth moment of an exponential. I can't seem to work out how we get from the 2nd line to the 3rd line; i.e. the ...
cdDC's user avatar
  • 11
3 votes
2 answers
774 views

Can we use backpropagation to fit other models?

It appears that backpropagation is exclusively used to train neural network models. Why not use it to fit other models. For example - Taylor polynomials: $$ f(x) = c_0+c_1(x-a)+c_2(x-a)^2...+c_n(x-a)...
Edv Beq's user avatar
  • 713
2 votes
0 answers
750 views

Understanding a Taylor expansion for the bias of local polynomial regression

I'm trying to understand the proof of an expression for the asymptotic bias in local polynomial regression of degree $p\ge0$. Specifically, I'm distraught with equation $(3.59)$ on page 102 of this ...
Epiousios's user avatar
  • 238
0 votes
1 answer
70 views

Priors on Taylor Expansion series

I'm wondering what priors can i choose for a Taylor series as follows: $\theta_{1}+\theta_{2} (y-\alpha) + \theta_{3} (y-\alpha)^2$ What priors should I use for updating these parameters ($\theta_{1},...
Afshin's user avatar
  • 11
5 votes
1 answer
6k views

Proof that ML Estimator is asymptotically Normal

I'm trying to prove that the Maximum Likelihood Estimator is Asymptotically Normal distributed. I'm stuck in the lasts steps. Here's what I've done: I do the Taylor's expansion of, that's the mean of ...
Mario Migliaccio's user avatar
1 vote
1 answer
84 views

Jaynes Probability theory 4.70 (Different answers with Jaynes when using Taylor power series.)

I have read this derivation. $$L(f)\equiv{lng(f|DX)}=nln(f)+(N-n)ln(1-f)+const \;(4.69) $$ expand L(f) in a power series about $\hat{f}$.The first terms as $$L(f) = L(\hat{f}) - \frac{(f-\hat{f})^2}{2\...
hello_god's user avatar
  • 141
0 votes
0 answers
562 views

Taylor expansion in xgboost [duplicate]

I'm reading through the math of xgboost: https://xgboost.readthedocs.io/en/latest/model.html Under the ADDITIVE TRAINING section of the objective function, I saw that in the derivation of the ...
clueless_undergrad37's user avatar