Questions tagged [taylor-series]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
-1
votes
1answer
26 views

Finding C for a PMF of a frequency distribution

N has probability mass function: $p_o = p_1 =0$ and $p_k = c/k!$ for $k=2,3,4,...$ I used exp series $\sum_{n=1}^{\infty} \frac{x^k}{k!} = e^x$ to get $ c\sum_{n=1}^{\infty} \frac{1}{k!}$ then $ce=1$ ...
0
votes
0answers
34 views

Question about delta method and variance-stabilization

The delta method or variance-stabilizing transformation can be applied to make the variance be "nearly constant" (https://en.wikipedia.org/wiki/Variance-stabilizing_transformation). They use ...
1
vote
0answers
20 views

how Taylor series expansion is applied on the pixel array in python?

I wanted to compute Taylor series expansion of each pixel values from the pixel array. I am wondering how Taylor expansion is going to approximate each pixel values with certain approximation order. ...
0
votes
0answers
20 views

Analytical Approximation for Conditional Moments

Say I have a function of a latent variable: $F(X_{t+1})$. $F(X_{t+1})=-log(\sum\limits_{\substack{k \neq j}}\alpha^{k}_{j}\frac{S^{k}_{t+1}}{S^{j}_{t+1}})$ The term in brackets is $X_{t+1}$. I know ...
2
votes
1answer
81 views

How does an influence function-based estimator estimate a target functional for an unknown distribution?

How exactly does a "1-step" influence function-based estimator estimate a target functional (like average treatment effect) for an unknown distribution? As described in Aaron Fisher and Edward H. ...
0
votes
0answers
27 views

Error Propagation for Unbiased Means

I am reading through https://arxiv.org/pdf/1210.3781.pdf, and do not understand its derivation for propagation of errors with respect to means. According to the text, when trying to estimate a ...
0
votes
0answers
17 views

How do you approximate the variance for arcsin of a proportion?

I want to calculate the variance of $$\sqrt{n}\arcsin \sqrt{P}$$ and I believe I'm supposed to use the Taylor approximation where $$\sqrt{n}\arcsin \sqrt{P} = Z,$$ where $$Z = g(P).$$ I'm a bit ...
4
votes
1answer
62 views

What is the general second-order Taylor approximation to $\mathbb{V}(f(X))$?

If $X \sim \text{N}(0, \sigma^2)$ it is well-known that we have the second-order Taylor approximation: $$\mathbb{V}[f(X)] \approx f'(\mu)^2 \cdot \sigma^2 + \frac{f''(\mu)^2}{2} \cdot \sigma^4.$$ ...
0
votes
0answers
42 views

Taylor expansion of a conditional expectation of a function of a random variable

I saw this post on hazard function and conditional pdf, Why is the Hazard function not a pdf?, and the main outcome there is that the argument of a conditional pdf cannot depend on the conditioning ...
4
votes
1answer
201 views

Probability distribution function expressed in terms of a divergent series

I'm interested in finding the CDF and PDF of $U_i$ defined as follows, $$U_i=\frac g{d^{\alpha}}$$ where $g$ is a gamma distributed random variable with shape $k$ and scale $\theta$, and $d$ is a ...
0
votes
0answers
35 views

Derivation of variance of Odds Ratio in a 2x2 table

Assuming that the elements of a 2x2 table (a,b,c,d) are binomial in distribution with a being $Bin(n_1,p_1)$ and c being $Bin(n_2,p_2)$. How can we show that $\hat {var(logOR)} = \frac 1 a + \frac 1 b ...
0
votes
1answer
22 views

Taylor explansion of the feed-forward network error

The Optimal Brain Damage paper proposes to approximate the error objective function by a Taylor series. Is E a generic error function? And how is the formula (1) obtained? The paper: http://...
0
votes
0answers
137 views

Quadratic Approximation of the binary logistic regression

I am using https://web.stanford.edu/~hastie/Papers/glmnet.pdf package to solve my optimization problem for the Binary Logistic Regression. On page 10 it is stated that the quadratic approximation of ...
1
vote
1answer
211 views

Multivariate Taylor series for moments of a random variable

In the expectation propagation for the generative aspect model, Minka uses Taylor series for the parameter estimation of the topics $p(w\mid a)$ eq 31. I am a little confused in the last equation. He ...
1
vote
0answers
28 views

Taylor Series Power Function in R

I'm trying to recreate a compartmental model from Yu et al (2017). In this model there is a expanded taylor series power function that is used that I am not understanding how to code. where W(t) = 0, ...
1
vote
0answers
65 views

The question of Taylor expansion of loss function in XGBoost [duplicate]

I am learning XGBoost from documentation, but there are a few questions in the derivation of it. In the part of ...
1
vote
1answer
300 views

xgboost tayler expansion detail [duplicate]

This is the objective function for Xgboost. I have no idea where $g_{i}$ and $h_{i}$ came from is some one explain how this two terms came form? or direct me to the related tutorial page then I ...
0
votes
1answer
74 views

How to use derivatives of a function to better estimate its variance over the domain?

How to use derivatives of a function to better estimate its variance over the domain? I have a scalar smooth function $f(x)$ and a multivariate random variable $x$ with known distribution (e.g. ...
1
vote
1answer
156 views

Iterated estimation of Taylor series

Say your data generating process is given by the function $y=f(x|\theta)$, where $y$ and $x$ represent variables (data) and $\theta$ represent parameter(s). For convergence reasons (e.g. $f(\cdot)$ is ...
2
votes
1answer
463 views

Taylor expansion for random variables

Let $X_n$ and $Y_n$ be random variables such that $X_n-Y_n\overset{p}{\longrightarrow}0$ as $n\rightarrow\infty$. Let $f(.)$ is a differentiable function. Is the following correct? $f(X_n) = f(Y_n) + ...
4
votes
0answers
160 views

Taylor Series Expansion of Unconditional Expectation

We know that the best 1st order approximation of an unconditional expectation is the following- $$E(y|x)=(E(y)-\beta E(x))+\beta x$$ where $\beta=\frac{\operatorname{Cov}(y,x)}{\operatorname{Var}(x)}...
2
votes
0answers
62 views

Is there any sort of higher-order SVD (quadratic and above) for dimensionality reduction?

X-Posted on math.stackexchange, apologies, though I thought this was equally relevant to both communities. I'm wondering if there exists any higher-order SVD for dimensionality reduction. Note that ...
0
votes
0answers
22 views

Exponential Kth Moment Derivation

I have essentially a mathematical question, relating to deriving the formula for the kth moment of an exponential. I can't seem to work out how we get from the 2nd line to the 3rd line; i.e. the ...
2
votes
2answers
518 views

Can we use backpropagation to fit other models?

It appears that backpropagation is exclusively used to train neural network models. Why not use it to fit other models. For example - Taylor polynomials: $$ f(x) = c_0+c_1(x-a)+c_2(x-a)^2...+c_n(x-a)...
1
vote
0answers
269 views

Understanding a Taylor expansion for the bias of local polynomial regression

I'm trying to understand the proof of an expression for the asymptotic bias in local polynomial regression of degree $p\ge0$. Specifically, I'm distraught with equation $(3.59)$ on page 102 of this ...
0
votes
1answer
55 views

Priors on Taylor Expansion series

I'm wondering what priors can i choose for a Taylor series as follows: $\theta_{1}+\theta_{2} (y-\alpha) + \theta_{3} (y-\alpha)^2$ What priors should I use for updating these parameters ($\theta_{1},...
3
votes
1answer
2k views

Proof that ML Estimator is asymptotically Normal

I'm trying to proof that the Maximum Likelihood Estimator is Asymptotically Normal distribuited. I'm stuck in the lasts steps. Here's what I've done: I do the Taylor's expansion of, that's the mean ...
1
vote
1answer
81 views

Jaynes Probability theory 4.70 (Different answers with Jaynes when using Taylor power series.)

I have read this derivation. $$L(f)\equiv{lng(f|DX)}=nln(f)+(N-n)ln(1-f)+const \;(4.69) $$ expand L(f) in a power series about $\hat{f}$.The first terms as $$L(f) = L(\hat{f}) - \frac{(f-\hat{f})^2}{2\...
0
votes
0answers
388 views

Taylor expansion in xgboost [duplicate]

I'm reading through the math of xgboost: https://xgboost.readthedocs.io/en/latest/model.html Under the ADDITIVE TRAINING section of the objective function, I saw that in the derivation of the ...
0
votes
0answers
127 views

Higher order delta / taylor series approximation relationship to normal distribution?

For a normally distributed variable X, one can call on the delta method to provide an asymptotically normally distributed variable for a non-linear function of it, g(X). This is based on a linear ...
0
votes
1answer
57 views

Estimating polymomial coefficient in R

I used a Taylor series to expand log(1 - ax) so I could estimate the value of parameter 'a'. The expansion becomes -ax - a^2*x^2/2 - a^3*x^3/3 . . . Now I need to estimate the parameter 'a' using ...
1
vote
1answer
513 views

Expansion of Cumulant Generating Function of Negbin

Let $X \thicksim Negbin(r,p)$ where $(0\lt p \lt 1) $ I want to derive skewness and kurtosis of $X$ by getting the Cgf of X. First, since Followance of Negative Binomial equals to the distribution ...
2
votes
1answer
202 views

What is the intuition behind neural networks?

Everywhere in the theory of neural networks, authors saying that idea came about by observing the work of the human brain. But I can not believe in that. I guess, everything is much simpler and neural ...
1
vote
0answers
102 views

Estimate mean and variance of pdf from truncated taylor expansion of logarithm of pdf

In a maximum likelihood fit, one estimates the parameter with the mode of the likelihood $L$, and the variance of this estimator with the second derivative of $\log(L)$: $$ \bar\theta = \mathrm{Mode}[...
3
votes
1answer
234 views

Characteristic function of distribution

$$p(x)=e^{-2 |x|}$$ with x in [-inf, +inf]. I've calculated the characteristic function as $E[e^{ikx}]=\frac{1}{ik+2}-\frac{1}{ik-2}=\frac{4}{k^2+4}$. Now i'd like the moments.. so I suppose I should ...
1
vote
0answers
737 views

Manually compute linearized standard errors of mean in survey

I am would like to manually compute the Taylor-linearized standard error and 95% confidence interval for the mean of a variable x in a survey with 1-stage primary ...
1
vote
0answers
231 views

Delta method with mix of continuous and discrete variables

This is my first question on Cross Validated so please bear with me if my question is lagging in any dimension. My question regards how to evaluate a Jacobian matrix when one variable is binary. I ...
6
votes
1answer
3k views

Are Maximum Likelihood Estimators asymptotically unbiased?

I can follow the proofs in which the asymptotic normal-distribution of a maximum likelihood estimator $\tilde{\theta}_n$ is derived. however, does this already imply that the maximum likelihood ...
5
votes
1answer
527 views

Alternative Confidence interval for Odds Ratio $\hat{p}\over{1-\hat{p}}$ from Logistic Regression?

I have a question about confidence interval calculations for the odds ratio $\hat{p}\over{1-\hat{p}}$ from a logistic regression model (perhaps obtained from the method of Generalized Estimating ...
1
vote
1answer
49 views

Characteristic function issue

As mentioned in a previous post, I've been trying to work through ALL of the problems in Jacod and Protter's Probability Essentials. The following problem has been giving me issues: Let $Z \sim N(0,...
32
votes
1answer
11k views

XGBoost Loss function Approximation With Taylor Expansion

As an example, take the objective function of the XGBoost model on the $t$'th iteration: $$\mathcal{L}^{(t)}=\sum_{i=1}^n\ell(y_i,\hat{y}_i^{(t-1)}+f_t(\mathbf{x}_i))+\Omega(f_t)$$ where $\ell$ is ...
1
vote
0answers
222 views

mixture of Gaussians vs mixture of quadratic denominators (Cauchy)

It is known that mixture of Gaussians are dense in the set of all distribution functions. A 1-dimensional Gaussian has the following density: $$ \frac{1}{\sqrt{2\pi \sigma^2}} e^{-\frac{(\omega-\beta)^...
0
votes
0answers
483 views

Taylor Expansion of Power of Cumulative Log Normal Distribution Function - Show Lagrange Remainder tends to Zero

QUESTION I am looking to find a simplification of the expression below. I have attempted this using the Taylor series. The question then remains if we can show the Lagrange remainder goes to zero. I ...
11
votes
1answer
208 views

Approximating $\log( E(X))$

I was casually reading an article (in economics) which had the following approximation for $\log(E(X))$: $\log(E(X)) \approx E(\log(X))+0.5 \mathrm{var}(\log(X))$, which the author says is exact if ...
5
votes
3answers
1k views

Best basis set for polynomial expansion

I want to do a regression of x onto y: $$f(y)=c_{1}x+c_{2}x^{2}+c_{3}x^{3}\cdots$$ Obviously a plain Taylor expansion as above is suboptimal since the coefficients will not be orthogonal/...
6
votes
0answers
1k views

Why is Fisher Scoring easier to compute?

In practice, the observed information matrix (Newton-Raphson) is usually replaced by its expectation, known as Fisher scoring. Link: https://en.wikipedia.org/wiki/Scoring_algorithm#Fisher_scoring ...
7
votes
2answers
2k views

Help with Taylor expansion of log likelihood function

I've reed the following part of a sketch of the proof that the maximum likelihood estimator is asymptotically normal: "Sketch of the second part of the proof. Recall that we may write the likelihood ...
0
votes
0answers
44 views

Express $E(x^{\alpha})$ in terms of $E(e^{-\zeta x})$? to a 1st or second order?

I have a random variable, $X$, and am able to find $\mathbf{E}(e^{-\zeta X})$ for many $\zeta$ (through the Laplace transform solving an ODE as this actually evolves over time) Is there any way I can ...
2
votes
1answer
317 views

Asking help with Taylor approximation of expectation of ratio

I am trying to understand how I should approach the problem of a Taylor approximation to the expectation of the ratio of two random variables. In my particular problem I am concerned with the ...
1
vote
1answer
136 views

Proof of a step of a lemma on the asymptotics of maximum likelihood where a Taylor expansion is used

I am trying to understand a proof of quite a long theorem that I report completely for the sake of completeness. This is From Jensen and Rahbek Asymptotic Inference for Nonstationary GARCH (2004). My ...