Questions tagged [taylor-series]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
0answers
31 views

Quadratic Approximation of the binary logistic regression

I am using https://web.stanford.edu/~hastie/Papers/glmnet.pdf package to solve my optimization problem for the Binary Logistic Regression. On page 10 it is stated that the quadratic approximation of ...
1
vote
0answers
61 views

Multivariate Taylor series for moments of a random variable

In the expectation propagation for the generative aspect model, Minka uses Taylor series for the parameter estimation of the topics $p(w\mid a)$ eq 31. I am a little confused in the last equation. He ...
1
vote
0answers
14 views

Taylor Series Power Function in R

I'm trying to recreate a compartmental model from Yu et al (2017). In this model there is a expanded taylor series power function that is used that I am not understanding how to code. where W(t) = 0, ...
1
vote
0answers
56 views

The question of Taylor expansion of loss function in XGBoost [duplicate]

I am learning XGBoost from documentation, but there are a few questions in the derivation of it. In the part of ...
1
vote
1answer
115 views

xgboost tayler expansion detail [duplicate]

This is the objective function for Xgboost. I have no idea where $g_{i}$ and $h_{i}$ came from is some one explain how this two terms came form? or direct me to the related tutorial page then I ...
0
votes
0answers
15 views

Derive taylor series expansion of df

I was trying to understand ito's lemma. When I came across the taylor series expansion of df(x). df(x) = f'(x) dx + (1/2!) f''(x) (dx)^2 + ... I searched everywhere for the derivation of this but ...
0
votes
1answer
27 views

How to use derivatives of a function to better estimate its variance over the domain?

How to use derivatives of a function to better estimate its variance over the domain? I have a scalar smooth function $f(x)$ and a multivariate random variable $x$ with known distribution (e.g. ...
1
vote
1answer
78 views

Iterated estimation of Taylor series

Say your data generating process is given by the function $y=f(x|\theta)$, where $y$ and $x$ represent variables (data) and $\theta$ represent parameter(s). For convergence reasons (e.g. $f(\cdot)$ is ...
2
votes
1answer
64 views

Taylor expansion for random variables

Let $X_n$ and $Y_n$ be random variables such that $X_n-Y_n\overset{p}{\longrightarrow}0$ as $n\rightarrow\infty$. Let $f(.)$ is a differentiable function. Is the following correct? $f(X_n) = f(Y_n) + ...
4
votes
0answers
119 views

Taylor Series Expansion of Unconditional Expectation

We know that the best 1st order approximation of an unconditional expectation is the following- $$E(y|x)=(E(y)-\beta E(x))+\beta x$$ where $\beta=\frac{\operatorname{Cov}(y,x)}{\operatorname{Var}(x)}...
2
votes
0answers
56 views

Is there any sort of higher-order SVD (quadratic and above) for dimensionality reduction?

X-Posted on math.stackexchange, apologies, though I thought this was equally relevant to both communities. I'm wondering if there exists any higher-order SVD for dimensionality reduction. Note that ...
0
votes
0answers
21 views

Exponential Kth Moment Derivation

I have essentially a mathematical question, relating to deriving the formula for the kth moment of an exponential. I can't seem to work out how we get from the 2nd line to the 3rd line; i.e. the ...
2
votes
2answers
463 views

Can we use backpropagation to fit other models?

It appears that backpropagation is exclusively used to train neural network models. Why not use it to fit other models. For example - Taylor polynomials: $$ f(x) = c_0+c_1(x-a)+c_2(x-a)^2...+c_n(x-a)...
1
vote
0answers
173 views

Understanding a Taylor expansion for the bias of local polynomial regression

I'm trying to understand the proof of an expression for the asymptotic bias in local polynomial regression of degree $p\ge0$. Specifically, I'm distraught with equation $(3.59)$ on page 102 of this ...
0
votes
1answer
53 views

Priors on Taylor Expansion series

I'm wondering what priors can i choose for a Taylor series as follows: $\theta_{1}+\theta_{2} (y-\alpha) + \theta_{3} (y-\alpha)^2$ What priors should I use for updating these parameters ($\theta_{1},...
3
votes
1answer
1k views

Proof that ML Estimator is asymptotically Normal

I'm trying to proof that the Maximum Likelihood Estimator is Asymptotically Normal distribuited. I'm stuck in the lasts steps. Here's what I've done: I do the Taylor's expansion of, that's the mean ...
1
vote
1answer
76 views

Jaynes Probability theory 4.70 (Different answers with Jaynes when using Taylor power series.)

I have read this derivation. $$L(f)\equiv{lng(f|DX)}=nln(f)+(N-n)ln(1-f)+const \;(4.69) $$ expand L(f) in a power series about $\hat{f}$.The first terms as $$L(f) = L(\hat{f}) - \frac{(f-\hat{f})^2}{2\...
0
votes
0answers
344 views

Taylor expansion in xgboost [duplicate]

I'm reading through the math of xgboost: https://xgboost.readthedocs.io/en/latest/model.html Under the ADDITIVE TRAINING section of the objective function, I saw that in the derivation of the ...
0
votes
0answers
104 views

Higher order delta / taylor series approximation relationship to normal distribution?

For a normally distributed variable X, one can call on the delta method to provide an asymptotically normally distributed variable for a non-linear function of it, g(X). This is based on a linear ...
0
votes
1answer
54 views

Estimating polymomial coefficient in R

I used a Taylor series to expand log(1 - ax) so I could estimate the value of parameter 'a'. The expansion becomes -ax - a^2*x^2/2 - a^3*x^3/3 . . . Now I need to estimate the parameter 'a' using ...
1
vote
1answer
412 views

Expansion of Cumulant Generating Function of Negbin

Let $X \thicksim Negbin(r,p)$ where $(0\lt p \lt 1) $ I want to derive skewness and kurtosis of $X$ by getting the Cgf of X. First, since Followance of Negative Binomial equals to the distribution ...
1
vote
1answer
180 views

What is the intuition behind neural networks?

Everywhere in the theory of neural networks, authors saying that idea came about by observing the work of the human brain. But I can not believe in that. I guess, everything is much simpler and neural ...
1
vote
0answers
88 views

Estimate mean and variance of pdf from truncated taylor expansion of logarithm of pdf

In a maximum likelihood fit, one estimates the parameter with the mode of the likelihood $L$, and the variance of this estimator with the second derivative of $\log(L)$: $$ \bar\theta = \mathrm{Mode}[...
3
votes
1answer
225 views

Characteristic function of distribution

$$p(x)=e^{-2 |x|}$$ with x in [-inf, +inf]. I've calculated the characteristic function as $E[e^{ikx}]=\frac{1}{ik+2}-\frac{1}{ik-2}=\frac{4}{k^2+4}$. Now i'd like the moments.. so I suppose I should ...
1
vote
0answers
594 views

Manually compute linearized standard errors of mean in survey

I am would like to manually compute the Taylor-linearized standard error and 95% confidence interval for the mean of a variable x in a survey with 1-stage primary ...
1
vote
0answers
184 views

Delta method with mix of continuous and discrete variables

This is my first question on Cross Validated so please bear with me if my question is lagging in any dimension. My question regards how to evaluate a Jacobian matrix when one variable is binary. I ...
5
votes
1answer
2k views

Are Maximum Likelihood Estimators asymptotically unbiased?

I can follow the proofs in which the asymptotic normal-distribution of a maximum likelihood estimator $\tilde{\theta}_n$ is derived. however, does this already imply that the maximum likelihood ...
5
votes
1answer
466 views

Alternative Confidence interval for Odds Ratio $\hat{p}\over{1-\hat{p}}$ from Logistic Regression?

I have a question about confidence interval calculations for the odds ratio $\hat{p}\over{1-\hat{p}}$ from a logistic regression model (perhaps obtained from the method of Generalized Estimating ...
1
vote
1answer
40 views

Characteristic function issue

As mentioned in a previous post, I've been trying to work through ALL of the problems in Jacod and Protter's Probability Essentials. The following problem has been giving me issues: Let $Z \sim N(0,...
28
votes
1answer
9k views

XGBoost Loss function Approximation With Taylor Expansion

As an example, take the objective function of the XGBoost model on the $t$'th iteration: $$\mathcal{L}^{(t)}=\sum_{i=1}^n\ell(y_i,\hat{y}_i^{(t-1)}+f_t(\mathbf{x}_i))+\Omega(f_t)$$ where $\ell$ is ...
0
votes
0answers
213 views

mixture of Gaussians vs mixture of quadratic denominators (Cauchy)

It is known that mixture of Gaussians are dense in the set of all distribution functions. A 1-dimensional Gaussian has the following density: $$ \frac{1}{\sqrt{2\pi \sigma^2}} e^{-\frac{(\omega-\beta)^...
0
votes
0answers
401 views

Taylor Expansion of Power of Cumulative Log Normal Distribution Function - Show Lagrange Remainder tends to Zero

QUESTION I am looking to find a simplification of the expression below. I have attempted this using the Taylor series. The question then remains if we can show the Lagrange remainder goes to zero. I ...
11
votes
1answer
185 views

Approximating $\log( E(X))$

I was casually reading an article (in economics) which had the following approximation for $\log(E(X))$: $\log(E(X)) \approx E(\log(X))+0.5 \mathrm{var}(\log(X))$, which the author says is exact if ...
2
votes
3answers
853 views

Best basis set for polynomial expansion

I want to do a regression of x onto y: $$f(y)=c_{1}x+c_{2}x^{2}+c_{3}x^{3}\cdots$$ Obviously a plain Taylor expansion as above is suboptimal since the coefficients will not be orthogonal/...
6
votes
0answers
927 views

Why is Fisher Scoring easier to compute?

In practice, the observed information matrix (Newton-Raphson) is usually replaced by its expectation, known as Fisher scoring. Link: https://en.wikipedia.org/wiki/Scoring_algorithm#Fisher_scoring ...
7
votes
2answers
1k views

Help with Taylor expansion of log likelihood function

I've reed the following part of a sketch of the proof that the maximum likelihood estimator is asymptotically normal: "Sketch of the second part of the proof. Recall that we may write the likelihood ...
0
votes
0answers
41 views

Express $E(x^{\alpha})$ in terms of $E(e^{-\zeta x})$? to a 1st or second order?

I have a random variable, $X$, and am able to find $\mathbf{E}(e^{-\zeta X})$ for many $\zeta$ (through the Laplace transform solving an ODE as this actually evolves over time) Is there any way I can ...
2
votes
1answer
271 views

Asking help with Taylor approximation of expectation of ratio

I am trying to understand how I should approach the problem of a Taylor approximation to the expectation of the ratio of two random variables. In my particular problem I am concerned with the ...
1
vote
1answer
120 views

Proof of a step of a lemma on the asymptotics of maximum likelihood where a Taylor expansion is used

I am trying to understand a proof of quite a long theorem that I report completely for the sake of completeness. This is From Jensen and Rahbek Asymptotic Inference for Nonstationary GARCH (2004). My ...
2
votes
0answers
79 views

linearization of an estiamtor

Suppose we have two variables $x$ and $y$ defined in some population, with all values of $x$ known. A Poisson sample is drawn, with corresponding inclusion probabilities $\pi_k$ that are proportional ...
2
votes
0answers
49 views

How can you cluster a set of functions with unknown functional forms?

Say you've $N$ functions $f_N(x)$ defined on a regular grid $x$. You don't know the form of $f(x)$, you've only got several realizations of it. The different functions are related to each other ...
7
votes
1answer
352 views

Given loads of data, can we always model it with polynomials?

Given Taylor series and enough data so as to not risk over-fitting, do you actually need to think about if your phenomenon is following an exponential, quadratic, logarithmic, ..., behaviour? I'm sure ...
3
votes
0answers
425 views

Is there a nice(r) Taylor expansion of the normal quantile function?

Letting $\Phi$ be the CDF of the standard normal, and $\Phi^{-1}$ be the quantile function of the normal, I am looking for the Taylor series expansion of $$ \Phi^{-1}\left(\Phi\left(x\right) + \...