Questions tagged [taylor-series]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
0answers
23 views

Quadratic Approximation of the binary logistic regression

I am using https://web.stanford.edu/~hastie/Papers/glmnet.pdf package to solve my optimization problem for the Binary Logistic Regression. On page 10 it is stated that the quadratic approximation of ...
2
votes
1answer
55 views

Moment Generating Function of Beta ( Taylor series)

Suppose X is a random variable with a Beta ( a =$\frac{1}{2}$ , b=1) distribution and x in (0,1) Then the moment generating function is calculated as below $ M_X(t) $ = $\mathbb{E}[e^{tX}]$ =$ \...
1
vote
0answers
40 views

Multivariate Taylor series for moments of a random variable

In the expectation propagation for the generative aspect model, Minka uses Taylor series for the parameter estimation of the topics $p(w\mid a)$ eq 31. I am a little confused in the last equation. He ...
1
vote
0answers
12 views

Taylor Series Power Function in R

I'm trying to recreate a compartmental model from Yu et al (2017). In this model there is a expanded taylor series power function that is used that I am not understanding how to code. where W(t) = 0, ...
1
vote
0answers
53 views

The question of Taylor expansion of loss function in XGBoost [duplicate]

I am learning XGBoost from documentation, but there are a few questions in the derivation of it. In the part of ...
1
vote
1answer
83 views

xgboost tayler expansion detail [duplicate]

This is the objective function for Xgboost. I have no idea where $g_{i}$ and $h_{i}$ came from is some one explain how this two terms came form? or direct me to the related tutorial page then I ...
0
votes
0answers
14 views

Derive taylor series expansion of df

I was trying to understand ito's lemma. When I came across the taylor series expansion of df(x). df(x) = f'(x) dx + (1/2!) f''(x) (dx)^2 + ... I searched everywhere for the derivation of this but ...
0
votes
1answer
25 views

How to use derivatives of a function to better estimate its variance over the domain?

How to use derivatives of a function to better estimate its variance over the domain? I have a scalar smooth function $f(x)$ and a multivariate random variable $x$ with known distribution (e.g. ...
0
votes
1answer
76 views

Iterated estimation of Taylor series

Say your data generating process is given by the function $y=f(x|\theta)$, where $y$ and $x$ represent variables (data) and $\theta$ represent parameter(s). For convergence reasons (e.g. $f(\cdot)$ is ...
2
votes
1answer
53 views

Taylor expansion for random variables

Let $X_n$ and $Y_n$ be random variables such that $X_n-Y_n\overset{p}{\longrightarrow}0$ as $n\rightarrow\infty$. Let $f(.)$ is a differentiable function. Is the following correct? $f(X_n) = f(Y_n) + ...
4
votes
0answers
109 views

Taylor Series Expansion of Unconditional Expectation

We know that the best 1st order approximation of an unconditional expectation is the following- $$E(y|x)=(E(y)-\beta E(x))+\beta x$$ where $\beta=\frac{\operatorname{Cov}(y,x)}{\operatorname{Var}(x)}...
2
votes
0answers
50 views

Is there any sort of higher-order SVD (quadratic and above) for dimensionality reduction?

X-Posted on math.stackexchange, apologies, though I thought this was equally relevant to both communities. I'm wondering if there exists any higher-order SVD for dimensionality reduction. Note that ...
0
votes
0answers
21 views

Exponential Kth Moment Derivation

I have essentially a mathematical question, relating to deriving the formula for the kth moment of an exponential. I can't seem to work out how we get from the 2nd line to the 3rd line; i.e. the ...
2
votes
2answers
460 views

Can we use backpropagation to fit other models?

It appears that backpropagation is exclusively used to train neural network models. Why not use it to fit other models. For example - Taylor polynomials: $$ f(x) = c_0+c_1(x-a)+c_2(x-a)^2...+c_n(x-a)...
1
vote
0answers
162 views

Understanding a Taylor expansion for the bias of local polynomial regression

I'm trying to understand the proof of an expression for the asymptotic bias in local polynomial regression of degree $p\ge0$. Specifically, I'm distraught with equation $(3.59)$ on page 102 of this ...
0
votes
1answer
53 views

Priors on Taylor Expansion series

I'm wondering what priors can i choose for a Taylor series as follows: $\theta_{1}+\theta_{2} (y-\alpha) + \theta_{3} (y-\alpha)^2$ What priors should I use for updating these parameters ($\theta_{1},...
3
votes
1answer
981 views

Proof that ML Estimator is asymptotically Normal

I'm trying to proof that the Maximum Likelihood Estimator is Asymptotically Normal distribuited. I'm stuck in the lasts steps. Here's what I've done: I do the Taylor's expansion of, that's the mean ...
1
vote
1answer
70 views

Jaynes Probability theory 4.70 (Different answers with Jaynes when using Taylor power series.)

I have read this derivation. $$L(f)\equiv{lng(f|DX)}=nln(f)+(N-n)ln(1-f)+const \;(4.69) $$ expand L(f) in a power series about $\hat{f}$.The first terms as $$L(f) = L(\hat{f}) - \frac{(f-\hat{f})^2}{2\...
0
votes
0answers
342 views

Taylor expansion in xgboost [duplicate]

I'm reading through the math of xgboost: https://xgboost.readthedocs.io/en/latest/model.html Under the ADDITIVE TRAINING section of the objective function, I saw that in the derivation of the ...
0
votes
0answers
100 views

Higher order delta / taylor series approximation relationship to normal distribution?

For a normally distributed variable X, one can call on the delta method to provide an asymptotically normally distributed variable for a non-linear function of it, g(X). This is based on a linear ...
0
votes
1answer
53 views

Estimating polymomial coefficient in R

I used a Taylor series to expand log(1 - ax) so I could estimate the value of parameter 'a'. The expansion becomes -ax - a^2*x^2/2 - a^3*x^3/3 . . . Now I need to estimate the parameter 'a' using ...
1
vote
1answer
378 views

Expansion of Cumulant Generating Function of Negbin

Let $X \thicksim Negbin(r,p)$ where $(0\lt p \lt 1) $ I want to derive skewness and kurtosis of $X$ by getting the Cgf of X. First, since Followance of Negative Binomial equals to the distribution ...
1
vote
1answer
177 views

What is the intuition behind neural networks?

Everywhere in the theory of neural networks, authors saying that idea came about by observing the work of the human brain. But I can not believe in that. I guess, everything is much simpler and neural ...
1
vote
0answers
85 views

Estimate mean and variance of pdf from truncated taylor expansion of logarithm of pdf

In a maximum likelihood fit, one estimates the parameter with the mode of the likelihood $L$, and the variance of this estimator with the second derivative of $\log(L)$: $$ \bar\theta = \mathrm{Mode}[...
3
votes
1answer
222 views

Characteristic function of distribution

$$p(x)=e^{-2 |x|}$$ with x in [-inf, +inf]. I've calculated the characteristic function as $E[e^{ikx}]=\frac{1}{ik+2}-\frac{1}{ik-2}=\frac{4}{k^2+4}$. Now i'd like the moments.. so I suppose I should ...
1
vote
0answers
567 views

Manually compute linearized standard errors of mean in survey

I am would like to manually compute the Taylor-linearized standard error and 95% confidence interval for the mean of a variable x in a survey with 1-stage primary ...
1
vote
0answers
162 views

Delta method with mix of continuous and discrete variables

This is my first question on Cross Validated so please bear with me if my question is lagging in any dimension. My question regards how to evaluate a Jacobian matrix when one variable is binary. I ...
5
votes
1answer
2k views

Are Maximum Likelihood Estimators asymptotically unbiased?

I can follow the proofs in which the asymptotic normal-distribution of a maximum likelihood estimator $\tilde{\theta}_n$ is derived. however, does this already imply that the maximum likelihood ...
5
votes
1answer
456 views

Alternative Confidence interval for Odds Ratio $\hat{p}\over{1-\hat{p}}$ from Logistic Regression?

I have a question about confidence interval calculations for the odds ratio $\hat{p}\over{1-\hat{p}}$ from a logistic regression model (perhaps obtained from the method of Generalized Estimating ...
1
vote
1answer
40 views

Characteristic function issue

As mentioned in a previous post, I've been trying to work through ALL of the problems in Jacod and Protter's Probability Essentials. The following problem has been giving me issues: Let $Z \sim N(0,...
28
votes
1answer
8k views

XGBoost Loss function Approximation With Taylor Expansion

As an example, take the objective function of the XGBoost model on the $t$'th iteration: $$\mathcal{L}^{(t)}=\sum_{i=1}^n\ell(y_i,\hat{y}_i^{(t-1)}+f_t(\mathbf{x}_i))+\Omega(f_t)$$ where $\ell$ is ...
0
votes
0answers
212 views

mixture of Gaussians vs mixture of quadratic denominators (Cauchy)

It is known that mixture of Gaussians are dense in the set of all distribution functions. A 1-dimensional Gaussian has the following density: $$ \frac{1}{\sqrt{2\pi \sigma^2}} e^{-\frac{(\omega-\beta)^...
0
votes
0answers
380 views

Taylor Expansion of Power of Cumulative Log Normal Distribution Function - Show Lagrange Remainder tends to Zero

QUESTION I am looking to find a simplification of the expression below. I have attempted this using the Taylor series. The question then remains if we can show the Lagrange remainder goes to zero. I ...
11
votes
1answer
181 views

Approximating $\log( E(X))$

I was casually reading an article (in economics) which had the following approximation for $\log(E(X))$: $\log(E(X)) \approx E(\log(X))+0.5 \mathrm{var}(\log(X))$, which the author says is exact if ...
2
votes
3answers
825 views

Best basis set for polynomial expansion

I want to do a regression of x onto y: $$f(y)=c_{1}x+c_{2}x^{2}+c_{3}x^{3}\cdots$$ Obviously a plain Taylor expansion as above is suboptimal since the coefficients will not be orthogonal/...
5
votes
0answers
871 views

Why is Fisher Scoring easier to compute?

In practice, the observed information matrix (Newton-Raphson) is usually replaced by its expectation, known as Fisher scoring. Link: https://en.wikipedia.org/wiki/Scoring_algorithm#Fisher_scoring ...
7
votes
2answers
1k views

Help with Taylor expansion of log likelihood function

I've reed the following part of a sketch of the proof that the maximum likelihood estimator is asymptotically normal: "Sketch of the second part of the proof. Recall that we may write the likelihood ...
0
votes
0answers
40 views

Express $E(x^{\alpha})$ in terms of $E(e^{-\zeta x})$? to a 1st or second order?

I have a random variable, $X$, and am able to find $\mathbf{E}(e^{-\zeta X})$ for many $\zeta$ (through the Laplace transform solving an ODE as this actually evolves over time) Is there any way I can ...
2
votes
1answer
267 views

Asking help with Taylor approximation of expectation of ratio

I am trying to understand how I should approach the problem of a Taylor approximation to the expectation of the ratio of two random variables. In my particular problem I am concerned with the ...
1
vote
1answer
119 views

Proof of a step of a lemma on the asymptotics of maximum likelihood where a Taylor expansion is used

I am trying to understand a proof of quite a long theorem that I report completely for the sake of completeness. This is From Jensen and Rahbek Asymptotic Inference for Nonstationary GARCH (2004). My ...
2
votes
0answers
74 views

linearization of an estiamtor

Suppose we have two variables $x$ and $y$ defined in some population, with all values of $x$ known. A Poisson sample is drawn, with corresponding inclusion probabilities $\pi_k$ that are proportional ...
2
votes
0answers
48 views

How can you cluster a set of functions with unknown functional forms?

Say you've $N$ functions $f_N(x)$ defined on a regular grid $x$. You don't know the form of $f(x)$, you've only got several realizations of it. The different functions are related to each other ...
7
votes
1answer
334 views

Given loads of data, can we always model it with polynomials?

Given Taylor series and enough data so as to not risk over-fitting, do you actually need to think about if your phenomenon is following an exponential, quadratic, logarithmic, ..., behaviour? I'm sure ...
3
votes
0answers
407 views

Is there a nice(r) Taylor expansion of the normal quantile function?

Letting $\Phi$ be the CDF of the standard normal, and $\Phi^{-1}$ be the quantile function of the normal, I am looking for the Taylor series expansion of $$ \Phi^{-1}\left(\Phi\left(x\right) + \...