Questions tagged [taylor-series]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
0answers
11 views

Taylor expansion of a conditional expectation of a function of a random variable

I saw this post on hazard function and conditional pdf, Why is the Hazard function not a pdf?, and the main outcome there is that the argument of a conditional pdf cannot depend on the conditioning ...
0
votes
0answers
28 views

Will the transformations and operations on a convergent series result in a convergent series?

This question is in the light of the question and answer of: Probability distribution function expressed in terms of a divergent series. We saw that the CDF represented in this form of infinite ...
4
votes
1answer
172 views

Probability distribution function expressed in terms of a divergent series

I'm interested in finding the CDF and PDF of $U_i$ defined as follows, $$U_i=\frac g{d^{\alpha}}$$ where $g$ is a gamma distributed random variable with shape $k$ and scale $\theta$, and $d$ is a ...
0
votes
0answers
13 views

Derivation of variance of Odds Ratio in a 2x2 table

Assuming that the elements of a 2x2 table (a,b,c,d) are binomial in distribution with a being $Bin(n_1,p_1)$ and c being $Bin(n_2,p_2)$. How can we show that $\hat {var(logOR)} = \frac 1 a + \frac 1 b ...
0
votes
1answer
22 views

Taylor explansion of the feed-forward network error

The Optimal Brain Damage paper proposes to approximate the error objective function by a Taylor series. Is E a generic error function? And how is the formula (1) obtained? The paper: http://...
0
votes
0answers
73 views

Quadratic Approximation of the binary logistic regression

I am using https://web.stanford.edu/~hastie/Papers/glmnet.pdf package to solve my optimization problem for the Binary Logistic Regression. On page 10 it is stated that the quadratic approximation of ...
1
vote
0answers
120 views

Multivariate Taylor series for moments of a random variable

In the expectation propagation for the generative aspect model, Minka uses Taylor series for the parameter estimation of the topics $p(w\mid a)$ eq 31. I am a little confused in the last equation. He ...
1
vote
0answers
17 views

Taylor Series Power Function in R

I'm trying to recreate a compartmental model from Yu et al (2017). In this model there is a expanded taylor series power function that is used that I am not understanding how to code. where W(t) = 0, ...
1
vote
0answers
58 views

The question of Taylor expansion of loss function in XGBoost [duplicate]

I am learning XGBoost from documentation, but there are a few questions in the derivation of it. In the part of ...
1
vote
1answer
179 views

xgboost tayler expansion detail [duplicate]

This is the objective function for Xgboost. I have no idea where $g_{i}$ and $h_{i}$ came from is some one explain how this two terms came form? or direct me to the related tutorial page then I ...
0
votes
0answers
23 views

Derive taylor series expansion of df

I was trying to understand ito's lemma. When I came across the taylor series expansion of df(x). df(x) = f'(x) dx + (1/2!) f''(x) (dx)^2 + ... I searched everywhere for the derivation of this but ...
0
votes
1answer
45 views

How to use derivatives of a function to better estimate its variance over the domain?

How to use derivatives of a function to better estimate its variance over the domain? I have a scalar smooth function $f(x)$ and a multivariate random variable $x$ with known distribution (e.g. ...
1
vote
1answer
104 views

Iterated estimation of Taylor series

Say your data generating process is given by the function $y=f(x|\theta)$, where $y$ and $x$ represent variables (data) and $\theta$ represent parameter(s). For convergence reasons (e.g. $f(\cdot)$ is ...
2
votes
1answer
240 views

Taylor expansion for random variables

Let $X_n$ and $Y_n$ be random variables such that $X_n-Y_n\overset{p}{\longrightarrow}0$ as $n\rightarrow\infty$. Let $f(.)$ is a differentiable function. Is the following correct? $f(X_n) = f(Y_n) + ...
4
votes
0answers
140 views

Taylor Series Expansion of Unconditional Expectation

We know that the best 1st order approximation of an unconditional expectation is the following- $$E(y|x)=(E(y)-\beta E(x))+\beta x$$ where $\beta=\frac{\operatorname{Cov}(y,x)}{\operatorname{Var}(x)}...
2
votes
0answers
59 views

Is there any sort of higher-order SVD (quadratic and above) for dimensionality reduction?

X-Posted on math.stackexchange, apologies, though I thought this was equally relevant to both communities. I'm wondering if there exists any higher-order SVD for dimensionality reduction. Note that ...
0
votes
0answers
22 views

Exponential Kth Moment Derivation

I have essentially a mathematical question, relating to deriving the formula for the kth moment of an exponential. I can't seem to work out how we get from the 2nd line to the 3rd line; i.e. the ...
2
votes
2answers
493 views

Can we use backpropagation to fit other models?

It appears that backpropagation is exclusively used to train neural network models. Why not use it to fit other models. For example - Taylor polynomials: $$ f(x) = c_0+c_1(x-a)+c_2(x-a)^2...+c_n(x-a)...
1
vote
0answers
217 views

Understanding a Taylor expansion for the bias of local polynomial regression

I'm trying to understand the proof of an expression for the asymptotic bias in local polynomial regression of degree $p\ge0$. Specifically, I'm distraught with equation $(3.59)$ on page 102 of this ...
0
votes
1answer
54 views

Priors on Taylor Expansion series

I'm wondering what priors can i choose for a Taylor series as follows: $\theta_{1}+\theta_{2} (y-\alpha) + \theta_{3} (y-\alpha)^2$ What priors should I use for updating these parameters ($\theta_{1},...
3
votes
1answer
1k views

Proof that ML Estimator is asymptotically Normal

I'm trying to proof that the Maximum Likelihood Estimator is Asymptotically Normal distribuited. I'm stuck in the lasts steps. Here's what I've done: I do the Taylor's expansion of, that's the mean ...
1
vote
1answer
77 views

Jaynes Probability theory 4.70 (Different answers with Jaynes when using Taylor power series.)

I have read this derivation. $$L(f)\equiv{lng(f|DX)}=nln(f)+(N-n)ln(1-f)+const \;(4.69) $$ expand L(f) in a power series about $\hat{f}$.The first terms as $$L(f) = L(\hat{f}) - \frac{(f-\hat{f})^2}{2\...
0
votes
0answers
354 views

Taylor expansion in xgboost [duplicate]

I'm reading through the math of xgboost: https://xgboost.readthedocs.io/en/latest/model.html Under the ADDITIVE TRAINING section of the objective function, I saw that in the derivation of the ...
0
votes
0answers
116 views

Higher order delta / taylor series approximation relationship to normal distribution?

For a normally distributed variable X, one can call on the delta method to provide an asymptotically normally distributed variable for a non-linear function of it, g(X). This is based on a linear ...
0
votes
1answer
56 views

Estimating polymomial coefficient in R

I used a Taylor series to expand log(1 - ax) so I could estimate the value of parameter 'a'. The expansion becomes -ax - a^2*x^2/2 - a^3*x^3/3 . . . Now I need to estimate the parameter 'a' using ...
1
vote
1answer
476 views

Expansion of Cumulant Generating Function of Negbin

Let $X \thicksim Negbin(r,p)$ where $(0\lt p \lt 1) $ I want to derive skewness and kurtosis of $X$ by getting the Cgf of X. First, since Followance of Negative Binomial equals to the distribution ...
1
vote
1answer
189 views

What is the intuition behind neural networks?

Everywhere in the theory of neural networks, authors saying that idea came about by observing the work of the human brain. But I can not believe in that. I guess, everything is much simpler and neural ...
1
vote
0answers
96 views

Estimate mean and variance of pdf from truncated taylor expansion of logarithm of pdf

In a maximum likelihood fit, one estimates the parameter with the mode of the likelihood $L$, and the variance of this estimator with the second derivative of $\log(L)$: $$ \bar\theta = \mathrm{Mode}[...
3
votes
1answer
230 views

Characteristic function of distribution

$$p(x)=e^{-2 |x|}$$ with x in [-inf, +inf]. I've calculated the characteristic function as $E[e^{ikx}]=\frac{1}{ik+2}-\frac{1}{ik-2}=\frac{4}{k^2+4}$. Now i'd like the moments.. so I suppose I should ...
1
vote
0answers
658 views

Manually compute linearized standard errors of mean in survey

I am would like to manually compute the Taylor-linearized standard error and 95% confidence interval for the mean of a variable x in a survey with 1-stage primary ...
1
vote
0answers
213 views

Delta method with mix of continuous and discrete variables

This is my first question on Cross Validated so please bear with me if my question is lagging in any dimension. My question regards how to evaluate a Jacobian matrix when one variable is binary. I ...
5
votes
1answer
3k views

Are Maximum Likelihood Estimators asymptotically unbiased?

I can follow the proofs in which the asymptotic normal-distribution of a maximum likelihood estimator $\tilde{\theta}_n$ is derived. however, does this already imply that the maximum likelihood ...
5
votes
1answer
507 views

Alternative Confidence interval for Odds Ratio $\hat{p}\over{1-\hat{p}}$ from Logistic Regression?

I have a question about confidence interval calculations for the odds ratio $\hat{p}\over{1-\hat{p}}$ from a logistic regression model (perhaps obtained from the method of Generalized Estimating ...
1
vote
1answer
40 views

Characteristic function issue

As mentioned in a previous post, I've been trying to work through ALL of the problems in Jacod and Protter's Probability Essentials. The following problem has been giving me issues: Let $Z \sim N(0,...
30
votes
1answer
10k views

XGBoost Loss function Approximation With Taylor Expansion

As an example, take the objective function of the XGBoost model on the $t$'th iteration: $$\mathcal{L}^{(t)}=\sum_{i=1}^n\ell(y_i,\hat{y}_i^{(t-1)}+f_t(\mathbf{x}_i))+\Omega(f_t)$$ where $\ell$ is ...
0
votes
0answers
216 views

mixture of Gaussians vs mixture of quadratic denominators (Cauchy)

It is known that mixture of Gaussians are dense in the set of all distribution functions. A 1-dimensional Gaussian has the following density: $$ \frac{1}{\sqrt{2\pi \sigma^2}} e^{-\frac{(\omega-\beta)^...
0
votes
0answers
440 views

Taylor Expansion of Power of Cumulative Log Normal Distribution Function - Show Lagrange Remainder tends to Zero

QUESTION I am looking to find a simplification of the expression below. I have attempted this using the Taylor series. The question then remains if we can show the Lagrange remainder goes to zero. I ...
11
votes
1answer
186 views

Approximating $\log( E(X))$

I was casually reading an article (in economics) which had the following approximation for $\log(E(X))$: $\log(E(X)) \approx E(\log(X))+0.5 \mathrm{var}(\log(X))$, which the author says is exact if ...
3
votes
3answers
950 views

Best basis set for polynomial expansion

I want to do a regression of x onto y: $$f(y)=c_{1}x+c_{2}x^{2}+c_{3}x^{3}\cdots$$ Obviously a plain Taylor expansion as above is suboptimal since the coefficients will not be orthogonal/...
6
votes
0answers
1k views

Why is Fisher Scoring easier to compute?

In practice, the observed information matrix (Newton-Raphson) is usually replaced by its expectation, known as Fisher scoring. Link: https://en.wikipedia.org/wiki/Scoring_algorithm#Fisher_scoring ...
7
votes
2answers
1k views

Help with Taylor expansion of log likelihood function

I've reed the following part of a sketch of the proof that the maximum likelihood estimator is asymptotically normal: "Sketch of the second part of the proof. Recall that we may write the likelihood ...
0
votes
0answers
44 views

Express $E(x^{\alpha})$ in terms of $E(e^{-\zeta x})$? to a 1st or second order?

I have a random variable, $X$, and am able to find $\mathbf{E}(e^{-\zeta X})$ for many $\zeta$ (through the Laplace transform solving an ODE as this actually evolves over time) Is there any way I can ...
2
votes
1answer
294 views

Asking help with Taylor approximation of expectation of ratio

I am trying to understand how I should approach the problem of a Taylor approximation to the expectation of the ratio of two random variables. In my particular problem I am concerned with the ...
1
vote
1answer
127 views

Proof of a step of a lemma on the asymptotics of maximum likelihood where a Taylor expansion is used

I am trying to understand a proof of quite a long theorem that I report completely for the sake of completeness. This is From Jensen and Rahbek Asymptotic Inference for Nonstationary GARCH (2004). My ...
2
votes
0answers
85 views

linearization of an estiamtor

Suppose we have two variables $x$ and $y$ defined in some population, with all values of $x$ known. A Poisson sample is drawn, with corresponding inclusion probabilities $\pi_k$ that are proportional ...
2
votes
0answers
50 views

How can you cluster a set of functions with unknown functional forms?

Say you've $N$ functions $f_N(x)$ defined on a regular grid $x$. You don't know the form of $f(x)$, you've only got several realizations of it. The different functions are related to each other ...
7
votes
1answer
395 views

Given loads of data, can we always model it with polynomials?

Given Taylor series and enough data so as to not risk over-fitting, do you actually need to think about if your phenomenon is following an exponential, quadratic, logarithmic, ..., behaviour? I'm sure ...
3
votes
0answers
444 views

Is there a nice(r) Taylor expansion of the normal quantile function?

Letting $\Phi$ be the CDF of the standard normal, and $\Phi^{-1}$ be the quantile function of the normal, I am looking for the Taylor series expansion of $$ \Phi^{-1}\left(\Phi\left(x\right) + \...