Questions tagged [calculus]

For statistical questions involving calculus. Please use also a more statistical tag. For purely mathemathical questions about the calculus, it is better to ask at math SE https://math.stackexchange.com/

Filter by
Sorted by
Tagged with
0 votes
0 answers
20 views

Variance of infinite weighted sum of normal distributions [closed]

We define the random variable $X_i\sim\mathcal N(0,I)$ where $\mathcal N$ is the multivariate normal distribution with a mean of $0$ and a variance matrix of $I$ (the identity). We also define a ...
  • 205
1 vote
0 answers
27 views

I am not able to understand how did the elementwise multiplication came into the picture of backpropagation in neural networks

I have understood the backpropagation algorithm along with the chain rule well enough that I can derive it on my own, but I don't understand where the elementwise multiplication came from and how does ...
0 votes
1 answer
41 views

Deriving the normal equations' coefficients

Suppose we use the least squares criterion to fit a linear model for the following dataset: $(x_1,y_1),...,(x_m,y_m)\in R \times R$, by solving the following optimisation problem: $$(a^*,b^*) = \text{...
0 votes
1 answer
29 views

What is the relationship between the derivative of a map and its image density? [duplicate]

Preliminaries Suppose we have a random variable $X$ with density $f$ and a suitably smooth function $g: \mathbb{R} \mapsto \mathbb{R}$. The random variable $Y = g(X)$ also has a density function $h$. ...
  • 4,590
2 votes
1 answer
20 views

How can make sure our deep neural network is differentiable

When we have a deep neural network, according to how much complicated that neural network is, how we can make sure that in each layer we can calculate the derivatives?( Is that differentiable or not). ...
0 votes
0 answers
17 views

Improper integrals of symmetric functions [duplicate]

First time poster here, so I apologize for any formatting errors. I recently came across the improper integral ∫xdx from -∞ to ∞ and have had a hard time understanding why it isn't zero. My approach ...
1 vote
0 answers
34 views

PRML Book: Calculus of Variance

I am reading through Pattern Recognition and Machine Learning (PRML) Appendix D (page 705). Here is my question: what does the term $O(\epsilon ^ 2)$ in equation (D.1) and (D.2) stand for?
1 vote
1 answer
630 views

Gradient of a multivariate function numpy

I'm trying to calculate the gradient of multivariate function g using NumPy. g = lambda w: -np.sin(np.pi*np.sum(w**2)) + np.log(np.sum(w**2)) ...
1 vote
1 answer
142 views

Finding E(XY) for joint probability density

$Joint \:probability\;f(x,y) = 2/3 \:for\: 0 < x < 1, 0 < y < 2, x < y, and\: 0\: otherwise $ $E(XY)=\int_{0}^{1}\int_{x}^{2} \frac{2}{3}xy \:dy \:dx = \frac{7}{12} - (1)$ $E(XY)=\...
0 votes
1 answer
37 views

BLUE from calculus

Let $p'\beta$ be an estimable LPF. Suppose that $l'y$ is the candidate which must satisfy the unbiasedness condition and the minimum-variance condition. Formulate this as an optimization problem with ...
2 votes
2 answers
216 views

Calculus for Statistics

If one were to learn calculus solely for the purpose of learning statistics, what should he focus on? If this is a ridiculous question and the honest answer is “All of it,” that is of course an ...
0 votes
0 answers
25 views

How to approximate the expression to $\sum x_i$

How to approximate the expression on the left hand side to $\sum_{i=1}^Nx_i$ as $n\to \infty$ $$ \frac{\sum\limits_{i=1}^{N}x_i^2}{n-2\frac{\sum\limits_{i=1}^{N}x_i}{N}} \left(\sqrt{1+\frac{Nn\left(...
  • 11
0 votes
0 answers
27 views

Plot Partial derivatives from Linear Regression

I am working on the link between Calculus and solutions for Linear Regression. Let's suppose a linear regression for a given individual. $$ y_i = \beta_0 + \beta_1x_i +\epsilon_i; \epsilon\sim N(0, \...
0 votes
0 answers
10 views

Differentiating a function with respect to a matrix [duplicate]

I'm new to matrix calculus and I'm trying to find the formulas for matrix differentiation. e.g. $\frac{\partial f}{\partial z}$ = zzx where z is a KxK matrix, and x is a vector in K I found a few ...
3 votes
0 answers
120 views

When will $\mathbb{E}[g(S_n/n)]$ exist given $\mathbb{E}[g(X_1)]$ exists?

Suppose $X_1, X_2,..., X_n$ are i.i.d. random variables with distribution $\pi$ on some probability space. Let $g$ be a measurable function such that $\mathbb E_\pi[g(X_1)]<\infty$. I am curious ...
  • 111
0 votes
0 answers
22 views

Forecasting Peak/Global Maximum from Raw Data

I'm trying to see what methods there are to predict when the data will peak based on raw values, along with how to accomplish it in R. Here's what you can assume... The data has a start and end point....
2 votes
1 answer
87 views

Pattern Recognition and ML Exercise 1.4

I am studying "Pattern Recognition and Machine Learning" by Christopher Bishop and I'm trying to understand his solution in the solution manual to exercise 1.4. The problem statement for ...
2 votes
1 answer
52 views

Differentiating a Vector and a Matrix w.r.t. a Vector [Matrix Calculus]

I am studying matrix calculus for linear regression and machine learning and I would like to know exactly if the following calculations are correct: Let $y=\sin(x+yz)$ and $r=\begin{bmatrix}x\\y\\z\...
3 votes
1 answer
229 views

Neural Networks: How to get the gradient vector for the xOr problem?

I'm reading about neural networks, but the material I find is sometimes very abstract or just copies of something. Well, when considering the $xOr$ problem, I have a network in the following structure ...
  • 98
2 votes
1 answer
44 views

Elements of Statistical Learning Integral Notation

In equation 2.9 and 2.10 on page 18 of ESL we have $$E(Y - f(X))^2 = \int [y - f(x)]^2 Pr(dx, dy)$$ However this notation confuses me. I'm rather expecting $$E(Y - f(X))^2 = \int [y - f(x)]^2 Pr(x, y)...
0 votes
0 answers
109 views

Multicollinearity in quadratic (polynomial) regression function [duplicate]

Multicollinearity problem could arise when we add quadratic variable in regression like this: So, one of the possible solutions to eliminate the problem is to add centered variables: This was ...
1 vote
0 answers
18 views

Directional derivative in regression (coefficients, after all, are partial derivatives)

The coefficients in a (let's stick with linear for now) regression are the partial derivatives. A regression equation is a function of several variables, so all of the multivariable calculus tricks ...
  • 35.4k
0 votes
0 answers
38 views

Is this statement about the sum of quantiles correct?

Let $X$ and $Y$ be continuous random variables both having some density, not identically distributed but independent. Imagine I'm interested in the quantile $q_{X+Y}(\alpha)$ for some $\alpha \in (0,1)...
  • 11
3 votes
1 answer
138 views

$f$ is a decreasing function whose integral converges. Does $\lim_{x \to \infty}xf(x) = 0$?

My finals are over and I cannot help but ruminate over this particular problem. Could anyone help prove this? Suppose $f$ is a continuous decreasing function on $[0,\infty)$ and $\int_0^\infty f(t)\, ...
4 votes
2 answers
260 views

Why is the formula for the density of a transformed random variable expressed in terms of the derivative of the inverse?

In this very nice answer, the intuitive explanation of the formula for the density of a transformed random variable, $Y = g(X)$, leads naturally to an expression like $$f_Y(y) = \frac{f_X(g^{-1}(y))}{...
  • 191
8 votes
1 answer
521 views

Limit of Integration of continuous function

How to evaluate the following limit- $$\lim_{n \to \infty} \int_0^1 \int_0^1\cdots\int_0^1 f \bigg(\frac{x_1 + x_2 + \cdots + x_n}{n} \bigg) dx_1 dx_2....dx_n$$. Here $f()$ is a continuous function $f:...
  • 127
2 votes
1 answer
32 views

Deriving OLS estimator

In my course on linear models we derived the OLS estimator by minimizing the residuals $F(\phi) = (Y-X\phi)'(Y-X\phi)$. However there is one step that I do not understand: to find the minimum over all ...
  • 31
0 votes
0 answers
51 views

Rewriting the probability density function as a probability function

Letting $dt$ be an infinitesimal interval, what is the argument to that $$f(t | H_{t_n})\;dt = P (t \in [t,t+dt] | H_{t_n}),$$ where $H_{t_n}$ denotes the history of the previous points before $t$? I ...
  • 21
1 vote
0 answers
47 views

Calculus in Moment Generating Function

On page 156 of the Statistics textbook, "Mathematical Statistics and Data Analysis" by John A. Rice, I came up with two questions on the section about Moment Generating Functions: 1. Why ...
2 votes
1 answer
86 views

Is the sample mean of the gradient the same as the gradient of the sample mean?

By the law of large numbers, given a continuous random vector $\mathbf{x}$, then: $$ \mathbb{E}[\mathbf{x}] \approx \frac{1}{N} \sum_{i=1}^{N} \mathbf{x}_i $$ Where $\mathbf{x}_1,\mathbf{x}_2,...,\...
  • 3,159
0 votes
0 answers
58 views

Algebra: When calculating the variance of Zero-inflated Poisson dist

I am deriving the variance of zero-inflated Poisson distribution, whose PMF is $$ P(X=k) = \begin{cases} \pi + (1-\pi)e^{-\lambda} \; , \; if \; k=0 \\ (1- \pi) e^{-\lambda} \frac{\lambda^k}{k!} \; , \...
1 vote
0 answers
232 views

How to differentiate the hinge loss?

I'm asked to differentiate the following hinge loss term. $$ \dfrac{1}{n}\sum _{\left( x_{i},y_{1}\right) \in S}\sum _{j'=1}L\left( w^{j'};\left( x_{i},y_{i}\right) \right) $$ where $$ L\left( w^{j'};\...
0 votes
1 answer
43 views

Approximate / Standardize value in certain range

I have table with numeric values like ...
  • 11
1 vote
2 answers
45 views

Backpropogation Derivatives

I've been working on trying to understand the backpropogation algorithm and the calculus behind it, and in my work I have stumbled across a sort of odd situation. I am just practicing on a 1 input, 1 ...
  • 31
1 vote
1 answer
16 views

What is the mean average of $y=kg^t$ from $t=a$ to $t=b$ [closed]

Mean average of $y$ in $y=kg^t$ from $t=a$ to $t=b$. $g$ is a constant, $t$ varies. I have looked this up in textbooks and online and all I can find is the mean average of a function where $t$ is a ...
1 vote
0 answers
63 views

Is there a smart algorithm of finding the maximum of $X^{\top}a$ with $X$ and $a$ both belong to some compact convex set? [closed]

Suppose $X\in\mathcal{X}\subset R^k$ and $a\in\mathcal{A}\subset R^k$, where $\mathcal{X}$ and $\mathcal{A}$ are both compact convex set. Is there a systematic way of finding the maximum of $X^{\top}a$...
  • 1,898
2 votes
1 answer
70 views

Book recommendations needed - building foundational knowledge for ISL - Introduction to Statistical Learning (by Gareth James)

I'm trying to build a data science base from scratch. I started a book called Introduction to Statistical Learning by Gareth James and found that there are many mathematical & statistical concepts ...
3 votes
1 answer
151 views

Differentiating $ (y-X\beta)^T(y - X \beta) $ with respect to $\beta$

How do I differentiate $$ (y-X\beta)^T(y - X \beta) $$ with respect to $\beta$. The result I saw was $$X^T(y - X\beta)$$
  • 299
0 votes
0 answers
80 views

Backpropagation through time for stacked RNNs

I was able to find the partial derivative of the cost function with respects to a single variable without much difficulty. However, this requires propagating backwards through the network for each ...
  • 123
1 vote
0 answers
62 views

Evaluation of Limit involved in the proof of Asymptotic Unbiasedness

We know that $S^{2}$ is an unbiased estimator of $\sigma^{2}$ and $S$ is a biased estimator of $\sigma$. But if $n\rightarrow\infty$, then $S$ is an asymptotically unbiased estimator of $\sigma$. I ...
  • 11
6 votes
1 answer
2k views

Derivation of M step for Gaussian mixture model

Summary So to summarize my question, how can I take \begin{align} = \sum_{i=1}^{n}W_{i1} \left(log (1-\sum_{j=2}^{K}\pi_j) -\frac{1}{2} log(|\Sigma_1|) -\frac{d}{2} log(2\pi) -\frac{1}{2}(x_i-\mu_1)^{...
1 vote
0 answers
25 views

Question about Functions of Several Random Variables

In the Mathematical Statistics and Data Analysis by John Rice, it states that for random variables $U,V$ which are functions of random variables $X,Y$, we have: We know that $$f_{UV}(u,v) = f_{XY}(h_1(...
  • 131
7 votes
1 answer
347 views

Correlation between normal and log-normal variables

(This is not a homework question.) Let $(X_1 \sim N(\mu_1,\sigma_1), X_2 \sim N(\mu_2, \sigma_2))$ be a bivariate normal random variable with the correlation between $X_1$ and $X_2$ given by $\rho$. ...
0 votes
2 answers
70 views

Maths for deeply understanding backpropgation

I have been trying to develop a deeper understanding of Neural Networks so I can understand the libraries such as tensorflow and others. I have had good success with pereceptron models, and have a ...
  • 31
3 votes
2 answers
108 views

How does one compute a variational derivative?

The expected regression loss is given as:$$E[L]=\int\int \{y(\mathbf x)-t\}^2 p(\mathbf x,t)d\mathbf xdt$$ To minimise the expected loss,Euler Lagrange equation is used which goes like this in the ...
0 votes
1 answer
579 views

Expected Predicted Error (EPE) with L1 loss

In Element of Statistical learning it is saying on page 20, equation 2.18. That using the L1 norm instead of the usual L2 norm leads to an $f(X)$ optimising the EPE being the median instead of the ...
  • 141
2 votes
0 answers
221 views

Why does the dimension of gradient and Hessian matrix not conform for this function?

The function is $f(\mathbf{x}) = e^{-\frac{1}{2}\mathbf{x^TAx}}$, where $\mathbf{A}$ is a square symmetric matrix, and $\mathbf{x}$ is an n-vector. What I found were: $$ \begin{align*} \nabla f ...
0 votes
0 answers
192 views

What is the first order derivative of linear regression's cost function using matrix calculus?

For linear regression's cost function J(b), where X is a n*m matrix, b is a m*1 vector and y is n*1 vector: First order derivative with respect to vector b (coefficients) is shown to be Using the ...
1 vote
0 answers
148 views

On the convergence of infinite sum of a hypergeometric function resulted from a nested sum

I am interested in finding the CDF of the sum $U=\sum_{i=0}^N U_i$ where: $$F_{U_i}(x)=\sum_{n=0}^\infty \frac {2(-1)^nR^{k\alpha+n\alpha}x^{k+n}}{\Gamma(k)n!\theta^{k+n}(k+n)(k\alpha+n\alpha+2)}$$ ...
  • 97
1 vote
0 answers
80 views

How to approximate expectation and variance of an integral from a discrete Time series financial dataset?

I have discrete time series financial data, with time($u$), price($S$) and someVariable($q$) which looks something like this. ...