Questions tagged [calculus]

For statistical questions involving calculus. Please use also a more statistical tag. For purely mathemathical questions about the calculus, it is better to ask at math SE https://math.stackexchange.com/

Filter by
Sorted by
Tagged with
0 votes
0 answers
13 views

Question Intuition behind mathematics of activation function in a neural network.

Does this intuition behind why an activation function is used in a neural network make sense mathematically : For this example lets consider a fully connected (NOT CONVOLUTIONAL) network that ...
Stef's user avatar
  • 1
0 votes
0 answers
26 views

Simple matrix calculus but I am struggling to understand [duplicate]

Here is my problem: We have $\mathbf{D} \in \Re^{m n}$, $\mathbf{W} \in \Re^{m q}$, and $\mathbf{X} \in \Re^{q n}$. Furthermore, $\mathbf{D} = \mathbf{W}\mathbf{X}$. (NOT an element wise ...
wrek's user avatar
  • 185
0 votes
0 answers
28 views

Theory and mathematical background of active contour and segmentation models

I am looking for reference books/online material for self-learning the mathematical background in active contours and segmentation models (e.g., snakes, level set, geodesic). Can someone help me with ...
2 votes
1 answer
54 views

Calculating derivative for the final layer of a neural network

I'm first learning about backpropagation in neural networks. We're doing stochastic gradient descent. The lecture provides incomplete detail on computing the derivatives for the final layer. We have ...
Ben G's user avatar
  • 153
0 votes
0 answers
17 views

Solving integrals using incomplete gamma function (upper gamma rule)

I am attempting to integrate this function, ∫15x^(0.28) * e^(-0.21x) dx and am struggling with what techniques to apply. The lower boundary is 0 and the upper boundary is infinity. From research, I ...
user avatar
2 votes
0 answers
100 views

What is the derivative of gamma variates with respect to the shape parameter? [closed]

Given a gamma distribution with unit scale and shape $\theta$, and given an arbitrary variate $x$, what is the derivative of the variate $x$ with respect to $\theta$? In other words, I would like to ...
Neil G's user avatar
  • 15.3k
0 votes
0 answers
40 views

Calculus versus matrix representation in OLS

In the Wikipedia article Ordinary Linear Squares there is an example for finding the estimators $\beta_i$ for a linear model of the sort: $$y_i = \beta_0 + x_1\beta_1 + x_2\beta_2 + \ldots$$ In the ...
Minsky's user avatar
  • 135
1 vote
1 answer
19 views

Changing Bounds and Multiply by -1 [closed]

Sorry my calculus knowledge is extremely rusty - what is the reason that we can flip the bounds from $-\infty\to 0$ to $0\to\infty$ and then also flip the $x$ value to $-x$?
CuriousPenguin's user avatar
5 votes
1 answer
183 views

derivation of coordinate ascent variational inference

From the slides of variational inference, it shows the evidence lower bound ($L$) and the derivative over a variational distribution $q(z_k)$, quoted as follows $$ L_k = \int q(z_k) E_{-k} \bigg[ \log ...
avocado's user avatar
  • 3,581
3 votes
1 answer
143 views

What are some good calculus resources relevant for Machine learning researcher aspirant?

I am trying to self-taught myself on Calculus for machine learning and read the book by Spivak. But it is too rigorous and need a lot of time to finish it. As far as I am concerned, Calculus is only ...
1 vote
1 answer
592 views

Calculate the output of a Neural Network

I have the following problem: Here is my approach: With the activation function: $F(x) = x^2 + 2x + 3$, we can calculate the activation of the two units of the second layer by: $a_1^2 = F(w_{13}\...
Hai Nguyen's user avatar
1 vote
0 answers
121 views

Deriving vectorized back propagation

I'm trying to derive vectorized backpropagation from mostly first principles, but I'm having trouble marrying how this paper explains backpropagation with the derivative of a loss function with ...
Nick Righi's user avatar
1 vote
1 answer
71 views

What is the derivative of a set or a string? [closed]

Neural networks operate on numbers, and it's well-known what the derivative of numeric functions are, as well as what the derivative of matrix functions are. What about functions that operate on maps ...
Wilhelm's user avatar
  • 111
0 votes
0 answers
181 views

Optimising Box-Cox lambda analytically

I'm taking a university course in statistics where the Box-Cox transform is being discussed. As I understand it, we assume that there is some $\lambda$ that makes the sample normally distributed after ...
Mew's user avatar
  • 111
6 votes
2 answers
545 views

Applying Leibniz's integral rule to the Gaussian distribution's normalization condition

I'm working on problem 1.8 of Bishop's Pattern Recognition and Machine Learning and am having a hard time understanding one of the technical details in a solution that I found online. Specifically, ...
SayNo2Decaf's user avatar
4 votes
1 answer
488 views

Gradient and Hessian of loss function

I'm trying to clear up the calculation of the gradient and Hessian of a loss function in an article that I am currently reading. The loss function is given by $$\ell(\beta)=\sum_{i=1}^{N} e^{-y_{i}{{x}...
ADAM's user avatar
  • 706
2 votes
1 answer
618 views

I am not able to understand how did the elementwise multiplication came into the picture of backpropagation in neural networks

I have understood the backpropagation algorithm along with the chain rule well enough that I can derive it on my own, but I don't understand where the elementwise multiplication came from and how does ...
Circuit_Breaker0.7's user avatar
1 vote
1 answer
47 views

Deriving the normal equations' coefficients

Suppose we use the least squares criterion to fit a linear model for the following dataset: $(x_1,y_1),...,(x_m,y_m)\in R \times R$, by solving the following optimisation problem: $$(a^*,b^*) = \text{...
Slim Shady's user avatar
0 votes
1 answer
45 views

What is the relationship between the derivative of a map and its image density? [duplicate]

Preliminaries Suppose we have a random variable $X$ with density $f$ and a suitably smooth function $g: \mathbb{R} \mapsto \mathbb{R}$. The random variable $Y = g(X)$ also has a density function $h$. ...
Galen's user avatar
  • 8,502
2 votes
1 answer
345 views

How can make sure our deep neural network is differentiable

When we have a deep neural network, according to how much complicated that neural network is, how we can make sure that in each layer we can calculate the derivatives?( Is that differentiable or not). ...
Mahdi Amrollahi's user avatar
0 votes
0 answers
18 views

Improper integrals of symmetric functions [duplicate]

First time poster here, so I apologize for any formatting errors. I recently came across the improper integral ∫xdx from -∞ to ∞ and have had a hard time understanding why it isn't zero. My approach ...
Grant Tozer's user avatar
1 vote
0 answers
72 views

PRML Book: Calculus of Variance

I am reading through Pattern Recognition and Machine Learning (PRML) Appendix D (page 705). Here is my question: what does the term $O(\epsilon ^ 2)$ in equation (D.1) and (D.2) stand for?
Jake2099's user avatar
1 vote
1 answer
2k views

Gradient of a multivariate function numpy

I'm trying to calculate the gradient of multivariate function g using NumPy. g = lambda w: -np.sin(np.pi*np.sum(w**2)) + np.log(np.sum(w**2)) ...
Oguz Aktas's user avatar
1 vote
1 answer
1k views

Finding E(XY) for joint probability density

$Joint \:probability\;f(x,y) = 2/3 \:for\: 0 < x < 1, 0 < y < 2, x < y, and\: 0\: otherwise $ $E(XY)=\int_{0}^{1}\int_{x}^{2} \frac{2}{3}xy \:dy \:dx = \frac{7}{12} - (1)$ $E(XY)=\...
Grey Han's user avatar
1 vote
1 answer
43 views

BLUE from calculus

Let $p'\beta$ be an estimable LPF. Suppose that $l'y$ is the candidate which must satisfy the unbiasedness condition and the minimum-variance condition. Formulate this as an optimization problem with ...
thedumbkid's user avatar
3 votes
2 answers
737 views

Calculus for Statistics

If one were to learn calculus solely for the purpose of learning statistics, what should he focus on? If this is a ridiculous question and the honest answer is “All of it,” that is of course an ...
0 votes
0 answers
27 views

How to approximate the expression to $\sum x_i$

How to approximate the expression on the left hand side to $\sum_{i=1}^Nx_i$ as $n\to \infty$ $$ \frac{\sum\limits_{i=1}^{N}x_i^2}{n-2\frac{\sum\limits_{i=1}^{N}x_i}{N}} \left(\sqrt{1+\frac{Nn\left(...
Sara's user avatar
  • 11
3 votes
0 answers
121 views

When will $\mathbb{E}[g(S_n/n)]$ exist given $\mathbb{E}[g(X_1)]$ exists?

Suppose $X_1, X_2,..., X_n$ are i.i.d. random variables with distribution $\pi$ on some probability space. Let $g$ be a measurable function such that $\mathbb E_\pi[g(X_1)]<\infty$. I am curious ...
Bravo's user avatar
  • 111
0 votes
0 answers
24 views

Forecasting Peak/Global Maximum from Raw Data

I'm trying to see what methods there are to predict when the data will peak based on raw values, along with how to accomplish it in R. Here's what you can assume... The data has a start and end point....
Econundrums's user avatar
3 votes
1 answer
191 views

Pattern Recognition and ML Exercise 1.4

I am studying "Pattern Recognition and Machine Learning" by Christopher Bishop and I'm trying to understand his solution in the solution manual to exercise 1.4. The problem statement for ...
Slim Shady's user avatar
2 votes
1 answer
140 views

Differentiating a Vector and a Matrix w.r.t. a Vector [Matrix Calculus]

I am studying matrix calculus for linear regression and machine learning and I would like to know exactly if the following calculations are correct: Let $y=\sin(x+yz)$ and $r=\begin{bmatrix}x\\y\\z\...
Slim Shady's user avatar
3 votes
1 answer
725 views

Neural Networks: How to get the gradient vector for the xOr problem?

I'm reading about neural networks, but the material I find is sometimes very abstract or just copies of something. Well, when considering the $xOr$ problem, I have a network in the following structure ...
David's user avatar
  • 110
2 votes
1 answer
55 views

Elements of Statistical Learning Integral Notation

In equation 2.9 and 2.10 on page 18 of ESL we have $$E(Y - f(X))^2 = \int [y - f(x)]^2 Pr(dx, dy)$$ However this notation confuses me. I'm rather expecting $$E(Y - f(X))^2 = \int [y - f(x)]^2 Pr(x, y)...
smartstix's user avatar
0 votes
0 answers
504 views

Multicollinearity in quadratic (polynomial) regression function [duplicate]

Multicollinearity problem could arise when we add quadratic variable in regression like this: So, one of the possible solutions to eliminate the problem is to add centered variables: This was ...
Gregory Stelmashenko's user avatar
2 votes
0 answers
39 views

Directional derivative in regression (coefficients, after all, are partial derivatives)

The coefficients in a (let's stick with linear for now) regression are the partial derivatives. A regression equation is a function of several variables, so all of the multivariable calculus tricks ...
Dave's user avatar
  • 62.5k
0 votes
0 answers
265 views

Is this statement about the sum of quantiles correct?

Let $X$ and $Y$ be continuous random variables both having some density, not identically distributed but independent. Imagine I'm interested in the quantile $q_{X+Y}(\alpha)$ for some $\alpha \in (0,1)...
Ivan's user avatar
  • 11
3 votes
1 answer
297 views

$f$ is a decreasing function whose integral converges. Does $\lim_{x \to \infty}xf(x) = 0$?

My finals are over and I cannot help but ruminate over this particular problem. Could anyone help prove this? Suppose $f$ is a continuous decreasing function on $[0,\infty)$ and $\int_0^\infty f(t)\, ...
Joel Tan's user avatar
5 votes
2 answers
629 views

Why is the formula for the density of a transformed random variable expressed in terms of the derivative of the inverse?

In this very nice answer, the intuitive explanation of the formula for the density of a transformed random variable, $Y = g(X)$, leads naturally to an expression like $$f_Y(y) = \frac{f_X(g^{-1}(y))}{...
R Hahn's user avatar
  • 201
9 votes
1 answer
587 views

Limit of Integration of continuous function

How to evaluate the following limit- $$\lim_{n \to \infty} \int_0^1 \int_0^1\cdots\int_0^1 f \bigg(\frac{x_1 + x_2 + \cdots + x_n}{n} \bigg) dx_1 dx_2....dx_n$$. Here $f()$ is a continuous function $f:...
edison's user avatar
  • 137
3 votes
2 answers
99 views

Deriving OLS estimator

In my course on linear models we derived the OLS estimator by minimizing the residuals $F(\phi) = (Y-X\phi)'(Y-X\phi)$. However there is one step that I do not understand: to find the minimum over all ...
mafe's user avatar
  • 51
0 votes
0 answers
79 views

Rewriting the probability density function as a probability function

Letting $dt$ be an infinitesimal interval, what is the argument to that $$f(t | H_{t_n})\;dt = P (t \in [t,t+dt] | H_{t_n}),$$ where $H_{t_n}$ denotes the history of the previous points before $t$? I ...
Yasmin's user avatar
  • 21
1 vote
0 answers
86 views

Calculus in Moment Generating Function

On page 156 of the Statistics textbook, "Mathematical Statistics and Data Analysis" by John A. Rice, I came up with two questions on the section about Moment Generating Functions: 1. Why ...
E. Kaufman's user avatar
2 votes
1 answer
176 views

Is the sample mean of the gradient the same as the gradient of the sample mean?

By the law of large numbers, given a continuous random vector $\mathbf{x}$, then: $$ \mathbb{E}[\mathbf{x}] \approx \frac{1}{N} \sum_{i=1}^{N} \mathbf{x}_i $$ Where $\mathbf{x}_1,\mathbf{x}_2,...,\...
mhdadk's user avatar
  • 4,950
0 votes
0 answers
137 views

Algebra: When calculating the variance of Zero-inflated Poisson dist

I am deriving the variance of zero-inflated Poisson distribution, whose PMF is $$ P(X=k) = \begin{cases} \pi + (1-\pi)e^{-\lambda} \; , \; if \; k=0 \\ (1- \pi) e^{-\lambda} \frac{\lambda^k}{k!} \; , \...
KLee's user avatar
  • 161
1 vote
0 answers
388 views

How to differentiate the hinge loss?

I'm asked to differentiate the following hinge loss term. $$ \dfrac{1}{n}\sum _{\left( x_{i},y_{1}\right) \in S}\sum _{j'=1}L\left( w^{j'};\left( x_{i},y_{i}\right) \right) $$ where $$ L\left( w^{j'};\...
WindBreeze's user avatar
0 votes
1 answer
75 views

Approximate / Standardize value in certain range

I have table with numeric values like ...
jas_0n's user avatar
  • 11
1 vote
2 answers
51 views

Backpropogation Derivatives

I've been working on trying to understand the backpropogation algorithm and the calculus behind it, and in my work I have stumbled across a sort of odd situation. I am just practicing on a 1 input, 1 ...
NathanK's user avatar
  • 31
1 vote
1 answer
16 views

What is the mean average of $y=kg^t$ from $t=a$ to $t=b$ [closed]

Mean average of $y$ in $y=kg^t$ from $t=a$ to $t=b$. $g$ is a constant, $t$ varies. I have looked this up in textbooks and online and all I can find is the mean average of a function where $t$ is a ...
HumbleOrange's user avatar
1 vote
0 answers
64 views

Is there a smart algorithm of finding the maximum of $X^{\top}a$ with $X$ and $a$ both belong to some compact convex set? [closed]

Suppose $X\in\mathcal{X}\subset R^k$ and $a\in\mathcal{A}\subset R^k$, where $\mathcal{X}$ and $\mathcal{A}$ are both compact convex set. Is there a systematic way of finding the maximum of $X^{\top}a$...
ExcitedSnail's user avatar
  • 2,788
2 votes
1 answer
109 views

Book recommendations needed - building foundational knowledge for ISL - Introduction to Statistical Learning (by Gareth James)

I'm trying to build a data science base from scratch. I started a book called Introduction to Statistical Learning by Gareth James and found that there are many mathematical & statistical concepts ...