Questions tagged [calculus]
For statistical questions involving calculus. Please use also a more statistical tag. For purely mathemathical questions about the calculus, it is better to ask at math SE https://math.stackexchange.com/
111
questions
1
vote
1
answer
29
views
How can I prove monotonicity of slope MLE in EIV regression model?
I'm trying to figure out Casella and Berger Exercise 12.4(c), regarding monotonicity of the maximum likelihood estimator of the slope of an errors-in-variables regression model. The goal is to show ...
- 11
0
votes
0
answers
15
views
Question Intuition behind mathematics of activation function in a neural network.
Does this intuition behind why an activation function is used in a neural network make sense mathematically :
For this example lets consider a fully connected (NOT CONVOLUTIONAL) network that ...
- 1
-1
votes
0
answers
30
views
Simple matrix calculus but I am struggling to understand [duplicate]
Here is my problem: We have $\mathbf{D} \in \Re^{m n}$, $\mathbf{W} \in \Re^{m q}$, and $\mathbf{X} \in \Re^{q n}$. Furthermore, $\mathbf{D} = \mathbf{W}\mathbf{X}$. (NOT an element wise ...
- 193
0
votes
0
answers
28
views
Theory and mathematical background of active contour and segmentation models
I am looking for reference books/online material for self-learning the mathematical background in active contours and segmentation models (e.g., snakes, level set, geodesic). Can someone help me with ...
2
votes
1
answer
59
views
Calculating derivative for the final layer of a neural network
I'm first learning about backpropagation in neural networks. We're doing stochastic gradient descent.
The lecture provides incomplete detail on computing the derivatives for the final layer.
We have ...
- 153
0
votes
0
answers
22
views
Solving integrals using incomplete gamma function (upper gamma rule)
I am attempting to integrate this function, ∫15x^(0.28) * e^(-0.21x) dx and am struggling with what techniques to apply. The lower boundary is 0 and the upper boundary is infinity. From research, I ...
user387381
2
votes
0
answers
105
views
What is the derivative of gamma variates with respect to the shape parameter? [closed]
Given a gamma distribution with unit scale and shape $\theta$, and given an arbitrary variate $x$, what is the derivative of the variate $x$ with respect to $\theta$?
In other words, I would like to ...
- 15.4k
0
votes
0
answers
44
views
Calculus versus matrix representation in OLS
In the Wikipedia article Ordinary Linear Squares there is an example for finding the estimators $\beta_i$ for a linear model of the sort:
$$y_i = \beta_0 + x_1\beta_1 + x_2\beta_2 + \ldots$$
In the ...
user306937
1
vote
1
answer
19
views
Changing Bounds and Multiply by -1 [closed]
Sorry my calculus knowledge is extremely rusty - what is the reason that we can flip the bounds from $-\infty\to 0$ to $0\to\infty$ and then also flip the $x$ value to $-x$?
5
votes
1
answer
198
views
derivation of coordinate ascent variational inference
From the slides of variational inference, it shows the evidence lower bound ($L$) and the derivative over a variational distribution $q(z_k)$, quoted as follows
$$
L_k = \int q(z_k) E_{-k} \bigg[ \log ...
- 3,623
3
votes
1
answer
154
views
What are some good calculus resources relevant for Machine learning researcher aspirant?
I am trying to self-taught myself on Calculus for machine learning and read the book by Spivak. But it is too rigorous and need a lot of time to finish it.
As far as I am concerned, Calculus is only ...
1
vote
1
answer
811
views
Calculate the output of a Neural Network
I have the following problem:
Here is my approach:
With the activation function: $F(x) = x^2 + 2x + 3$, we can calculate the activation of the two units of the second layer by: $a_1^2 = F(w_{13}\...
- 11
1
vote
0
answers
126
views
Deriving vectorized back propagation
I'm trying to derive vectorized backpropagation from mostly first principles, but I'm having
trouble marrying how
this
paper explains backpropagation with the derivative of a loss function with ...
- 11
1
vote
1
answer
77
views
What is the derivative of a set or a string? [closed]
Neural networks operate on numbers, and it's well-known what the derivative of numeric functions are, as well as what the derivative of matrix functions are.
What about functions that operate on maps ...
- 111
0
votes
0
answers
260
views
Optimising Box-Cox lambda analytically
I'm taking a university course in statistics where the Box-Cox transform is being discussed. As I understand it, we assume that there is some $\lambda$ that makes the sample normally distributed after ...
- 111
6
votes
2
answers
608
views
Applying Leibniz's integral rule to the Gaussian distribution's normalization condition
I'm working on problem 1.8 of Bishop's Pattern Recognition and Machine Learning and am having a hard time understanding one of the technical details in a solution that I found online. Specifically, ...
- 63
4
votes
1
answer
640
views
Gradient and Hessian of loss function
I'm trying to clear up the calculation of the gradient and Hessian of a loss function in an article that I am currently reading. The loss function is given by
$$\ell(\beta)=\sum_{i=1}^{N} e^{-y_{i}{{x}...
- 741
2
votes
1
answer
724
views
I am not able to understand how did the elementwise multiplication came into the picture of backpropagation in neural networks
I have understood the backpropagation algorithm along with the chain rule well enough that I can derive it on my own, but I don't understand where the elementwise multiplication came from and how does ...
1
vote
1
answer
47
views
Deriving the normal equations' coefficients
Suppose we use the least squares criterion to fit a linear model for the following dataset: $(x_1,y_1),...,(x_m,y_m)\in R \times R$, by solving the following optimisation problem:
$$(a^*,b^*) = \text{...
- 309
0
votes
1
answer
50
views
What is the relationship between the derivative of a map and its image density? [duplicate]
Preliminaries
Suppose we have a random variable $X$ with density $f$ and a suitably smooth function $g: \mathbb{R} \mapsto \mathbb{R}$. The random variable $Y = g(X)$ also has a density function $h$.
...
- 9,512
2
votes
1
answer
471
views
How can make sure our deep neural network is differentiable
When we have a deep neural network, according to how much complicated that neural network is, how we can make sure that in each layer we can calculate the derivatives?( Is that differentiable or not). ...
- 159
0
votes
0
answers
18
views
Improper integrals of symmetric functions [duplicate]
First time poster here, so I apologize for any formatting errors. I recently came across the improper integral ∫xdx from -∞ to ∞ and have had a hard time understanding why it isn't zero. My approach ...
1
vote
0
answers
74
views
PRML Book: Calculus of Variance
I am reading through Pattern Recognition and Machine Learning (PRML) Appendix D (page 705).
Here is my question: what does the term $O(\epsilon ^ 2)$ in equation (D.1) and (D.2) stand for?
- 11
1
vote
1
answer
2k
views
Gradient of a multivariate function numpy
I'm trying to calculate the gradient of multivariate function g using NumPy.
g = lambda w: -np.sin(np.pi*np.sum(w**2)) + np.log(np.sum(w**2))
...
- 13
1
vote
1
answer
1k
views
Finding E(XY) for joint probability density
$Joint \:probability\;f(x,y) = 2/3 \:for\: 0 < x < 1, 0 < y < 2, x < y, and\: 0\: otherwise $
$E(XY)=\int_{0}^{1}\int_{x}^{2} \frac{2}{3}xy \:dy \:dx = \frac{7}{12} - (1)$
$E(XY)=\...
- 45
1
vote
1
answer
44
views
BLUE from calculus
Let $p'\beta$ be an estimable LPF. Suppose
that $l'y$ is the candidate which must satisfy the unbiasedness
condition and the minimum-variance condition. Formulate this as an optimization problem with ...
- 184
3
votes
2
answers
804
views
Calculus for Statistics
If one were to learn calculus solely for the purpose of learning statistics, what should he focus on? If this is a ridiculous question and the honest answer is “All of it,” that is of course an ...
0
votes
0
answers
27
views
How to approximate the expression to $\sum x_i$
How to approximate the expression on the left hand side to $\sum_{i=1}^Nx_i$ as $n\to \infty$
$$ \frac{\sum\limits_{i=1}^{N}x_i^2}{n-2\frac{\sum\limits_{i=1}^{N}x_i}{N}}
\left(\sqrt{1+\frac{Nn\left(...
- 11
3
votes
0
answers
121
views
When will $\mathbb{E}[g(S_n/n)]$ exist given $\mathbb{E}[g(X_1)]$ exists?
Suppose $X_1, X_2,..., X_n$ are i.i.d. random variables with distribution $\pi$ on some probability space. Let $g$ be a measurable function such that $\mathbb E_\pi[g(X_1)]<\infty$. I am curious ...
- 111
0
votes
0
answers
25
views
Forecasting Peak/Global Maximum from Raw Data
I'm trying to see what methods there are to predict when the data will peak based on raw values, along with how to accomplish it in R. Here's what you can assume...
The data has a start and end point....
3
votes
1
answer
205
views
Pattern Recognition and ML Exercise 1.4
I am studying "Pattern Recognition and Machine Learning" by Christopher Bishop and I'm trying to understand his solution in the solution manual to exercise 1.4.
The problem statement for ...
- 309
2
votes
1
answer
147
views
Differentiating a Vector and a Matrix w.r.t. a Vector [Matrix Calculus]
I am studying matrix calculus for linear regression and machine learning and I would like to know exactly if the following calculations are correct:
Let $y=\sin(x+yz)$ and $r=\begin{bmatrix}x\\y\\z\...
- 309
3
votes
1
answer
755
views
Neural Networks: How to get the gradient vector for the xOr problem?
I'm reading about neural networks, but the material I find is sometimes very abstract or just copies of something. Well, when considering the $xOr$ problem, I have a network in the following structure
...
- 110
2
votes
1
answer
61
views
Elements of Statistical Learning Integral Notation
In equation 2.9 and 2.10 on page 18 of ESL we have
$$E(Y - f(X))^2 = \int [y - f(x)]^2 Pr(dx, dy)$$
However this notation confuses me. I'm rather expecting
$$E(Y - f(X))^2 = \int [y - f(x)]^2 Pr(x, y)...
- 21
0
votes
0
answers
568
views
Multicollinearity in quadratic (polynomial) regression function [duplicate]
Multicollinearity problem could arise when we add quadratic variable in regression like this:
So, one of the possible solutions to eliminate the problem is to add centered variables:
This was ...
2
votes
0
answers
42
views
Directional derivative in regression (coefficients, after all, are partial derivatives)
The coefficients in a (let's stick with linear for now) regression are the partial derivatives. A regression equation is a function of several variables, so all of the multivariable calculus tricks ...
- 65.6k
0
votes
0
answers
311
views
Is this statement about the sum of quantiles correct?
Let $X$ and $Y$ be continuous random variables both having some density, not identically distributed but independent.
Imagine I'm interested in the quantile $q_{X+Y}(\alpha)$ for some $\alpha \in (0,1)...
- 11
3
votes
1
answer
331
views
$f$ is a decreasing function whose integral converges. Does $\lim_{x \to \infty}xf(x) = 0$?
My finals are over and I cannot help but ruminate over this particular problem. Could anyone help prove this?
Suppose $f$ is a continuous decreasing function on $[0,\infty)$ and $\int_0^\infty f(t)\, ...
- 49
5
votes
2
answers
744
views
Why is the formula for the density of a transformed random variable expressed in terms of the derivative of the inverse?
In this very nice answer, the intuitive explanation of the formula for the density of a transformed random variable, $Y = g(X)$, leads naturally to an expression like
$$f_Y(y) = \frac{f_X(g^{-1}(y))}{...
- 201
10
votes
1
answer
625
views
Limit of Integration of continuous function
How to evaluate the following limit-
$$\lim_{n \to \infty} \int_0^1 \int_0^1\cdots\int_0^1 f \bigg(\frac{x_1 + x_2 + \cdots + x_n}{n} \bigg) dx_1 dx_2....dx_n$$.
Here $f(\cdot)$ is a continuous ...
- 167
3
votes
2
answers
101
views
Deriving OLS estimator
In my course on linear models we derived the OLS estimator by minimizing the residuals $F(\phi) = (Y-X\phi)'(Y-X\phi)$. However there is one step that I do not understand: to find the minimum over all ...
- 51
0
votes
0
answers
80
views
Rewriting the probability density function as a probability function
Letting $dt$ be an infinitesimal interval, what is the argument to that
$$f(t | H_{t_n})\;dt = P (t \in [t,t+dt] | H_{t_n}),$$
where $H_{t_n}$ denotes the history of the previous points before $t$? I ...
- 21
1
vote
0
answers
102
views
Calculus in Moment Generating Function
On page 156 of the Statistics textbook, "Mathematical Statistics and Data Analysis" by John A. Rice, I came up with two questions on the section about Moment Generating Functions:
1.
Why ...
- 163
2
votes
1
answer
193
views
Is the sample mean of the gradient the same as the gradient of the sample mean?
By the law of large numbers, given a continuous random vector $\mathbf{x}$, then:
$$
\mathbb{E}[\mathbf{x}] \approx \frac{1}{N} \sum_{i=1}^{N} \mathbf{x}_i
$$
Where $\mathbf{x}_1,\mathbf{x}_2,...,\...
- 5,110
0
votes
0
answers
143
views
Algebra: When calculating the variance of Zero-inflated Poisson dist
I am deriving the variance of zero-inflated Poisson distribution, whose PMF is
$$
P(X=k) = \begin{cases} \pi + (1-\pi)e^{-\lambda} \; , \; if \; k=0 \\ (1- \pi) e^{-\lambda} \frac{\lambda^k}{k!} \; , \...
- 335
1
vote
0
answers
413
views
How to differentiate the hinge loss?
I'm asked to differentiate the following hinge loss term.
$$ \dfrac{1}{n}\sum _{\left( x_{i},y_{1}\right) \in S}\sum _{j'=1}L\left( w^{j'};\left( x_{i},y_{i}\right) \right) $$
where
$$
L\left( w^{j'};\...
- 131
0
votes
1
answer
75
views
1
vote
2
answers
52
views
Backpropogation Derivatives
I've been working on trying to understand the backpropogation algorithm and the calculus behind it, and in my work I have stumbled across a sort of odd situation. I am just practicing on a 1 input, 1 ...
- 31
1
vote
1
answer
16
views
What is the mean average of $y=kg^t$ from $t=a$ to $t=b$ [closed]
Mean average of $y$ in $y=kg^t$ from $t=a$ to $t=b$.
$g$ is a constant, $t$ varies.
I have looked this up in textbooks and online and all I can find is the mean average of a function where $t$ is a ...
- 459
1
vote
0
answers
64
views
Is there a smart algorithm of finding the maximum of $X^{\top}a$ with $X$ and $a$ both belong to some compact convex set? [closed]
Suppose $X\in\mathcal{X}\subset R^k$ and $a\in\mathcal{A}\subset R^k$, where $\mathcal{X}$ and $\mathcal{A}$ are both compact convex set.
Is there a systematic way of finding the maximum of $X^{\top}a$...
- 2,956