Questions tagged [calculus]
For statistical questions involving calculus. Please use also a more statistical tag. For purely mathemathical questions about the calculus, it is better to ask at math SE https://math.stackexchange.com/
105
questions
-3
votes
0
answers
16
views
How we can check that three variable function is increasing or decreasing ? Please tell me the method of this [closed]
[
3
votes
1
answer
86
views
What are some good calculus resources relevant for Machine learning researcher aspirant?
I am trying to self-taught myself on Calculus for machine learning and read the book by Spivak. But it is too rigorous and need a lot of time to finish it.
As far as I am concerned, Calculus is only ...
0
votes
0
answers
27
views
Calculate the output of a Neural Network
I have the following problem:
Here is my approach:
With the activation function: $F(x) = x^2 + 2x + 3$, we can calculate the activation of the two units of the second layer by: $a_1^2 = F(w_{13}\...
0
votes
0
answers
12
views
Calculating initial values of a Linear Regression Model
I have the following problem
Given the training data for a linear regresison problem as follow:
Input
Output
0
0
1
2
-1
-2
2
3
After the first iteration, the values of the two coefficients are ...
1
vote
0
answers
37
views
Deriving vectorized back propagation
I'm trying to derive vectorized backpropagation from mostly first principles, but I'm having
trouble marrying how
this
paper explains backpropagation with the derivative of a loss function with ...
- 11
1
vote
1
answer
63
views
What is the derivative of a set or a string? [closed]
Neural networks operate on numbers, and it's well-known what the derivative of numeric functions are, as well as what the derivative of matrix functions are.
What about functions that operate on maps ...
- 111
0
votes
0
answers
29
views
Optimising Box-Cox lambda analytically
I'm taking a university course in statistics where the Box-Cox transform is being discussed. As I understand it, we assume that there is some $\lambda$ that makes the sample normally distributed after ...
- 101
0
votes
0
answers
9
views
Neural Network training as non stationary stateless continuous reinforcement learning problem
Say I have a neural network denoted as f(\theta), and we want to optimize $\theta$.
What I thought is that $\theta$ can be seen as an action sampled from a ...
6
votes
2
answers
400
views
Applying Leibniz's integral rule to the Gaussian distribution's normalization condition
I'm working on problem 1.8 of Bishop's Pattern Recognition and Machine Learning and am having a hard time understanding one of the technical details in a solution that I found online. Specifically, ...
- 63
0
votes
0
answers
22
views
Bayes' Theorem and Nested Normal Distribution
Let's say I have a variable, x, with a normal distribution. For the sake of example, let's say it has a mean of 1, and a standard deviation of 2.
Afterwards, let's say I can predict the value of the ...
3
votes
1
answer
204
views
Gradient and Hessian of loss function
I'm trying to clear up the calculation of the gradient and Hessian of a loss function in an article that I am currently reading. The loss function is given by
$$\ell(\beta)=\sum_{i=1}^{N} e^{-y_{i}{{x}...
- 95
1
vote
1
answer
185
views
I am not able to understand how did the elementwise multiplication came into the picture of backpropagation in neural networks
I have understood the backpropagation algorithm along with the chain rule well enough that I can derive it on my own, but I don't understand where the elementwise multiplication came from and how does ...
0
votes
1
answer
45
views
Deriving the normal equations' coefficients
Suppose we use the least squares criterion to fit a linear model for the following dataset: $(x_1,y_1),...,(x_m,y_m)\in R \times R$, by solving the following optimisation problem:
$$(a^*,b^*) = \text{...
- 157
0
votes
1
answer
39
views
What is the relationship between the derivative of a map and its image density? [duplicate]
Preliminaries
Suppose we have a random variable $X$ with density $f$ and a suitably smooth function $g: \mathbb{R} \mapsto \mathbb{R}$. The random variable $Y = g(X)$ also has a density function $h$.
...
- 6,135
2
votes
1
answer
71
views
How can make sure our deep neural network is differentiable
When we have a deep neural network, according to how much complicated that neural network is, how we can make sure that in each layer we can calculate the derivatives?( Is that differentiable or not). ...
- 159
0
votes
0
answers
18
views
Improper integrals of symmetric functions [duplicate]
First time poster here, so I apologize for any formatting errors. I recently came across the improper integral ∫xdx from -∞ to ∞ and have had a hard time understanding why it isn't zero. My approach ...
1
vote
0
answers
52
views
PRML Book: Calculus of Variance
I am reading through Pattern Recognition and Machine Learning (PRML) Appendix D (page 705).
Here is my question: what does the term $O(\epsilon ^ 2)$ in equation (D.1) and (D.2) stand for?
- 11
1
vote
1
answer
1k
views
Gradient of a multivariate function numpy
I'm trying to calculate the gradient of multivariate function g using NumPy.
g = lambda w: -np.sin(np.pi*np.sum(w**2)) + np.log(np.sum(w**2))
...
- 13
1
vote
1
answer
540
views
Finding E(XY) for joint probability density
$Joint \:probability\;f(x,y) = 2/3 \:for\: 0 < x < 1, 0 < y < 2, x < y, and\: 0\: otherwise $
$E(XY)=\int_{0}^{1}\int_{x}^{2} \frac{2}{3}xy \:dy \:dx = \frac{7}{12} - (1)$
$E(XY)=\...
- 45
0
votes
1
answer
40
views
BLUE from calculus
Let $p'\beta$ be an estimable LPF. Suppose
that $l'y$ is the candidate which must satisfy the unbiasedness
condition and the minimum-variance condition. Formulate this as an optimization problem with ...
- 164
2
votes
2
answers
393
views
Calculus for Statistics
If one were to learn calculus solely for the purpose of learning statistics, what should he focus on? If this is a ridiculous question and the honest answer is “All of it,” that is of course an ...
0
votes
0
answers
25
views
How to approximate the expression to $\sum x_i$
How to approximate the expression on the left hand side to $\sum_{i=1}^Nx_i$ as $n\to \infty$
$$ \frac{\sum\limits_{i=1}^{N}x_i^2}{n-2\frac{\sum\limits_{i=1}^{N}x_i}{N}}
\left(\sqrt{1+\frac{Nn\left(...
- 11
3
votes
0
answers
120
views
When will $\mathbb{E}[g(S_n/n)]$ exist given $\mathbb{E}[g(X_1)]$ exists?
Suppose $X_1, X_2,..., X_n$ are i.i.d. random variables with distribution $\pi$ on some probability space. Let $g$ be a measurable function such that $\mathbb E_\pi[g(X_1)]<\infty$. I am curious ...
- 111
0
votes
0
answers
23
views
Forecasting Peak/Global Maximum from Raw Data
I'm trying to see what methods there are to predict when the data will peak based on raw values, along with how to accomplish it in R. Here's what you can assume...
The data has a start and end point....
2
votes
1
answer
119
views
Pattern Recognition and ML Exercise 1.4
I am studying "Pattern Recognition and Machine Learning" by Christopher Bishop and I'm trying to understand his solution in the solution manual to exercise 1.4.
The problem statement for ...
- 157
2
votes
1
answer
80
views
Differentiating a Vector and a Matrix w.r.t. a Vector [Matrix Calculus]
I am studying matrix calculus for linear regression and machine learning and I would like to know exactly if the following calculations are correct:
Let $y=\sin(x+yz)$ and $r=\begin{bmatrix}x\\y\\z\...
- 157
3
votes
1
answer
442
views
Neural Networks: How to get the gradient vector for the xOr problem?
I'm reading about neural networks, but the material I find is sometimes very abstract or just copies of something. Well, when considering the $xOr$ problem, I have a network in the following structure
...
- 110
2
votes
1
answer
47
views
Elements of Statistical Learning Integral Notation
In equation 2.9 and 2.10 on page 18 of ESL we have
$$E(Y - f(X))^2 = \int [y - f(x)]^2 Pr(dx, dy)$$
However this notation confuses me. I'm rather expecting
$$E(Y - f(X))^2 = \int [y - f(x)]^2 Pr(x, y)...
- 21
0
votes
0
answers
201
views
Multicollinearity in quadratic (polynomial) regression function [duplicate]
Multicollinearity problem could arise when we add quadratic variable in regression like this:
So, one of the possible solutions to eliminate the problem is to add centered variables:
This was ...
1
vote
0
answers
19
views
Directional derivative in regression (coefficients, after all, are partial derivatives)
The coefficients in a (let's stick with linear for now) regression are the partial derivatives. A regression equation is a function of several variables, so all of the multivariable calculus tricks ...
- 46.8k
0
votes
0
answers
83
views
Is this statement about the sum of quantiles correct?
Let $X$ and $Y$ be continuous random variables both having some density, not identically distributed but independent.
Imagine I'm interested in the quantile $q_{X+Y}(\alpha)$ for some $\alpha \in (0,1)...
- 11
3
votes
1
answer
224
views
$f$ is a decreasing function whose integral converges. Does $\lim_{x \to \infty}xf(x) = 0$?
My finals are over and I cannot help but ruminate over this particular problem. Could anyone help prove this?
Suppose $f$ is a continuous decreasing function on $[0,\infty)$ and $\int_0^\infty f(t)\, ...
- 49
5
votes
2
answers
292
views
Why is the formula for the density of a transformed random variable expressed in terms of the derivative of the inverse?
In this very nice answer, the intuitive explanation of the formula for the density of a transformed random variable, $Y = g(X)$, leads naturally to an expression like
$$f_Y(y) = \frac{f_X(g^{-1}(y))}{...
- 201
8
votes
1
answer
551
views
Limit of Integration of continuous function
How to evaluate the following limit-
$$\lim_{n \to \infty} \int_0^1 \int_0^1\cdots\int_0^1 f \bigg(\frac{x_1 + x_2 + \cdots + x_n}{n} \bigg) dx_1 dx_2....dx_n$$.
Here $f()$ is a continuous function $f:...
- 127
3
votes
2
answers
64
views
Deriving OLS estimator
In my course on linear models we derived the OLS estimator by minimizing the residuals $F(\phi) = (Y-X\phi)'(Y-X\phi)$. However there is one step that I do not understand: to find the minimum over all ...
- 41
0
votes
0
answers
63
views
Rewriting the probability density function as a probability function
Letting $dt$ be an infinitesimal interval, what is the argument to that
$$f(t | H_{t_n})\;dt = P (t \in [t,t+dt] | H_{t_n}),$$
where $H_{t_n}$ denotes the history of the previous points before $t$? I ...
- 21
1
vote
0
answers
67
views
Calculus in Moment Generating Function
On page 156 of the Statistics textbook, "Mathematical Statistics and Data Analysis" by John A. Rice, I came up with two questions on the section about Moment Generating Functions:
1.
Why ...
- 71
2
votes
1
answer
121
views
Is the sample mean of the gradient the same as the gradient of the sample mean?
By the law of large numbers, given a continuous random vector $\mathbf{x}$, then:
$$
\mathbb{E}[\mathbf{x}] \approx \frac{1}{N} \sum_{i=1}^{N} \mathbf{x}_i
$$
Where $\mathbf{x}_1,\mathbf{x}_2,...,\...
- 3,698
0
votes
0
answers
81
views
Algebra: When calculating the variance of Zero-inflated Poisson dist
I am deriving the variance of zero-inflated Poisson distribution, whose PMF is
$$
P(X=k) = \begin{cases} \pi + (1-\pi)e^{-\lambda} \; , \; if \; k=0 \\ (1- \pi) e^{-\lambda} \frac{\lambda^k}{k!} \; , \...
- 117
1
vote
0
answers
274
views
How to differentiate the hinge loss?
I'm asked to differentiate the following hinge loss term.
$$ \dfrac{1}{n}\sum _{\left( x_{i},y_{1}\right) \in S}\sum _{j'=1}L\left( w^{j'};\left( x_{i},y_{i}\right) \right) $$
where
$$
L\left( w^{j'};\...
- 131
0
votes
1
answer
62
views
1
vote
2
answers
45
views
Backpropogation Derivatives
I've been working on trying to understand the backpropogation algorithm and the calculus behind it, and in my work I have stumbled across a sort of odd situation. I am just practicing on a 1 input, 1 ...
- 31
1
vote
1
answer
16
views
What is the mean average of $y=kg^t$ from $t=a$ to $t=b$ [closed]
Mean average of $y$ in $y=kg^t$ from $t=a$ to $t=b$.
$g$ is a constant, $t$ varies.
I have looked this up in textbooks and online and all I can find is the mean average of a function where $t$ is a ...
- 459
1
vote
0
answers
64
views
Is there a smart algorithm of finding the maximum of $X^{\top}a$ with $X$ and $a$ both belong to some compact convex set? [closed]
Suppose $X\in\mathcal{X}\subset R^k$ and $a\in\mathcal{A}\subset R^k$, where $\mathcal{X}$ and $\mathcal{A}$ are both compact convex set.
Is there a systematic way of finding the maximum of $X^{\top}a$...
- 2,204
2
votes
1
answer
85
views
Book recommendations needed - building foundational knowledge for ISL - Introduction to Statistical Learning (by Gareth James)
I'm trying to build a data science base from scratch. I started a book called Introduction to Statistical Learning by Gareth James and found that there are many mathematical & statistical concepts ...
3
votes
1
answer
223
views
Differentiating $ (y-X\beta)^T(y - X \beta) $ with respect to $\beta$
How do I differentiate
$$ (y-X\beta)^T(y - X \beta) $$
with respect to $\beta$. The result I saw was
$$X^T(y - X\beta)$$
- 309
0
votes
0
answers
92
views
Backpropagation through time for stacked RNNs
I was able to find the partial derivative of the cost function with respects to a single variable without much difficulty. However, this requires propagating backwards through the network for each ...
- 123
1
vote
0
answers
68
views
Evaluation of Limit involved in the proof of Asymptotic Unbiasedness
We know that $S^{2}$ is an unbiased estimator of $\sigma^{2}$ and $S$ is a biased estimator of $\sigma$. But if $n\rightarrow\infty$, then $S$ is an asymptotically unbiased estimator of $\sigma$. I ...
- 11
6
votes
1
answer
3k
views
Derivation of M step for Gaussian mixture model
Summary
So to summarize my question, how can I take
\begin{align}
= \sum_{i=1}^{n}W_{i1} \left(log (1-\sum_{j=2}^{K}\pi_j) -\frac{1}{2} log(|\Sigma_1|) -\frac{d}{2} log(2\pi) -\frac{1}{2}(x_i-\mu_1)^{...
- 849
1
vote
0
answers
26
views
Question about Functions of Several Random Variables
In the Mathematical Statistics and Data Analysis by John Rice, it states that for random variables $U,V$ which are functions of random variables $X,Y$, we have:
We know that $$f_{UV}(u,v) = f_{XY}(h_1(...
- 131