Questions tagged [calculus]

For statistical questions involving calculus. Please use also a more statistical tag. For purely mathemathical questions about the calculus, it is better to ask at math SE https://math.stackexchange.com/

Filter by
Sorted by
Tagged with
1 vote
1 answer
86 views

Gradient of a multivariate function numpy

I'm trying to calculate the gradient of multivariate function g using NumPy. g = lambda w: -np.sin(np.pi*np.sum(w**2)) + np.log(np.sum(w**2)) ...
user avatar
1 vote
1 answer
62 views

Finding E(XY) for joint probability density

$Joint \:probability\;f(x,y) = 2/3 \:for\: 0 < x < 1, 0 < y < 2, x < y, and\: 0\: otherwise $ $E(XY)=\int_{0}^{1}\int_{x}^{2} \frac{2}{3}xy \:dy \:dx = \frac{7}{12} - (1)$ $E(XY)=\...
user avatar
0 votes
1 answer
37 views

BLUE from calculus

Let $p'\beta$ be an estimable LPF. Suppose that $l'y$ is the candidate which must satisfy the unbiasedness condition and the minimum-variance condition. Formulate this as an optimization problem with ...
user avatar
2 votes
2 answers
147 views

Calculus for Statistics

If one were to learn calculus solely for the purpose of learning statistics, what should he focus on? If this is a ridiculous question and the honest answer is “All of it,” that is of course an ...
0 votes
0 answers
24 views

How to approximate the expression to $\sum x_i$

How to approximate the expression on the left hand side to $\sum_{i=1}^Nx_i$ as $n\to \infty$ $$ \frac{\sum\limits_{i=1}^{N}x_i^2}{n-2\frac{\sum\limits_{i=1}^{N}x_i}{N}} \left(\sqrt{1+\frac{Nn\left(...
user avatar
  • 11
0 votes
0 answers
21 views

Plot Partial derivatives from Linear Regression

I am working on the link between Calculus and solutions for Linear Regression. Let's suppose a linear regression for a given individual. $$ y_i = \beta_0 + \beta_1x_i +\epsilon_i; \epsilon\sim N(0, \...
user avatar
0 votes
0 answers
10 views

Differentiating a function with respect to a matrix [duplicate]

I'm new to matrix calculus and I'm trying to find the formulas for matrix differentiation. e.g. $\frac{\partial f}{\partial z}$ = zzx where z is a KxK matrix, and x is a vector in K I found a few ...
user avatar
0 votes
0 answers
61 views

Maximal/Minimal Difference Between Output Probabilities Under the 2PL Model

The output probability of the IRT 2PL model is defined as $$P(Y=1|\theta,a,b)=\frac{e^{a(\theta-b)}}{1+e^{a(\theta-b)}}$$ for $\theta,b\in\mathbb{R}$ and $a>0$. $\theta$ is our independent variable ...
user avatar
  • 2,351
3 votes
0 answers
117 views

When will $\mathbb{E}[g(S_n/n)]$ exist given $\mathbb{E}[g(X_1)]$ exists?

Suppose $X_1, X_2,..., X_n$ are i.i.d. random variables with distribution $\pi$ on some probability space. Let $g$ be a measurable function such that $\mathbb E_\pi[g(X_1)]<\infty$. I am curious ...
user avatar
  • 111
0 votes
0 answers
21 views

Forecasting Peak/Global Maximum from Raw Data

I'm trying to see what methods there are to predict when the data will peak based on raw values, along with how to accomplish it in R. Here's what you can assume... The data has a start and end point....
user avatar
2 votes
1 answer
61 views

Pattern Recognition and ML Exercise 1.4

I am studying "Pattern Recognition and Machine Learning" by Christopher Bishop and I'm trying to understand his solution in the solution manual to exercise 1.4. The problem statement for ...
user avatar
2 votes
1 answer
38 views

Differentiating a Vector and a Matrix w.r.t. a Vector [Matrix Calculus]

I am studying matrix calculus for linear regression and machine learning and I would like to know exactly if the following calculations are correct: Let $y=\sin(x+yz)$ and $r=\begin{bmatrix}x\\y\\z\...
user avatar
3 votes
1 answer
119 views

Neural Networks: How to get the gradient vector for the xOr problem?

I'm reading about neural networks, but the material I find is sometimes very abstract or just copies of something. Well, when considering the $xOr$ problem, I have a network in the following structure ...
user avatar
  • 98
0 votes
0 answers
22 views

How to find mean of multivariate normal distribution when holding a variable constant? [duplicate]

I wanted to know if there is a way to calculate the mean of a multivariate normal distribution when a certain variable is held constant. For example, if I had a continuous bivariate normal ...
user avatar
2 votes
1 answer
42 views

Elements of Statistical Learning Integral Notation

In equation 2.9 and 2.10 on page 18 of ESL we have $$E(Y - f(X))^2 = \int [y - f(x)]^2 Pr(dx, dy)$$ However this notation confuses me. I'm rather expecting $$E(Y - f(X))^2 = \int [y - f(x)]^2 Pr(x, y)...
user avatar
0 votes
0 answers
22 views

What are the differences between different volatility models?

I would like to understand the differences between different volatility models like in simple terms and what are pros and cons over the other models Local volatility Model(Dupire) Heston Model SABR ...
user avatar
0 votes
0 answers
63 views

Multicollinearity in quadratic (polynomial) regression function [duplicate]

Multicollinearity problem could arise when we add quadratic variable in regression like this: So, one of the possible solutions to eliminate the problem is to add centered variables: This was ...
user avatar
1 vote
0 answers
16 views

Directional derivative in regression (coefficients, after all, are partial derivatives)

The coefficients in a (let's stick with linear for now) regression are the partial derivatives. A regression equation is a function of several variables, so all of the multivariable calculus tricks ...
user avatar
  • 31.1k
0 votes
0 answers
19 views

batch renormalization questions

I was going in details through paper about batch renormalization (arxiv link). I don't quite understand two things there. Maybe there is anyone who faced similar issues / knows the answer and could ...
user avatar
0 votes
0 answers
20 views

Improved/novel/useful Hazard Rate Analysis Based on Approximating the Derivative of Log Survival Ratio?

Those acquainted with the Hazard Function Analysis are likely familiar with the literature which notes that a hazard ratio can be expressed as the derivative of the log of the survival function (which ...
user avatar
  • 1,810
0 votes
0 answers
32 views

Is this statement about the sum of quantiles correct?

Let $X$ and $Y$ be continuous random variables both having some density, not identically distributed but independent. Imagine I'm interested in the quantile $q_{X+Y}(\alpha)$ for some $\alpha \in (0,1)...
user avatar
  • 11
3 votes
1 answer
119 views

$f$ is a decreasing function whose integral converges. Does $\lim_{x \to \infty}xf(x) = 0$?

My finals are over and I cannot help but ruminate over this particular problem. Could anyone help prove this? Suppose $f$ is a continuous decreasing function on $[0,\infty)$ and $\int_0^\infty f(t)\, ...
user avatar
3 votes
2 answers
247 views

Why is the formula for the density of a transformed random variable expressed in terms of the derivative of the inverse?

In this very nice answer, the intuitive explanation of the formula for the density of a transformed random variable, $Y = g(X)$, leads naturally to an expression like $$f_Y(y) = \frac{f_X(g^{-1}(y))}{...
user avatar
  • 181
8 votes
1 answer
507 views

Limit of Integration of continuous function

How to evaluate the following limit- $$\lim_{n \to \infty} \int_0^1 \int_0^1\cdots\int_0^1 f \bigg(\frac{x_1 + x_2 + \cdots + x_n}{n} \bigg) dx_1 dx_2....dx_n$$. Here $f()$ is a continuous function $f:...
user avatar
  • 127
2 votes
1 answer
28 views

Deriving OLS estimator

In my course on linear models we derived the OLS estimator by minimizing the residuals $F(\phi) = (Y-X\phi)'(Y-X\phi)$. However there is one step that I do not understand: to find the minimum over all ...
user avatar
  • 31
0 votes
0 answers
48 views

Rewriting the probability density function as a probability function

Letting $dt$ be an infinitesimal interval, what is the argument to that $$f(t | H_{t_n})\;dt = P (t \in [t,t+dt] | H_{t_n}),$$ where $H_{t_n}$ denotes the history of the previous points before $t$? I ...
user avatar
  • 21
1 vote
0 answers
43 views

Calculus in Moment Generating Function

On page 156 of the Statistics textbook, "Mathematical Statistics and Data Analysis" by John A. Rice, I came up with two questions on the section about Moment Generating Functions: 1. Why ...
user avatar
2 votes
1 answer
73 views

Is the sample mean of the gradient the same as the gradient of the sample mean?

By the law of large numbers, given a continuous random vector $\mathbf{x}$, then: $$ \mathbb{E}[\mathbf{x}] \approx \frac{1}{N} \sum_{i=1}^{N} \mathbf{x}_i $$ Where $\mathbf{x}_1,\mathbf{x}_2,...,\...
user avatar
  • 2,956
0 votes
0 answers
40 views

Algebra: When calculating the variance of Zero-inflated Poisson dist

I am deriving the variance of zero-inflated Poisson distribution, whose PMF is $$ P(X=k) = \begin{cases} \pi + (1-\pi)e^{-\lambda} \; , \; if \; k=0 \\ (1- \pi) e^{-\lambda} \frac{\lambda^k}{k!} \; , \...
user avatar
1 vote
0 answers
215 views

How to differentiate the hinge loss?

I'm asked to differentiate the following hinge loss term. $$ \dfrac{1}{n}\sum _{\left( x_{i},y_{1}\right) \in S}\sum _{j'=1}L\left( w^{j'};\left( x_{i},y_{i}\right) \right) $$ where $$ L\left( w^{j'};\...
user avatar
0 votes
1 answer
29 views

Approximate / Standardize value in certain range

I have table with numeric values like ...
user avatar
  • 11
1 vote
2 answers
44 views

Backpropogation Derivatives

I've been working on trying to understand the backpropogation algorithm and the calculus behind it, and in my work I have stumbled across a sort of odd situation. I am just practicing on a 1 input, 1 ...
user avatar
  • 31
1 vote
1 answer
16 views

What is the mean average of $y=kg^t$ from $t=a$ to $t=b$ [closed]

Mean average of $y$ in $y=kg^t$ from $t=a$ to $t=b$. $g$ is a constant, $t$ varies. I have looked this up in textbooks and online and all I can find is the mean average of a function where $t$ is a ...
user avatar
1 vote
0 answers
61 views

Is there a smart algorithm of finding the maximum of $X^{\top}a$ with $X$ and $a$ both belong to some compact convex set? [closed]

Suppose $X\in\mathcal{X}\subset R^k$ and $a\in\mathcal{A}\subset R^k$, where $\mathcal{X}$ and $\mathcal{A}$ are both compact convex set. Is there a systematic way of finding the maximum of $X^{\top}a$...
user avatar
  • 1,682
2 votes
1 answer
62 views

Book recommendations needed - building foundational knowledge for ISL - Introduction to Statistical Learning (by Gareth James)

I'm trying to build a data science base from scratch. I started a book called Introduction to Statistical Learning by Gareth James and found that there are many mathematical & statistical concepts ...
3 votes
1 answer
123 views

Differentiating $ (y-X\beta)^T(y - X \beta) $ with respect to $\beta$

How do I differentiate $$ (y-X\beta)^T(y - X \beta) $$ with respect to $\beta$. The result I saw was $$X^T(y - X\beta)$$
user avatar
  • 299
0 votes
0 answers
72 views

Backpropagation through time for stacked RNNs

I was able to find the partial derivative of the cost function with respects to a single variable without much difficulty. However, this requires propagating backwards through the network for each ...
user avatar
  • 123
1 vote
0 answers
60 views

Evaluation of Limit involved in the proof of Asymptotic Unbiasedness

We know that $S^{2}$ is an unbiased estimator of $\sigma^{2}$ and $S$ is a biased estimator of $\sigma$. But if $n\rightarrow\infty$, then $S$ is an asymptotically unbiased estimator of $\sigma$. I ...
user avatar
  • 11
6 votes
1 answer
2k views

Derivation of M step for Gaussian mixture model

Summary So to summarize my question, how can I take \begin{align} = \sum_{i=1}^{n}W_{i1} \left(log (1-\sum_{j=2}^{K}\pi_j) -\frac{1}{2} log(|\Sigma_1|) -\frac{d}{2} log(2\pi) -\frac{1}{2}(x_i-\mu_1)^{...
user avatar
1 vote
0 answers
25 views

Question about Functions of Several Random Variables

In the Mathematical Statistics and Data Analysis by John Rice, it states that for random variables $U,V$ which are functions of random variables $X,Y$, we have: We know that $$f_{UV}(u,v) = f_{XY}(h_1(...
user avatar
  • 131
7 votes
1 answer
295 views

Correlation between normal and log-normal variables

(This is not a homework question.) Let $(X_1 \sim N(\mu_1,\sigma_1), X_2 \sim N(\mu_2, \sigma_2))$ be a bivariate normal random variable with the correlation between $X_1$ and $X_2$ given by $\rho$. ...
user avatar
0 votes
2 answers
69 views

Maths for deeply understanding backpropgation

I have been trying to develop a deeper understanding of Neural Networks so I can understand the libraries such as tensorflow and others. I have had good success with pereceptron models, and have a ...
user avatar
  • 31
3 votes
2 answers
84 views

How does one compute a variational derivative?

The expected regression loss is given as:$$E[L]=\int\int \{y(\mathbf x)-t\}^2 p(\mathbf x,t)d\mathbf xdt$$ To minimise the expected loss,Euler Lagrange equation is used which goes like this in the ...
user avatar
0 votes
1 answer
520 views

Expected Predicted Error (EPE) with L1 loss

In Element of Statistical learning it is saying on page 20, equation 2.18. That using the L1 norm instead of the usual L2 norm leads to an $f(X)$ optimising the EPE being the median instead of the ...
user avatar
  • 141
2 votes
0 answers
199 views

Why does the dimension of gradient and Hessian matrix not conform for this function?

The function is $f(\mathbf{x}) = e^{-\frac{1}{2}\mathbf{x^TAx}}$, where $\mathbf{A}$ is a square symmetric matrix, and $\mathbf{x}$ is an n-vector. What I found were: $$ \begin{align*} \nabla f ...
user avatar
0 votes
0 answers
173 views

What is the first order derivative of linear regression's cost function using matrix calculus?

For linear regression's cost function J(b), where X is a n*m matrix, b is a m*1 vector and y is n*1 vector: First order derivative with respect to vector b (coefficients) is shown to be Using the ...
user avatar
1 vote
0 answers
148 views

On the convergence of infinite sum of a hypergeometric function resulted from a nested sum

I am interested in finding the CDF of the sum $U=\sum_{i=0}^N U_i$ where: $$F_{U_i}(x)=\sum_{n=0}^\infty \frac {2(-1)^nR^{k\alpha+n\alpha}x^{k+n}}{\Gamma(k)n!\theta^{k+n}(k+n)(k\alpha+n\alpha+2)}$$ ...
user avatar
  • 97
1 vote
0 answers
62 views

How to approximate expectation and variance of an integral from a discrete Time series financial dataset?

I have discrete time series financial data, with time($u$), price($S$) and someVariable($q$) which looks something like this. ...
user avatar
0 votes
0 answers
43 views

Calculus with quadratic B-spline

After fitting a quadratic B-spline in R with the cobs package : Rbs <- cobs(x,y, constraint= "decrease", pointwise = con) I would like to do some calculus on ...
user avatar
  • 1,388
1 vote
0 answers
127 views

what is the relation between change of variables in probability and calculus?

I want to know if the change of variables formula in probability is a special case of the change of variables in calculus, or something different. Trying to think for myself, first write the CoV for ...
user avatar