20 votes
Accepted

Under the 0-1 loss function, the Bayesian estimator is the mode of the posterior distribution

You need to be a bit careful with this kind of problem because the definition of the zero-one loss function will depend on whether you are dealing with a discrete or continuous parameter. For a ...
Ben's user avatar
  • 125k
14 votes
Accepted

Finding the slope at different points in a sigmoid curve

Your question is very broad. There are many ways to do this, even without assuming a specific function. For the following I assume that you have a good reason to use the Gompertz model. First let's ...
Roland's user avatar
  • 6,601
13 votes
Accepted

Limit of Integration of continuous function

Clearly, the integral can be rewritten as $E[f(Y_n)]$, where $Y_n = \frac{1}{n}(X_1 + \cdots + X_n)$, and $X_1, \ldots, X_n \text{ i.i.d.} \sim U(0, 1)$. By (weak) law of large numbers, we have $Y_n \...
Zhanxiong's user avatar
  • 18.3k
10 votes

gradient versus partial derivatives

Gradient is the partial derivatives : $$\nabla f = \left(\frac{\partial f}{\partial x_1};\frac{\partial f}{\partial x_2};...;\frac{\partial f}{\partial x_n}\right)$$ Eg : $f=x^2y$ $$\nabla f =(2xy;...
Benoit Sanchez's user avatar
10 votes

How do you obtain the standard error for a slope at a given data point, for curvilinear regression?

The model is $$\mathbb{E}(y) = \beta_0 + \beta_1 x + \beta_2 x^2.$$ Adding a fixed (usually small) quantity $\delta x$ to $x$ and comparing gives the difference $$\eqalign{ \frac{\delta\,\mathbb{E}(...
whuber's user avatar
  • 322k
8 votes
Accepted

Why does Judea Pearl call his causal graphs Markovian?

He is referring to the Parental Markov Condition (see theorems 1.2.7 and 1.4.1 of Causality). Given a graph $G$, we say a distribution $P$ is Markov relative to $G$ if every variable is independent of ...
Carlos Cinelli's user avatar
6 votes
Accepted

Calculating t-SNE gradient (a mistake in the original t-SNE paper)

I just signed up for this forum due to your question :) Nice question! It shows someone is indeed trying to follow & derive the nitty gritty. Your question is totally valid, (28) is indeed ...
N. Wong's user avatar
  • 76
6 votes
Accepted

What is the second derivative of a B-spline?

This document gives (with a corrected typo) \begin{equation} \frac{\text{d}^{(n)}B_{i,j}(x)}{\text{d}x^{(n)}} = (j-1) \left( \frac{- \text{d}^{(n-1)} B_{i+1,j-1}(x) / \text{d}...
rhombidodecahedron's user avatar
5 votes

Why is optimisation solved with gradient descent rather than with an analytical solution?

The system of equations you get setting the derivatives equal to zero cannot generally be solved analytically. For instance suppose I want choose $10>x>0$ to minimize $xln(x)-\sqrt{x}$ (which ...
SecretlyAnEconomist's user avatar
5 votes

gradient versus partial derivatives

If a function $f$ takes the parameters $x_1, \ldots, x_n$, then the partial derivatives w.r.t. the $x_i$ determine the gradient: \begin{equation} \nabla f = \frac{\partial f}{\partial x_1 }\mathbf{e}...
Ami Tavory's user avatar
  • 4,590
5 votes
Accepted

Correlation between normal and log-normal variables

As is often the case, precisely formulating the question helped me work out the answer. My approach makes use of the marginal expectation of the bivariate normal: $$E_X(y) = E(X|Y=y) = \mu_x + \rho\...
Will Bradshaw's user avatar
5 votes

Applying Leibniz's integral rule to the Gaussian distribution's normalization condition

Leibniz' rule can be extended to infinite regions of integration with an extra condition on the function being integrated. Let's write out the basic form: $${d \over dy}\int_a^b f(x,y)dx = \int_a^b {...
jbowman's user avatar
  • 38.6k
4 votes
Accepted

Differentiating $ (y-X\beta)^T(y - X \beta) $ with respect to $\beta$

Let us assume that you are working in a setup where $y$ is $N \times 1$ and $X$ is $N \times K$ and $\beta$ is $K \times 1$. I prefer to define $e(\beta) := (y - X\beta)$ and similarly the $i$'th ...
Jesper for President's user avatar
4 votes
Accepted

Is the sample mean of the gradient the same as the gradient of the sample mean?

The derivative and expectation have the associative property (you can exchange the order) by Leibniz integral rule (computing the expectation is just some sort of integration) $$\frac{\partial}{\...
Sextus Empiricus's user avatar
4 votes
Accepted

Why is the formula for the density of a transformed random variable expressed in terms of the derivative of the inverse?

It seems that the heuristic described by @whuber in their answer to the linked problem can be modified slightly to yield the change of variables formula for the density in its more familiar form. ...
R Hahn's user avatar
  • 201
4 votes

Calculus for Statistics

Learn how to differentiate and integrate, and what each of these concepts mean when applied to multi variate functions. I mean Riemann integrals. This alone will get you a long way in statistics. It’s ...
4 votes
Accepted

Applying Leibniz's integral rule to the Gaussian distribution's normalization condition

Nice question! option 1 Derivatives are limits (of fractions), and moving limits inside an expectation often triggers an invocation of a theorem such as the monotone convergence theorem, Lebesgue's ...
Taylor's user avatar
  • 20.6k
3 votes

How does one compute a variational derivative?

I will carry out the steps in a way I hope is clear enough to indicate what assumptions must be made about $y,$ $p,$ and $\delta$ to justify the steps. When a function $y$ is a local minimum of a ...
whuber's user avatar
  • 322k
3 votes
Accepted

Derivative of expectation where the variable appears in the integration limit and in the integrand?

Consider any differentiable function $f:\mathbb{R}^2\to \mathbb{R}.$ It is a theorem that the derivative of $f$, written $Df,$ is given by its partial derivatives, $$Df(x,y) = \left(\frac{\partial f}...
whuber's user avatar
  • 322k
3 votes
Accepted

Expected Loss Calculation

Here are two methods, a nested numerical integration and a Monte Carlo integration, in R. The nested numerical integration has an outer integration that goes from 0 to 1 with respect to $\theta_B$ ...
jbowman's user avatar
  • 38.6k
3 votes

Simple Log-likelihood question

By rewriting $$\eqalign{ l(\lambda_1, \lambda_2) &= y_1\log(\lambda_1F_1) - \lambda_1 F_1 - \log(y_1!)+y_2\log(\lambda_2F_2)-\lambda_2F_2-\log(y_2!) \\ &= y_1\left(\log(\lambda_1)+\log(F_1)\...
whuber's user avatar
  • 322k
3 votes

Calculating t-SNE gradient (a mistake in the original t-SNE paper)

Let our cost function be $C=KL(P||Q)=\sum_k\sum_{l\neq k}p_{kl}\log(p_{kl})-p_{kl}\log(q_{kl})$, so that we are trying to find the gradient $\frac{\partial C}{\partial y_i}$. We will define some ...
Scout Jarman's user avatar
3 votes

Neural Networks: How to get the gradient vector for the xOr problem?

You have several mistakes. You're missing the minus sign due to $-\hat y_i$ term If you have $n$ samples, $x_1$ and $x_2$ should also have indices Derivative wrt $b_3$ can't be $0$ because it's ...
gunes's user avatar
  • 57.1k
3 votes
Accepted

derivation of coordinate ascent variational inference

I'll start with a quick derivation of the Euler-Lagrange formula, then I'll show how you can use it, then I'll show how it applies to your problem. Background Equation 22 is a functional -- a function ...
HappyDog's user avatar
  • 421
2 votes
Accepted

Help with derivation of Mean Field Variational Inference

The main issue is that the integrals involved are multivariate. A confusing thing about Bishop's notation is that, inside those integrals, $q_i$ should actually be $q_i(\mathbf{Z}_i)$. So we want to ...
PAM's user avatar
  • 311
2 votes
Accepted

Expectation / Summation inequality with multiple indices

Assuming the $I_i$'s are identically distributed,\begin{align*}\sum_{i\ge1}\mathbb P\left(I_i \ge i \right) &= \sum_{i\ge1}\sum_{j \ge i}\mathbb P\left(I_i = j \right)\\&= \sum_{\underbrace{j \...
Xi'an's user avatar
  • 105k
2 votes

Why does Judea Pearl call his causal graphs Markovian?

These graphs do satisfy the Markov property - once you condition on the parent node, from which the causal arrow comes, the variable is independent of earlier ancestors that causally affect that ...
Ben's user avatar
  • 125k
2 votes
Accepted

Symmetry between integrals including absolute value

In the left hand you have the integral of a function of $|v|$ (considering the quadratic term too), hence that function is symmetric on $v$'s domain. This comes simply from the fact that $f(|-v|) = f(|...
carlo's user avatar
  • 4,545
2 votes
Accepted

Simple Appplication of Law of Iterated Expectation

I believe, you can; we can think over $E[Y_i|T_i=1,D_i=1]$ first. By the law of iterated expectations, we have $$E[Y_i|T_i=1,D_i=1] = E[ E[Y_i|T_i=1,D_i=1,M_i] ] \\ =\int{E[Y_i|T_i=1,D_i=1,M_i=m]\ \ ...
gunes's user avatar
  • 57.1k
2 votes
Accepted

Derivation of M step for Gaussian mixture model

TL;DR, we have that $$\mu^*_k = \frac{\sum_{i=1}^n W_{ik}x_i}{\sum_{i=1}^n W_{ik}}$$ $$\Sigma^*_k = \frac{\sum_{i=1}^{n} W_{ik}(x_i -\mu^*_k)(x_i - \mu^*_k)'}{\sum_{i=1}^n W_{ik}}$$ In particular, ...
doubled's user avatar
  • 4,867

Only top scored, non community-wiki answers of a minimum length are eligible