50 votes

A generalization of the Law of Iterated Expectations

The way I understand conditional expectation and teach my students is the following: conditional expectation $E[Y|\sigma(X)]$ is a picture taken by a camera with resolution $\sigma(X)$ As mentioned ...
KevinKim's user avatar
  • 6,869
44 votes

Intuition for Conditional Expectation of $\sigma$-algebra

One way to think about conditional expectation is as a projection onto the $\sigma$-algebra $\mathscr{G}$. (from Wikimedia commons) This is actually rigorously true when talking about square-...
Chill2Macht's user avatar
  • 6,229
31 votes
Accepted

Why do we care more about test error than expected test error in Machine Learning?

Why do we care more about $\operatorname{Err}_{\mathcal{T}}$ than Err? I can only guess, but I think it is a reasonable guess. The former concerns the error for the training set we have right now. ...
Demetri Pananos's user avatar
22 votes
Accepted

Regular conditional distribution vs conditional distribution

Why do we need these two different concepts? Regular conditional distributions are useful because they allow us to generalize the elementary notions of conditional distribution where we consider ...
Ariel's user avatar
  • 2,467
17 votes
Accepted

Conditioning a variable on itself and some other variable

What may be tripping you up here is a common imprecision in notation, where people (myself included) will use the same symbol to denote both a random variable, and a particular assignment or ...
Ruben van Bergen's user avatar
17 votes
Accepted

What's the relationship between these two definitions of martingales?

Durrett's definition is the general correct definition of a martingale, while the Wikipedia's definition is at best a "restricted definition". The qualifier "with respect to $\mathcal{...
Zhanxiong's user avatar
  • 17.3k
16 votes
Accepted

OLS as approximation for non-linear function

WARNING: The results claimed in this post are of contested validity (by the writer himself. When the fog clears I will report back) Ok. It's a bit long to include the whole proof here, so I will just ...
Alecos Papadopoulos's user avatar
16 votes

Conditioning a variable on itself and some other variable

It is conditional expectation (not probability), and $E[X|X,Y]=X$ because $X$ is already given.
gunes's user avatar
  • 57k
16 votes

When mathematical statistics outsmarts probability theory

I do not find this any more surprising than saying that if $Y \sim \mathcal N(0,1)$ then $\mathbb E\left[\frac1Y\right]$ is undefined even though $\frac1Y$ has a distribution symmetric about $0$. So ...
Henry's user avatar
  • 38.8k
15 votes
Accepted

Calculating the expected value of truncated normal

Your formula implementation is wrong because, $$\phi\left(\frac{x-\mu}{\sigma}\right)=\frac{1}{\sqrt{2\pi}}e^{-\frac{1}{2}\left(\frac{x-\mu}{\sigma}\right)^2}\neq f_{X,\mu,\sigma}(x)=\frac{1}{\sqrt{2\...
gunes's user avatar
  • 57k
14 votes

When mathematical statistics outsmarts probability theory

While it is a pleasant remark, I do not find this occurrence that surprising or paradoxical, and this for several reasons: (i) $\mathbb E^X[0]=0$ remains true, where $\mathbb E^X[\cdot]$ denotes the ...
Xi'an's user avatar
  • 104k
13 votes
Accepted

If $X_1,\cdots,X_n \sim \mathcal{N}(\mu, 1)$ are IID, then compute $\mathbb{E}\left( X_1 \mid T \right)$, where $T = \sum_i X_i$

The idea's right--but there's a question of expressing it a little more rigorously. I will therefore focus on notation and on exposing the essence of the idea. Let's begin with the idea of ...
whuber's user avatar
  • 321k
12 votes
Accepted

Is $E[E(X|Y)|Z] =E[X|Y,Z]$ If so, how to prove?

Those two conditional expectations differ in general:$$\mathbb{E}[\mathbb{E}(X|Y)|Z] \ne\mathbb{E}[X|Y,Z]$$ As a matter of fact, strictly speaking, they do not even live in the same functional ...
Xi'an's user avatar
  • 104k
12 votes
Accepted

What is the derivation for "Partial Expectation"?

One way is to think about the conditional density. The density of $X|X>k$ is zero when $X\leq k$, so it's proportional to $f_X(x)I(X>k)$. The constant of proportionality is given by the fact ...
Thomas Lumley's user avatar
12 votes

Conditional Expectation as a function of X

If you take the expectation of $X$, it's not a function of $X$. You integrate $xf(x)$ over $x$, and so $x$ is gone, producing only a number. If you take the expectation of $X|Y=y$, it's not a function ...
det's user avatar
  • 263
11 votes

Mathematical definition of causality

We say $Y$ is not causally related to $X$ if $E(Y|X)$ does not depend on $X$, which implies it is equal to $E(Y)$. This is wrong. Causal relations are about functional/structural dependencies, not ...
Carlos Cinelli's user avatar
11 votes

Intuition for Conditional Expectation of $\sigma$-algebra

I am going to try to elaborate what William suggested. Let $\Omega$ be the sample space of tossing a coin twice. Define the ran. var. $\xi$ to be the num. of heads that occur in the experiment. ...
Nicolas Bourbaki's user avatar
11 votes

Why do we care more about test error than expected test error in Machine Learning?

+1 to Demetri Pananos's answer. It may well be that we apply the same model $f$ to two different training datasets $\mathcal{T}$ and $\mathcal{T}'$. And $\mathrm{Err}_{\mathcal{T}}$ may be quite ...
Stephan Kolassa's user avatar
11 votes
Accepted

Can we always write a random variable as conditional expectation plus independent error?

Suppose $$ Y=X^2+u $$ where $u|X\sim(0,X^2)$ has conditional heteroskedasticity. Then, $$ \epsilon=Y-E(Y|X)=X^2+u-E(Y|X)=u, $$ which has conditional mean zero but is not independent of $X$, as its ...
Christoph Hanck's user avatar
10 votes
Accepted

Conditional Mean in Linear Regression

It's important to be precise in these situations, and distinguish between the data model, and the data itself. One way to think about linear regression is that we hypothesize the following ...
Matthew Drury's user avatar
10 votes

Loss functions for regression proof

I would like to explain the way I understood it, explaining each and every step on the way. Assumptions: $g(x,t)$ is a function of x and t. $p(x,t)$ is a joint distribution over $x$ and $t$. Basic ...
Chayan Ghosh's user avatar
10 votes

Conditional expectation of uniform random variable given order statistics

Consider the case of an iid sample $X_1, X_2, \ldots, X_n$ from a Uniform$(0,1)$ distribution. Scaling these variables by $\theta$ and translating them by $\theta$ endows them with a Uniform$(\theta, ...
whuber's user avatar
  • 321k
10 votes

What‘s wrong with my proof of the Law of Total Variance?

The transition from the second to the third line does not follow. Since $\mathbb{E}(X) \neq \mathbb{E}(X|Y)$ you have: $$\mathbb{E}[ (X - \mathbb{E}(X))^2 | Y ] \neq \mathbb{E}[ (X - \mathbb{E}(X|Y)...
Ben's user avatar
  • 123k
10 votes
Accepted

Conditional Expectation as a function of X

Without getting very much into measure theory,consider the random vector $(X,Y)$ with density $f_{X,Y}(\cdot,\cdot)$ (wrt a dominating measure $\text d\mu(x,y)$) decomposed as $$f_{X,Y}(x,y)=f_Y(y)\...
Xi'an's user avatar
  • 104k
9 votes
Accepted

Conditional expectation of a truncated RV derivation, gumbel distribution (logistic difference)

Since the parameters $(\mu,\beta)$ of the Gumbel distribution are location and scale, respectively, the problem simplifies into computing $$\mathbb{E}[\epsilon_1|\epsilon_1+c>\epsilon_0]= \frac{\...
Xi'an's user avatar
  • 104k
9 votes
Accepted

Technical point about convergence with conditional expectation

Yes, $X_{n} \to 0$ almost surely. The argument I have is a little convoluted, so bear with me. First, consider the events $F_{k} = \bigcup_{n \geq k} \{ C_{n} > 2 \}$. By the almost sure ...
Jason's user avatar
  • 331
9 votes

Simulation to estimate a conditional expectation

Unless the conditional density varies rapidly with the conditioning event, a brute-force rejection sample works. Let's state the general problem. You wish to study the distribution of some function $...
whuber's user avatar
  • 321k
9 votes

Conditional expectation for doubly truncated bivariate normal distribution

To be explicit, write $f(x_1,x_2,\rho,\sigma_1,\sigma_2)$ for the bivariate normal density. From the analysis at https://stats.stackexchange.com/a/71303/919 it is clear that the normalizing integral ...
whuber's user avatar
  • 321k
8 votes
Accepted

Marginal independence does not imply joint independence

Your first question assumes that $X, Y$, and $Z$ are pairwise independent random variables and asks whether $$ X,Y, Z~~\text{pairwise independent} \implies (X,Y)~~\text{and}~~Z~~\text{independent}??\...
Dilip Sarwate's user avatar
8 votes
Accepted

E[X| X>Y] for independent X, Y ~ N(0,1)

Rewriting and using linearity of expectation, $$ E[X | X > Y] = E[X | X - Y > 0] = E[X - Y | X - Y > 0] + E[Y | X - Y > 0]. $$ Define now $X' = -X, Y' = -Y$. Then $$ E[Y | X - Y > 0] ...
Ami Tavory's user avatar
  • 4,590

Only top scored, non community-wiki answers of a minimum length are eligible