Minimizing an Integral of function by derivation

Currently I am reading ESL. I am struggling to unterstand the step 2.15 to 2.16 in Chapter 2.4.

The authors mention that the stationary function can be found by differentiation. The commentary by Weatherwax and Epstein develops this step in more detail.

Why is $$\ \dfrac{d}{ d \beta} \int (y-x^T\beta)^2 Pr(dx, dy) = \int \dfrac{d}{ d \beta} (y-x^T\beta)^2 Pr(dx, dy)$$, that is differentiating the integral partially is equivalent to differentiating the function inside the integral? Any links or explanations are much appreciated.

For starters, I want to make sure that I understand correctly that when you say "deriving," you mean "differentiating".

What you are asking is called "differentiating under the equal sign" and is covered in basic calculus classes. If you still have your calculus textbook, you might want to do a quick review before going on in machine learning. I don't mean that condescendingly, just a suggestion.

The method is called the Leibnitz Integration Rule. You can read about it here with examples, and an interesting anecdote from Richard Feynmann. Working through the steps:

Notice that $$EPE$$ is a function of $$\beta$$ only because the dependence on $$x$$ and $$y$$ is being integrated out.

\begin{align} \frac{d EPE}{d \beta} & = \frac{d}{d \beta} \int (y - x^T\beta)^2Pr(dx,dy) \\ & = \int \frac{\partial}{\partial \beta} (y - x^T\beta)^2 Pr(dx,dy) ]] \\ & \text{using the chain rule...} \\ & = \int -2(y - x^T\beta)Pr(dx,dy) \\ & = -2 \int (y - x^T\beta)Pr(dx,dy) \end{align}

The partial derivative is needed inside the integral because we have not yet integrated out $$x$$ and $$y$$, so the function we are differentiating is still a function of those variables.

• My question is: why is $\ \dfrac{d}{ d \beta} \int (y-x^T\beta)^2 Pr(dx, dy) = \int \dfrac{d}{ d \beta} (y-x^T\beta)^2 Pr(dx, dy)$. Or expressed as functions $\ \dfrac{d}{ d \beta} F(f(\beta)) = F(\dfrac{d}{ d \beta} f(\beta))$ Edited question for more clarity. – Bor Is Aug 13 at 19:19
• It's called the Leibnitz Integration Rule. I've edited my answer with this and corrected a mistake. – abalter Aug 13 at 19:30