regression functions Could somebody explain to me in simple terms what regression functions are?
In bishop's pattern recognition it says 
"Consider a pair of random variables θ and z governed by a joint distribution p(z, θ). The conditional expectation of z given θ defines a deterministic function
$$f(\theta )=\int zp(z|\theta)dz $$
Functions defined in this way are called regression functions."
can someone please expand on this?
 A: Basics
Say you know $\theta = 3$. You want this to estimate $z$. You use the regression function to estimate $z$ as 
$$
E[z|\theta=3] = f(3) = \int z p(z|\theta=3)dz.
$$
Say the above equals five. Okay, then you guess that $z$ is 5, assuming you know $\theta =3$.
General
A regression function gives you a number for $z$ for any input $\theta$ you have. 
Optimality
Well there are a lot of functions that can take $\theta$ and give you a guess for $z$. Why do we use the conditional expectation? Well because it's the "best" or "closest" in a sense. That's what DeltaIV is referring to. 
Say you have your conditional expectation function $f(\theta)$. Is it better than any other function $g(\theta)$? Well each of them will be off by $Z-f(\theta)$ and $Z-g(\theta)$, respectively. 
We want these numbers to be positive all the time, though, so we square them. That means we're looking at $(Z-f(\theta))^2$ and $(Z-g(\theta))^2$ now. 
One more problem: these numbers are random. So we take the expectation of these random distances. $f$ is better than any other $g$ in the sense that
$$
E[(Z-f(\theta))^2] \le E[(Z-g(\theta)^2].
$$
For a proof, you can see the link DeltaIV posted.
