Suppose there is a function of two variables $f(X,\theta)$ which is convex with respect to $\theta$. Can I use the the Jensen's inequality in this case $E_{X}[f(x,\theta)]\geq f(E(x),\theta)$. The reason for my confusion is that the expectation is not with respect to the variable $\theta$ which $f$ is convex in.
1 Answer
$\begingroup$
$\endgroup$
Your function may depend on several variables, like $\theta$; but you're allowed to use Jensen's Inequality, when your function is convex with respect to $X$, as the definition states. How about when $\theta$ is known and fixed (say $\theta^*$)? We can plug in the value and obtain a function $g(X)=f(X,\theta=\theta^*)$. We're not guaranteed to have $g(X)$ convex, but as a special case of the inequality you wrote, we need to have $E_X[g(X)]\geq g(E[X])$, which is not guaranteed.
