I often find the following notation in ML Papers, with $k$ as kernel:
$E_{x, y \sim p} [ k(x, y ) ]$
I am not familiar with it. Is it the expected value with $x$ and $y$ as a random variable with distribution $p$?
It's not a standard notation, but very common in papers. Especially papers including variational analysis. Although it's not entirely possible to say w/o seeing the context, your interpretation is possibly correct, i.e. the expected value is taken over the joint distribution of $X$ and $Y$ wrt the function $p_{X,Y}(x,y)$.
Normally, if there are no other alternative joint distributions associated with these variables, then the subscript doesn't clarify anything because $E[k(X,Y)]$ would mean the same thing since the expected value is always taken wrt to the joint distribution of the variables inside the parentheses.
It's also a common abuse of notation to denote the random variables with lowercase letters, but typically these are reserved for special values for those RVs, i.e. $E[k(x,y)]$ is typically not the same thing with $E[k(X,Y)]$ in more rigorous texts.