If the sample space is $\{N(\theta,1)\vert \theta\in[0,1]\}$, then every point in that sample space is a distribution with a finite expectation (since they are normal). Consequently, for any random variable distributed according to an element of the sample space, that random variable will have a finite expectation, so there is zero probability of an element of this sample space having an undefined or infinite expectation, no matter the probability measure on the probability space, as one of the axioms of a measure (any measure, not just a probability measure) is that the empty set has measure zero.