I have a model class $M$. Each $\theta \in M$ defines a probability distribution $p(x,y | \theta)$ such that $x \in A$ and $y \in B$.
I am interested only in probability distributions of $\theta$ such that their support is contained in $A_0 \subseteq A$ and $B_0 \subseteq B$ -- other distributions that are not of this form are "malformed" (they define a distribution over too permissive space).
Now, I have a prior $\pi(\theta)$. This prior can give positive probability to "malformed" probability distributions.
But now I assume that I observe $x_1,\ldots,x_n$, where $x_i \in A_0$.
I also assume that $p(y | x,\theta)$ for any $\theta$ has a support in $B_0$, if $x \in A_0$.
If I do Bayesian inference with this prior $\pi$ and $x_1,\ldots,x_n$ to compute the posterior $p(\theta, y_1,\ldots,y_n | x_1,\ldots,x_n)$ -- is there any practical or theoretical problem here?
Supposedly, my prior can give positive probability to "bad" $\theta$, but it seems to me that since $x_1,\ldots,x_n$ are always from $A_0$ in practice, there aren't too many problems with the posterior either, because $y_1,\ldots,y_n$ are always going to be from $B_0$.
