I'm trying to figure out some convergence statements on an MCMC example.
The setup is: I'm generating data samples as observations from a (known) deterministic parameter, say $s$ (using a forward equation and a random pertubation term). The data are input to my MCMC and I try to get estimates of the parameter $s$. I know it's not sensible to use MCMC to estimate a known deterministic parameter but this is just a toy example. So I assume I don't know much about $s$, give it an uninformative prior and set up a likelihood according to my problem.
Then I run my MCMC and I want to quantify how good my solutions are. So the assessements I want to make are along the lines of "I have this and that assumptions on $s$, I would expect e.g. the posterior mean be of an error magnitude of [whatever]... and my simulations meet these expectations yes or no..." How do I go about this?
Is it possible to make such statements without an analytical MCMC target/posterior? (i.e. in my example I know the "true" distribution of $s$ (deterministic in this case), but I'm not putting this into the MCMC algorithm - the posterior for $s$ I get from likelihood and prior are something I don't have an analytical expression for) So far I figured that for error assessement of the posterior mean one could examine
$| s - \int_{\mathbb{R}} s \ p(s | data) \ ds | \leq \ldots$
but I'm not getting anywhere here if I don't know the posterior p(s | data)...
I would appreciate any hints.
Edit: Thanks for the help! My original question was obviously a little vague and also contains more than one question. In think the numerical error by MCMC is answered very clearly by Greenparkers post. I have more questions on the model error of the inverse problem itself but I think that's a different issue and so I'll mark this question answered and maybe make new ones for the other stuff.