i have just started out in MCMC and I am not sure if I fully understand the concepts of MCMC with respect to the above terms. Let me try to explain that in my own words and include some thoughts / questions that I still have on it.
My understanding for MCMC (in Bayesian inference) is that it help us identify the final distribution of the samples (also known as the posterior distribution). And we can then use it to determine the sample's mean, range etc.
Example: We wanted to find out the mean test score of students. We are given 2 samples (i.e. the test scores of 2 students only and let's say the score is 65 and 75). We conduct the MCMC.
So to implement MCMC, we need the following "ingredients" - Target distribution: This is actually the posterior distribution that we actually want the MCMC to sample from. In other words, we need to determine a distribution of the mean score.
Q1: If our goal is to determine the final distribution of the samples and yet we are required to specify the target distribution that the MCMC is working on, isnt that like forcing the solution to a distribution that we already have in mind?
Q2: What if we have no idea what this distribution is supposed to be, including the mean and standard deviation. What should we do to proceed from here?
- Proposal distribution: I see this as a distribution to determine the step / noise to be added on the parameter when we sample from the target distribution. Let say we had a starting test score at 68. We use the proposal distribution to generate a random noise (e.g. 3) and this is added to the test score for testing. So the next state is 78.
Q3: Am i right in my interpretation here?
- Likelihood function: As far I understand so far, this is used in the acceptance testing and is also where we incorporate our observed data in (ie. 2 students test score we have on hand).
Q4: Did i interpret this wrongly? So should the incorporation of our observed data be when we are determining the target distribution?
I think I may be already confused. I really hope you can help me out with this. Thank you in advance.