6
$\begingroup$

I have the following sorted data (sampling from parametric space [1,5]) with respect to their distances of parameter Theta. i.e., Let say N = 1000,

Theta : 1.1, 1.7, 1.9, 2.4, 2.8, . . . , 4.9

Distance : 0.2, 0.3, 0.5, 0.9, 1.1, . . . , 1.9

From literature I know that more than 3% (i.e., 0.03*N) data are not useful, but I don't know where to cutoff? Could you suggest any re-sampling methods? How can I treat this problem? Classification or Regression?

This data is basically output of rejection sampling (see http://en.wikipedia.org/wiki/Approximate_Bayesian_computation). Now I hope that would be clear.

Improve this question
$\endgroup$
2
  • 3
    $\begingroup$ What is theta, and what is the distance measured from? What is the goal? What do you mean that the data is not useful? $\endgroup$
    – Aniko
    Commented Aug 22, 2011 at 18:59
  • 1
    $\begingroup$ In place of "from literature" you might cite a specific paper. $\endgroup$
    – Karl
    Commented Aug 22, 2011 at 22:23

2 Answers 2

Reset to default
3
$\begingroup$

One approach to choosing the cutoff value $\epsilon$ for ABC rejection sampling is the following (similar to Aniko's answer). Simulate several test data sets from known parameter values which are vaguely similar to your observed data (e.g. by performing ABC with a relatively large $\epsilon$). From the ABC output for a test data set, some criterion of performance compared to the true parameters can be calculated, such as mean squared error. Calculate this for all test data sets at many $\epsilon$ values, and choose $\epsilon$ to optimise the mean criterion (as this is a Monte Carlo estimate of its expectation). This requires many repetitions of the ABC algorithm, but can be done efficiently by using the same $N$ data simulations in every ABC algorithm (although this introduces some dependency between simulations).

In general, there is not a lot of published work on the choice of $\epsilon$. I think the approach above has been used somewhere and I will edit if I remember the references. An alternative is in "Choosing the Summary Statistics and the Acceptance Rate in Approximate Bayesian Computation" by Michael Blum. Other methods that I'm aware of apply only to SMC or MCMC methods.

Improve this answer
$\endgroup$
1
  • $\begingroup$ But how I know the true parameter value in advance? $\endgroup$
    – love-stats
    Commented Sep 28, 2011 at 19:07
2
$\begingroup$

Based on your edit, it appears that you are looking for guidance in selecting the tolerance parameter $\epsilon$ for ABC sampling. I don't know much about the topic, but $\epsilon$ should be small. A simple possibility is to select several different values and see whether the resulting posterior distributions look similar (based on new sets of samples). The largest value that still gives the same posterior can be used.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I have shifted the choice of tolerance parameter (choose advance distance between simulated and observed summary statistics) into choosing an acceptance cutoff. For simple ABC algorithm 3% data is meaningful, but I want to be more precise for this cutoff as it varies with parametric of interest. $\endgroup$
    – love-stats
    Commented Aug 24, 2011 at 15:22
  • $\begingroup$ How you would compare the posterior? Should I calculate distance between them? $\endgroup$
    – love-stats
    Commented Sep 28, 2011 at 19:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.