I am arguing that I can control error vs. coverage by modifying a certain parameter. After running an experiment with leave-many-out validation I have a set of errors along with the parameter value for each error. I want to visualize how I can trade coverage for accuracy by placing restrictions on my parameter.

If I didn't have a parameter to modify I could use a regression error characteristic curve. These curves plot the error tolerance on the $x$-axis vs. the fraction of test points within that tolerance on the $y$-axis.

Drawing several REC curves using different values of my parameter leads to a plot with many overcrossing lines which is hard to interpret. I've also found some work on "ROC surfaces" but I don't know how widespread it is.

Right now I'm using a scatter of mean error vs. test points retained with points colored by the parameter value: enter image description here

It's clear to me what is going on in my figure (e.g. I can set the parameter to 10^2 and get 80% coverage at mean error 200). However, I still feel like I've run into a very common problem that has standardized solutions. Is there is another visualization technique I should be looking into?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.