1
$\begingroup$

I am arguing that I can control error vs. coverage by modifying a certain parameter. After running an experiment with leave-many-out validation I have a set of errors along with the parameter value for each error. I want to visualize how I can trade coverage for accuracy by placing restrictions on my parameter.

If I didn't have a parameter to modify I could use a regression error characteristic curve. These curves plot the error tolerance on the $x$-axis vs. the fraction of test points within that tolerance on the $y$-axis.

Drawing several REC curves using different values of my parameter leads to a plot with many overcrossing lines which is hard to interpret. I've also found some work on "ROC surfaces" but I don't know how widespread it is.

Right now I'm using a scatter of mean error vs. test points retained with points colored by the parameter value: enter image description here

It's clear to me what is going on in my figure (e.g. I can set the parameter to 10^2 and get 80% coverage at mean error 200). However, I still feel like I've run into a very common problem that has standardized solutions. Is there is another visualization technique I should be looking into?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.