I am currently estimating a bunch of ARMA models, and using them to predict subsets of my data. In order to evaluate their predictive accuracy I would like to make some ROC plots, however since all of my variables are continuous, I wonder how this could be done in R.
I have looked at the ROCR package, but this seems to only work for dichotomous variables.
Well, that is the basis for ROC curves. You see what proportion of correct predictions (i.e. yes or no) are at a variety of predictor levels. The analog of an ROC curve for continuous outcomes would be a validation plot. You develop a prediction score on a training set and validate it on a test set. Or you develop it on the full set and then use bootstrap methods to create neo-samples for validation.
The ROC analysis is for binary data, but the AUC of a ROC is just one case of an ordinal rank effect size that can also be used for continuous predictors, which is referred to as Ruscio's A (2008), or the probability of superiority, or several other names. It's a fully non parametric version of the Common Language Effect Size.
I have an implementation of both this statistic and its bootstrapped 95% CIs in an R package here, where it's labelled Ruscio's A effect.
Also see discussion and other implementations here and here.
Finally, if you if you're interested in a full regression framework for ordinal, semi parametric modelling, see the PIM package (De Neve, 2017).
