Compare two sets of linear mixed effects models

Question

I have two questions that (i think) should be solved in the same way.

I'm using the lme4 package in R to analyze how well a set of subjective ratings (from 12 participants) tracks an objective measure of distance between sets of points. Each participant did 270 different ratings.

I'm not interested in the absolute value of the rating, or of the distance measured per se, but rather in the relationship between the (subjective) rating provided and the (objective) distance measured.

So I compare the following two models (full and restricted) to get the predictive value of the rating.

Model.full <- ( DistanceMeasured ~ ratingOfDistance + age + (1|participant) )
Model.null <- ( DistanceMeasured ~ age + (1|participant) )
anova(Model.full, Model.null)

The two questions are:

1. How could I compare the predictive power of two independent variables in the same dataset? (In the example above, I'd like to know whether rating is a significantly better predictor of the distance than age is).
2. How could I compare the predictive power of the same variable in two separate datasets? (So for example I'd like to compare whether the relationship between the rating and the distance is better in men tahn in women).

Essentially, I'm looking for a way in which I can compare the model comparisons themselves. If the data are normalized, this should be somehow possible.

Answer 1

Predictive power... as in mean squared error, $||Y - \hat{Y}||^2$?

1. I would just calculate the difference of the two MSE's, and then cross-validating this (with leave-one-out or so) to get approximate p-values of the difference.
2. If you have independent data sets then you don't need cross-validation I guess.

Maybe others have more clever solutions?