I am trying to make a variable selection using AIC & Cp Mallows methods.

For AIC:

b <- regsubsets(time~radius_mean+texture_mean+perimeter_mean+area_mean+smoothness_mean+compactness_mean+concavity_mean+concave_points_mean+symmetry_mean+fractal_dimension_mean+radius_se+texture_se+perimeter_se+area_se+smoothness_se+compactness_se+concavity_se+concave_points_se+symmetry_se+fractal_dimension_se+radius_worst+texture_worst+perimeter_worst+area_worst+smoothness_worst+smoothness_worst+compactness_worst+concavity_worst+concave_points_worst+symmetry_worst+fractal_dimension_worst+tumor_size+lymph_node, datasetX)
plot(AIC~I(1:(p-1)),ylab="AIC",xlab="Number of Predictors")

The plot I get is:

enter image description here

The plot says that the minimum AIC is for 8 predictors that, looking at the which table, are perimeter_mean, area_mean, smoothness_mean, concave_points_mean, perimeter_se, area_se, smoothness_se and concave_points_worst.

Everything seems ok. But when I try to do the same for Cp Mallows criteria, I get:

plot(1:(p-1),rs$cp,xlab="Number of predictors",ylab="Cp Mallows")

Error in xy.coords(x, y, xlabel, ylabel, log) : 
  'x' and 'y' lengths differ

That's true. The model has 32 variables and rs$cp is just 8. But I cannot understand why. Could you help me? Maybe am I doing something wrong?

Thank you in advance. Regards!

  • $\begingroup$ You may want to migrate to StackOverflow. $\endgroup$ – Michael R. Chernick Jan 21 '17 at 18:34
  • $\begingroup$ @Scortchi I'm not sure about if the question is on-topic - because it arises from statistical problem, although the OP didn't notice that it had coding causes - but if it is off-topic it should be migrated, not put on hold. $\endgroup$ – Pere Jan 23 '17 at 16:00
  • 2
    $\begingroup$ @Pere: I can't see a statistical question in there at all. The reason I put it on hold is that Stack Overflow don't like us to migrate questions that they then have to close, & this one lacks a minimal complete verifiable example, making it hard to answer conclusively. But because your answer seems very convincing, I tried just now to migrate the post, only to find the OP is blocked from asking questions on Stack Overflow. So it's stuck here, closed - sorry!. $\endgroup$ – Scortchi - Reinstate Monica Jan 23 '17 at 16:48
  • $\begingroup$ Some of the general comments in your earlier thread apply here too. stats.stackexchange.com/questions/257439/… $\endgroup$ – Nick Cox Jan 24 '17 at 12:02

The problem here is that regsubsets checks by default models with just up to a few predictors - something about 8 predictors. Therefore, rs\$rss and rs\$cp length is 8, but 1:(p-1) is longer. That prompts the error message and triggers vector recycling, that causes the pattern we can see in the graph.

You can check in-program help for regsubsets to adjust the maximum number of predictors (but beware that with large numbers of predictors it may take a long time or just crash), or you can adjust the plot order just to plot values from the models actually adjusted by regsubsets. Whatever action that makes the two vectors be of the same length may avoid that error.

BTW: my master's students at UOC are solving this same problem with the same dataset and some of them are finding the same error when using a code just like yours copied from an example problem with less than 10 predictors. Nice to meet you here.


Not the answer you're looking for? Browse other questions tagged or ask your own question.