I guess that you have specified choicenum as a factor, but you assume that it is numeric. But you need a factor in the random effects part of the equation for lmer. So you might try the following:

First, specify choicenum as numeric in your data.frame. Then, run

m1 <- lmer(rtln~choicenum + ifrelevant + condition + (1|subject/as.factor(choicenum)), 
  data= YourData)

Of course you could also specify ... as.numeric(choicenum) ...

I guess that you have specified choicenum as a factor, but you assume that it is numeric.

If you specify choicenum as a factor in the fixed effects part, lmer will --- as you should know from the help pages of lmer, lm and the R Introduction, estimate effects comparing all factor levels to a 'baseline' level (see ?contr.treatment and ?contrasts), so that is why you get 5 entries for choicenum in the fixed effects part.

But you need a factor in the random effects part of the equation for lmer. So you might try the following:

First, specify choicenum as numeric in your data.frame. Then, run

m1 <- lmer(rtln~choicenum + ifrelevant + condition + (1|subject/as.factor(choicenum)), 
  data= YourData)

Of course you could also specify ... as.numeric(choicenum) ...