4
$\begingroup$

I built an rpart pruned tree as dictated by cross validation and the prp graph of the node only shows a simple tree with 3 variables, the same as the text print out.

Question #1: How do I get the list of variables only used in the text print out?

Question #2: Why are so many variables listed as important if it did not use them? (See below)

Question #3: Is the tree actually secretly using all of those variables? (See far below)

> print(mtree)
n= 2313 

node), split, n, deviance, yval
      * denotes terminal node

 1) root 2313 1521599000000  85.66406  
   2) demographics.pct_not_proficient_in_english>=3.873321 908  392806500000  68.50856  
     4) social_associations.association_rate< 7.301414 370  112270700000  57.16540 *
     5) social_associations.association_rate>=7.301414 538  234775200000 101.37220  
      10) access_to_exercise_opportunities.pct_with>=82.59758 360   96954670000  92.94713 *
      11) access_to_exercise_opportunities.pct_with< 82.59758 178  101827200000 237.00360 *
   3) demographics.pct_not_proficient_in_english< 3.873321 1405  924041700000 165.73420  
     6) access_to_exercise_opportunities.pct_with>=70.575 1002  557071900000 148.54600  
      12) access_to_exercise_opportunities.pct_with>=85.6941 408  173252300000 121.82280 *
      13) access_to_exercise_opportunities.pct_with< 85.6941 594  333415200000 224.46240 *
     7) access_to_exercise_opportunities.pct_with< 70.575 403  234362500000 459.06940 *
> 

In the above only 3 unique predictor variables are used. But variable.importance lists 21 variables.

> print(names(mtree$variable.importance))
 [1] "demographics.pct_not_proficient_in_english"     "access_to_exercise_opportunities.pct_with"     
 [3] "social_associations.association_rate"           "severe_housing_problems.pct"                   
 [5] "commuting_alone.pct_drive"                      "motor_vehicle_crash_deaths.mv_mortality_rate"  
 [7] "high_housing_costs.pct"                         "diabetes.pct_diabetic"                         
 [9] "access_to_parks.pct_park"                       "college_degrees.pct"                           
[11] "dentists.dentist_rate"                          "access_to_recreational_facilities.rec_fac_rate"
[13] "pct_illiterate"                                 "other_primary_care_providers.pcp_rate"         
[15] "primary_care_provider_rate.pcp"                 "limited_access_to_healthy_foods.pct"           
[17] "some_college.pct"                               "excessive_drinking.pct"                        
[19] "physically_unhealthy_days"                      "teen_birth_rate"                               
[21] "adult_smoking.pct_smokers"                     

summary(mtree) lists a lot of information but lets focus on Node 6:

Node number 6: 1002 observations,    complexity param=0.03312596
  mean=148.546, MSE=22421.61 
  left son=12 (408 obs) right son=13 (594 obs)
  Primary splits:
      access_to_exercise_opportunities.pct_with < 85.6941  to the right, improve=0.09048102, (0 missing)
      access_to_parks.pct_park                  < 22.5     to the right, improve=0.06748198, (1 missing)
      high_housing_costs.pct                    < 30.4761  to the right, improve=0.05777937, (0 missing)
      dentists.dentist_rate                     < 53.61436 to the right, improve=0.05159972, (0 missing)
      long_commute_driving_alone.pct_drives     < 16.75    to the right, improve=0.04868044, (0 missing)
  Surrogate splits:
      access_to_parks.pct_park                     < 34.5     to the right, agree=0.834, adj=0.363, (0 split)
      limited_access_to_healthy_foods.pct          < 9.110028 to the left,  agree=0.818, adj=0.300, (0 split)
      motor_vehicle_crash_deaths.mv_mortality_rate < 14.93367 to the left,  agree=0.805, adj=0.252, (0 split)
      excessive_drinking.pct                       < 12.95    to the right, agree=0.791, adj=0.198, (0 split)
      dentists.dentist_rate                        < 50.45416 to the right, agree=0.788, adj=0.188, (0 split)

According to print(mtree), Node 6 only decides with access_to_exercise_opportunities.pct_with>=70.575

But access_to_parks.pct_park is mentioned in the summary. Is access_to_parks.pct_park actually used in the final tree, or is it just telling me that it was a close 2nd choice for that node?

$\endgroup$
3
  • $\begingroup$ as.character(unique(mtree$frame$var[!(mtree$frame$var == "<leaf>")])) answers Q1. Retrieved from code of printcp $\endgroup$
    – Chris
    Commented Dec 5, 2017 at 2:21
  • $\begingroup$ +1 for Q2: I fitted a tree and got positive importance for variables that are never used for splitting. Should be 0. Any updates? $\endgroup$
    – PaulG
    Commented Dec 11, 2020 at 13:35
  • $\begingroup$ @PaulG - I like to use something like Boruta which augments the training with random columns, and also randomly permutes the columns. This can be used to inform removal of actually non-contributing variables. Given the way that the importance is computed, if there is an interaction between variables that is significant, Boruta indicates it as important, even if the raw variable might be suspicious or non-informative in something that doesn't account for variable interactions. $\endgroup$ Commented Sep 29, 2022 at 19:49

1 Answer 1

2
$\begingroup$

This question has been there for a while, but I will answer hoping it will help some.

Question #1: How do I get the list of variables only used in the text print out?

As Chris already answer, you can get it from the frame of the tree with

as.character(unique(mtree$frame$var[!(mtree$frame$var == "<leaf>")]))

Question #2: Why are so many variables listed as important if it did not use them?

The CART algorithm used in rpart quantify for each variable how much they reduce the sum of squared residuals (residual being the difference between the predicted value and the target value). You can see this as how much it reduces the error.

Variables are therefore ranked based on their capacity to reduce the sum of squared residuals. The final combination of variables used in the optimal tree is not necessarily the variables that reduce the most the sum of squared residuals (a simple way to understand this is for instance the two most important variables are exactly the same, only one would be practically useful). The final tree uses the combination of variable that reduce best the sum of squares.

Question #3: Is the tree actually secretly using all of those variables?

The list is a summary of the best choices at a given node ranked by how much they reduce the error, so only the one on top of the list is used.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.