# How to model features with NULL (not necessarily missing) values

I’d like some advice on my options for modeling features (to be used in R) in the following scenario: I want to make a binary prediction using the positions of the planets as the predictors (the X’s). My issue is how best to model the planet positions to allow them to be used by functions such as randomForest and SVM. The problem is with missing, or more appropriately NULL, values for a feature.

As X’s, for each day, I want to capture:

1. Whether the moon is new or full (possibly 1 or 2 features – 1 if value = N|F, 2 if each is a Y|N value))
2. Whether a planet is changing signs (e.g., going into Aries – the same as crossing a 30 degree multiple) (one feature per planet, where value = Y|N if planet is changing signs today))
3. Whether there is an important aspect (angular division) between any pair of planets. An aspect is important if it’s 0, 60, 120, 180 degrees. (N(N-1)/2 features where N = 9 … 36 features). The feature will be a factor with likely values: 0, 60, 120, 180

My big issue is with #3 and how to represent a day when there is no interesting (0,60,120,180) angular difference between a given pair of planets. For example, consider Mercury and Mars. On a day when the angular difference between the Mercury and Mars is 47.52 degrees I really don’t want to record anything, i.e., the value of feature Mercury.Mars would be empty/NULL as 47.52 is not considered important and I don’t want to use it as input to the prediction function (randomForest, SVM, …, etc). The value on such a day is not really missing in the sense that it’s not known or it’s been left out, there just isn’t a value for that feature for that day. Most days will not have an interesting aspect value.

So, how best to model this situation? My concerns:

1. If I don’t record a value (i.e., it’s really missing) then randomForest won’t work as it can’t handle missing values. Not sure about SVM and other functions.
2. If I put some generic value to indicate “nothing here today”, for example *, then I’m concerned that the prediction function (rf, SVM) may associate some uncontrolled-by-me meaning to the “nothing here today” value

There's no problem I can see: for each pair of planets use a set of four dummy variables with binary values for the aspects 0°, 60°, 120°, & 180°. The days when all there are no important aspects for that pair have a value of zero for all dummies. You're perhaps confused because you're expressing the angular differences as numbers: think of it as a factor with the classes "no conjunction", "sextile", "square", "trine", & "opposition". If the application weren't astrology I'd suggest using the sine of the angle as a predictor.

• I have a, possibly unfounded, desire to minimize the number of features -- no concrete reason for it just more features bad, less features good. In the case you suggested, with 9 planets, I'd have (9*(9-1))/2 * 4 == 144 binary variables to represent the possible aspects of interest. Seems like a lot. And double that if you want to take into account both geocentric and heliocentric aspects. Would you lean towards two binary vars to represent new and full moon or a single variable with three values (new, full, neither)? Thanks for your thoughtful feedback. – RobertL Feb 6 '14 at 20:36
• Also, for input to something like randomForest, does it matter if I have a single factor variable with 5 values (conjunction, opposition, sextile, trine, NOTHING) versus 4 binary values where all of the being 0 is equivalent to NOTHING? I.e., is it better to have binary versus N-value factors as predictors (X's)? Just thinking out loud. – RobertL Feb 6 '14 at 20:43
• Doesn't make any difference as far as growing the tree goes whether you call something a factor having $k$ levels or a set of $k-1$ binary variables. If you're interested in "variable importance" or perhaps other stuff it might be useful for the program to know that (so e.g. it can tell you the importance of aspect), so go with categorical variables having $k$ levels. As for reducing the number of features, it can be useful to lessen overfitting, but of course you can't get answers to questions you haven't asked: How many observations do you have? – Scortchi - Reinstate Monica Feb 6 '14 at 21:19
• I have about 3500 (10 years) of observations. I don't think, at least at for now,I'm interested in aspect importance. So you think using categorical values with k levels is the way to go? Where k = number-of-relevant-aspects + 1 for no-aspect? I haven't tested this yet with any predictive function but I just have (unfounded) concerns that given the large number of aspect variables with NOTHING as the value will result in them having more influence than the non-NOTHING values which are the true influencers. Again, these are unfounded concerns. A few tests will answer a lot of questions. – RobertL Feb 6 '14 at 21:42