I have a data-set with 7 features, 6 numeric and 1 categorical. the categorical data is "Species", which has the ability to be sub-categorized (species, genus, family, order, etc...).
I want to build a linear regression model from this dataset. How should I decide which dummy variables to include in the final model? Can I mix dummy variables from different category levels? If I include a dummy variable from one category level like (species=dog) do I need to include all dummy variables from that level (species=cat), or can I just roll up the non-significant ones into "Other"?
My initial thought was to start with ALL the variables numeric and dummy from all the possible levels (there will be ~100 dummy variables from all levels), and all treated equally. Then build the model using variable addition selecting the most significant variable at each step, till I got to a P value, on the new variable I felt like stopping at (likely 0.01).
After that I was considering using target shuffling on the final model and eliminating any variables that showed a P of 0.01 or more.
Are there any other specific issues I need to be aware of when dealing with sub-categories like this on a linear regression model?