Correlated variables - classification I have some data set and need to use a few classification methods to make prediction. I first need to pre-process the data set.
France is administratively divided in regions (13), and regions are divided in departments (96). Then at a smaller level you have towns.
My data set contains quite a few predictors, including "regions", "departments", "towns".
It seems obvious that those 3 predictors will be correlated as for instance: if department = "Finistère" then regions = "Bretagne" ; if department = "Morbihan" then regions is also equal to "Bretagne".
So what should I do with those kind of variables, that are "included" into other variables ? Should I take the one with more factor levels (here towns) ? Or maybe town is too specific and I should keep departments ? 
EDIT:
In my case, the outpout is either 1, 0 or -1 (3 categories), so SVM would not work.
The classifiers I'm gonna use are sensitive to varible correlation I think.
What would be the best this to do in this case ?
 A: The effect of correlation depends on the type of classifier. Some classifiers like Naive Bayes assume feature independence but others like SVM are less sensitive to the correlation of variables. Note that SVM usually performs well in image analysis where having correlated predictors is extremely common.
You can also try creating a new predictor using the correlated ones.
A: This will likely depend on how much data you have and what your goal is for the analysis.
Ideally, the more specific your information is the more specific your output is. You can always aggregate later, but you can't dis-aggregate. If you make a classification model at the town level you can later aggregate everything to the region level. However, if you model at the region level you cannot dis-aggregate to the town level. In other words, it often makes sense to keep the town level and drop the region/department for the clustering/prediction and then bring the region/department back in later.
That being said, you need enough data at the town level to be able to do that. Does your data have many (more is better) points of data for each town that you can use for your analysis? Is the amount of data for each town reasonably representative of the population? If you are missing data for certain towns, or don't have enough data for each town, then you may need to perform the classification and analysis at the department level or even the region level. 
