I have some data set and need to use a few classification methods to make prediction. I first need to pre-process the data set.

France is administratively divided in regions (13), and regions are divided in departments (96). Then at a smaller level you have towns.

My data set contains quite a few predictors, including "regions", "departments", "towns".

It seems obvious that those 3 predictors will be correlated as for instance: if department = "Finistère" then regions = "Bretagne" ; if department = "Morbihan" then regions is also equal to "Bretagne".

So what should I do with those kind of variables, that are "included" into other variables ? Should I take the one with more factor levels (here towns) ? Or maybe town is too specific and I should keep departments ?


In my case, the outpout is either 1, 0 or -1 (3 categories), so SVM would not work.

The classifiers I'm gonna use are sensitive to varible correlation I think.

What would be the best this to do in this case ?

  • 2
    $\begingroup$ If you are prepared to deal with some complexity you could look into the type of approach variously known as nested, multilevel, or hierarchical linear modeling. $\endgroup$
    – rolando2
    Apr 23 '17 at 12:06
  • $\begingroup$ It seems that your data will have group effects which will be removed if you use vanilla classification. I think these effects should be preserved. Hence, like @rolando2 said, you might need to look at hierarchical models. Could you please share a sample layout of your data so that we can have clearer understanding of your problem? $\endgroup$
    – kusur
    Dec 11 '18 at 6:05

The effect of correlation depends on the type of classifier. Some classifiers like Naive Bayes assume feature independence but others like SVM are less sensitive to the correlation of variables. Note that SVM usually performs well in image analysis where having correlated predictors is extremely common.

You can also try creating a new predictor using the correlated ones.


This will likely depend on how much data you have and what your goal is for the analysis.

Ideally, the more specific your information is the more specific your output is. You can always aggregate later, but you can't dis-aggregate. If you make a classification model at the town level you can later aggregate everything to the region level. However, if you model at the region level you cannot dis-aggregate to the town level. In other words, it often makes sense to keep the town level and drop the region/department for the clustering/prediction and then bring the region/department back in later.

That being said, you need enough data at the town level to be able to do that. Does your data have many (more is better) points of data for each town that you can use for your analysis? Is the amount of data for each town reasonably representative of the population? If you are missing data for certain towns, or don't have enough data for each town, then you may need to perform the classification and analysis at the department level or even the region level.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.