3
$\begingroup$

I am working on this dataset: https://www.kaggle.com/russellyates88/suicide-rates-overview-1985-to-2016, and it has a lot of categorical variables, while I am more used to work with the continuous ones.

Except for the binary "sex" variable, which is simple, there are variables with more categories: "generation", "continent" and, especially, "country". There are ~100 countries, and countries are definitely not ordinal, so I suppose I cannot just convert countries to numbers since the distances between them will not make sense. But at the same time I do not have a good feeling about make ~100 columns for countries dummy variable.

  1. Is it a good approach to create these dummy variables and then just carry out dimensionality reduction? Which kind of dimensionality reduction would be the most suitable?
  2. Is there a better alternative I don't know about?
Improve this question
$\endgroup$
6
  • $\begingroup$ Have a look at stats.stackexchange.com/questions/146907/… $\endgroup$ – kjetil b halvorsen♦ Jan 12 '20 at 3:36
  • 3
    $\begingroup$ Everything depends on how you are going to use these categorical variables. If you are talking of reducing the number of categories - see the link provided above. If by dimensionality reduction you mean (as normally) imbedding the many categories in few dimensions that are continuous variables - then this quantification of categorical variables is known as (multiple) correspondence analysis. $\endgroup$ – ttnphns Jan 12 '20 at 3:41
  • $\begingroup$ @ttnphns yes, I was originally thinking about performing MCA, but I am not sure if it make sense in this case (as I've mentioned, I have experience only with categorical data). So, if have columns country_0, country_1,..., country_100 and all the other categorical columns and then perform MCA, this would be a viable solution to this problem? And if I want to combine it with the continuous data, should I eg. perform PCA separately on the continuous data, and then take results both from MCA and PCA? (I know that FAMD exists, but I currently cannot seem to run the only implementation I've found). $\endgroup$ – Valeria Jan 12 '20 at 3:46
  • $\begingroup$ What is the problem with having 100 one-hot encodings? $\endgroup$ – Tim♦ Jan 12 '20 at 7:49
  • 3
    $\begingroup$ "Lack of correlation" is not a valid criteria for excluding variables. $\endgroup$ – Tim♦ Jan 12 '20 at 8:33
1
$\begingroup$

In the case of countries I would recommend using predefined groups (at least in the beginning). Geographic groups (continent, region etc.) Economic groups (currency, trade, income etc.) Cultural/political groups (religion, state, war etc.)

I would prefer a solution using this kind of variables because interpreting results will be a lot easier if results are based on groups that are already studied.

Improve this answer
$\endgroup$
1
  • $\begingroup$ In the dataset there is total GDP & GDP per capita & population. Do you think that it would make sense to perform clustering on those, and from these create several groups of countries for these dataset? (As it's my exploration and not real-life project, I'm just curious to do as much as possible with the dataset itself, without bringing additional sources.) $\endgroup$ – Valeria Jan 12 '20 at 4:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.