9
$\begingroup$

I have a panel data set (country and year) on which I would like to run a cluster analysis by country. My data set has around 20 variables.

Here's a summary for my panel data:

panel variable: country (strongly balanced) time variable: year, 2010 to 2013

Running a kmeans cluster analysis on 2013 data only is pretty straightforward. But how would you do the analysis considering all observations in the 2010-2013 period? Is k-means clustering an appropriate approach?

Here's what I ran in Stata for 2013 only:

cluster kmeans var1 var2 var3 var4 var5 var6 if year==2013, k(4) name(test1)

Thanks!

$\endgroup$
2
  • $\begingroup$ The panel structure of the dataset would make for an odd interpretation of the output. You might get the same people in different clusters. Maybe this is what you want? A lot of climate scientists use something called empirical orthogonal functions -- from what I understand it is basically a time-varying PCA. Might be worth looking into for your application? $\endgroup$ Feb 28, 2014 at 21:11
  • $\begingroup$ Thanks for the suggestion ACD! Actually, I would like to identify which countries have the same characteristics. So I have info for around 30 countries, and I would like to group these countries by similar characteristics. Not sure if this helps understand the problem. Thanks again! $\endgroup$
    – Dan Aronne
    Feb 28, 2014 at 21:24

3 Answers 3

4
$\begingroup$

I would reshape wide so each year's data is its own variable and then cluster. This will group countries that follow similar timepaths for your 6 variables.

Try something like this in Stata:

reshape wide var@1 var@2 var@3 var@4 var@5 var@6, i(country) j(year);
cluster kmeans var*1 var*2 var*3 var*4 var*6, k(4) name(test1)
$\endgroup$
6
$\begingroup$

I have done this several times before (clustering with panel data) and my approach has been to aggregate information over time to create a dataset with one row per country. The aggregate variables could be means over the 4 years or min or max or something else. Results are difficult to interpret if you use every year as a data point for every country. For example, what does it mean if a country falls in each of the four clusters over the four years?

This is simple to do with data.table in R. An example:

require(data.table)
ClusterData = data.table(PanelDataSet)[, list(MeanVar1 = mean(var1)
, MeanVar2 = mean(var2) #create the aggregate variables
...
)
, by = country] #name the variable to aggregate 

Then convert to a matrix (needed for the kmeans function in R)

x = as.matrix(DLQData[,c("MeanVar1", "MeanVar2", ...
), with = FALSE])

Next, standardize the matrix using the scale() function. Then you are ready to cluster!

$\endgroup$
2
  • $\begingroup$ Sorry for the switch in programming languages, but I have no idea how to use Stata and a lot of people on here use R. $\endgroup$
    – wcampbell
    Feb 28, 2014 at 21:37
  • $\begingroup$ This approach sounds interesting we are doing the same thing i.e. converting it into a cross section before using clustering. Can you provide examples of some literature that uses this approach? $\endgroup$ Dec 24, 2018 at 19:40
4
$\begingroup$

I saw something like this in the paper available at the following link:

http://www.ersj.eu/repec/ers/papers/12_1_p2.pdf

basically, they pooled the data and run a pca and subsequent cluster analysis by considering the observations as independent.

$\endgroup$
1
  • 2
    $\begingroup$ This is a potentially useful contribution. Can you provide a full citation & spell out the procedure a little more fully, in case the link goes dead? $\endgroup$ Mar 17, 2015 at 19:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.