# Cluster analysis on panel data

I have a panel data set (country and year) on which I would like to run a cluster analysis by country. My data set has around 20 variables.

Here's a summary for my panel data:

panel variable: country (strongly balanced) time variable: year, 2010 to 2013

Running a kmeans cluster analysis on 2013 data only is pretty straightforward. But how would you do the analysis considering all observations in the 2010-2013 period? Is k-means clustering an appropriate approach?

Here's what I ran in Stata for 2013 only:

cluster kmeans var1 var2 var3 var4 var5 var6 if year==2013, k(4) name(test1)

Thanks!

• The panel structure of the dataset would make for an odd interpretation of the output. You might get the same people in different clusters. Maybe this is what you want? A lot of climate scientists use something called empirical orthogonal functions -- from what I understand it is basically a time-varying PCA. Might be worth looking into for your application? Feb 28, 2014 at 21:11
• Thanks for the suggestion ACD! Actually, I would like to identify which countries have the same characteristics. So I have info for around 30 countries, and I would like to group these countries by similar characteristics. Not sure if this helps understand the problem. Thanks again! Feb 28, 2014 at 21:24

I would reshape wide so each year's data is its own variable and then cluster. This will group countries that follow similar timepaths for your 6 variables.

Try something like this in Stata:

reshape wide var@1 var@2 var@3 var@4 var@5 var@6, i(country) j(year);
cluster kmeans var*1 var*2 var*3 var*4 var*6, k(4) name(test1)


I have done this several times before (clustering with panel data) and my approach has been to aggregate information over time to create a dataset with one row per country. The aggregate variables could be means over the 4 years or min or max or something else. Results are difficult to interpret if you use every year as a data point for every country. For example, what does it mean if a country falls in each of the four clusters over the four years?

This is simple to do with data.table in R. An example:

require(data.table)
ClusterData = data.table(PanelDataSet)[, list(MeanVar1 = mean(var1)
, MeanVar2 = mean(var2) #create the aggregate variables
...
)
, by = country] #name the variable to aggregate


Then convert to a matrix (needed for the kmeans function in R)

x = as.matrix(DLQData[,c("MeanVar1", "MeanVar2", ...
), with = FALSE])


Next, standardize the matrix using the scale() function. Then you are ready to cluster!

• Sorry for the switch in programming languages, but I have no idea how to use Stata and a lot of people on here use R. Feb 28, 2014 at 21:37
• This approach sounds interesting we are doing the same thing i.e. converting it into a cross section before using clustering. Can you provide examples of some literature that uses this approach? Dec 24, 2018 at 19:40

I saw something like this in the paper available at the following link:

http://www.ersj.eu/repec/ers/papers/12_1_p2.pdf

basically, they pooled the data and run a pca and subsequent cluster analysis by considering the observations as independent.

• This is a potentially useful contribution. Can you provide a full citation & spell out the procedure a little more fully, in case the link goes dead? Mar 17, 2015 at 19:33