1
$\begingroup$

I'm trying to analyze voting patterns of Ukraine's parliament deputies. I scraped all the data on their voting during last session. Each data entry has following information: Deputy name, date, bill number, vote. The vote field can be yes, no, didn't vote (treated as no) and not present. My problem is with "not present" vote type. I don't want to have deputies that were often absent to be classified as similar as I only care when they voted yes or no. I encoded votes as dummy variables, but not sure what to do with "not present" vote. Treat it as No will definitely skew the results making those who missed a lot of voting sessions look alike. Removing whole columns where one of the deputies wasn't present is also not a solution as I will have no data remaining.

My primary tool of analysis is python with scikit-learn.

$\endgroup$
  • $\begingroup$ What kind of analysis are you doing? Clustering? Are you attached to a particular method? $\endgroup$ – Dougal Apr 30 '15 at 19:50
  • $\begingroup$ As a first step I wanted to do some sort of dimensionality reduction and see how it looks in 2D. But I'm open to any suggestions. $\endgroup$ – Dennis Sakva Apr 30 '15 at 19:51
  • 2
    $\begingroup$ I don't follow your logic of why you want to exclude the absents. From your description, there is a lot of information in there - I'm not a poli-scientist, but it seems to me that there is a hypothesis to be tested as to whether the deputies often absent are in fact similar. Maybe there are similar underlying mechanisms causing them to miss votes. $\endgroup$ – robin.datadrivers May 1 '15 at 14:48
  • $\begingroup$ Lets say we have 5 bills that were voted. Two deputies voted as follows:YYAAAA, NNAAAA (Yes, No, Absent), they will be closer to each other than to say YYYYY for the former and NNNNNN for the latter despite their radically different actual voting pattern. $\endgroup$ – Dennis Sakva May 1 '15 at 18:03
1
$\begingroup$

I suggest that you encode your data so that yes, no, and absent are all represented.

If you have prepared your data with rows for voters and columns for votes, and encoded yes, no, and missing as 1, 0, 9, you can use sklearn.preprocessing.OneHotEncoder to map this into a feature space with columns for yes, no, and missing:

import sklearn.preprocessing
X = sklearn.preprocessing.OneHotEncoder().fit_transform(observed_votes)

import sklearn.decomposition
X_2d = sklearn.decomposition.PCA(n_components=2).fit_transform(X.toarray())

plt.plot(X_2d[:,0], X_2d[:,1], 'o')

enter image description here

Here is an IPython Notebook that simulates some data for the observed_votes so you can see how I'm thinking of organizing things.

$\endgroup$
  • $\begingroup$ Abraham, many thanks for such an extensive answer! LEarned a lot from your post. $\endgroup$ – Dennis Sakva May 5 '15 at 12:26
0
$\begingroup$

In the end I decided to change my voting encoding scheme. I used -1 for No votes, 0 for absent and +1 for Yes votes which makes sense, i think. Here is what I got after I TSNEed the data. enter image description here

Very clear delineation of different groups in the parliament. Plus gives an insight with whom "unaligned" deputies are actually aligned.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.