6
$\begingroup$

I have 200 vectors representing the percentage marks for 200 different students in the different classes they took. The vectors are 22 dimensional (as there were 22 different classes in total) even though each student only took 6 classes. In other words, the students don't all take exactly the same classes and so the vectors are sparse. If the vectors were dense I would use TSNE. However, as they are sparse I am not even sure how I should represent the values for the classes each student doesn't take.

What is a sensible way to do dimension reduction for sparse vectors of this sort? The ultimate goal, in addition to visualisation, is to cluster the students.

Improve this question
$\endgroup$
7
  • $\begingroup$ You should thoroughly think over what do those zeros mean. If they are the values that must not influence the analysis anyhow then I would doubt doing a dim. reduction on the combined sample of students at all because students come from different "spaces". Or, if the zeros are valid values indicating zero-level attendance then you may do any analysis putting the origin into zero. It could be even PCA - based on cosine similarity or SSCP (that is, without centering). $\endgroup$
    – ttnphns
    Jun 17, 2018 at 15:49
  • $\begingroup$ @ttnphns I have NaNs currently for the classes the student simply didn't choose. These are what I am wondering what to do with. $\endgroup$
    – graffe
    Jun 17, 2018 at 15:49
  • $\begingroup$ Are you saying NaN is student refusing to answer or that they don't attend? If they know about the course but disregard listening it I'd maybe say it is valid 0 value. $\endgroup$
    – ttnphns
    Jun 17, 2018 at 15:53
  • $\begingroup$ On the other hand, consider students are from different departments with very few common courses, ie they are incomparable populations. Then doing one dim. reduction for them combined would look silly. $\endgroup$
    – ttnphns
    Jun 17, 2018 at 15:56
  • $\begingroup$ @ttnphns No sorry. I student simply has to choose 6 out of 22 classes to take in a year. The remaining class marks are in the data as NaNs currently. All the students are from the same department. $\endgroup$
    – graffe
    Jun 17, 2018 at 15:56

2 Answers 2

Reset to default
4
$\begingroup$

This depends on the goal of clustering.

t-SNE as well as various clustering methods (like hierarchical clustering) can work on distance matrices. And it's your job to construct a distance measure that captures what you wish to achieve. A few examples below.

Example 1

If you want to group the students based on their ability to get good grades the simplest solution would be to ignore the missing classes and simply compare the average grades they achieved. So the distance between two students might simply be the difference of their average grades.

A good additional idea here would be to weight each class according to how hard it was (based for example on the average grades students get in that class)

Example 2

If students are free to choose their classes you might want to group them by their interests. In this case the interests would be reflected by the type of classes they actually chose. In this scenario you would ignore all of their marks and simply code the missing classes as 0 and attended classes as 1. Then compute a distance measure between students based on how many classes they overlap at.

Example 3

Another possible scenario is if you wish to group students based on their ability on various subjects. Here you would have to incorporate both the grades as well as the selection of subjects. A simple (read dumb) solution would be to replace all missing entries for each student with their average ability. Or with the average ability for every student on that subject.

The idea is that when the student didn't take the class - your best guess that he is average at that class.

But you could possibly construct better metrics after some reflection. Just need to think about what the similarity should be between students when none of their classes overlap.

t-SNE and clustering

The examples above show some ways how one could construct a distance matrix between students. After that you can use that matrix for both t-SNE and clustering.

Improve this answer
$\endgroup$
1
$\begingroup$

Singular value decomposition is a very common strategy for dimension reduction applied to sparse data types. This is because you can leverage specialized sparse SVD solvers (e.g. ARPACK), and for SVD the inputs do not have to be manipulated in any special way which could disrupt sparsity.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.