How to do dimension reduction with sparse data I have 200 vectors representing the percentage marks for 200 different students in the different classes they took. The vectors are 22 dimensional (as there were 22 different classes in total) even though each student only took 6 classes. In other words, the students don't all take exactly the same classes and so the vectors are sparse. If the vectors were dense I would use TSNE. However, as they are sparse I am not even sure how I should represent the values for the classes each student doesn't take.
What is a sensible way to do dimension reduction for sparse vectors of this sort? The ultimate goal, in addition to visualisation, is to cluster the students.
 A: This depends on the goal of clustering.
t-SNE as well as various clustering methods (like hierarchical clustering) can work on distance matrices. And it's your job to construct a distance measure that captures what you wish to achieve. A few examples below.
Example 1
If you want to group the students based on their ability to get good grades the simplest solution would be to ignore the missing classes and simply compare the average grades they achieved. So the distance between two students might simply be the difference of their average grades.
A good additional idea here would be to weight each class according to how hard it was (based for example on the average grades students get in that class)
Example 2
If students are free to choose their classes you might want to group them by their interests. In this case the interests would be reflected by the type of classes they actually chose. In this scenario you would ignore all of their marks and simply code the missing classes as 0 and attended classes as 1. Then compute a distance measure between students based on how many classes they overlap at.
Example 3
Another possible scenario is if you wish to group students based on their ability on various subjects. Here you would have to incorporate both the grades as well as the selection of subjects. A simple (read dumb) solution would be to replace all missing entries for each student with their average ability. Or with the average ability for every student on that subject.
The idea is that when the student didn't take the class - your best guess that he is average at that class.
But you could possibly construct better metrics after some reflection. Just need to think about what the similarity should be between students when none of their classes overlap.

t-SNE and clustering
The examples above show some ways how one could construct a distance matrix between students. After that you can use that matrix for both t-SNE and clustering.
A: Singular value decomposition is a very common strategy for dimension reduction applied to sparse data types. This is because you can leverage specialized sparse SVD solvers (e.g. ARPACK), and for SVD the inputs do not have to be manipulated in any special way which could disrupt sparsity.
