Balanced datasets are almost all predicted negative Problem
I am trying to do sentiment analysis using pretrained word vectors GloVe, which is essentially a look-up table that maps word to a fix-dimension vector. Since GloVe is initially designed to provide word embedding for each individual word, for the entire document, I take the average of the all words and use the resulting averaged vector to represent the document. Then the vectors are treated as features and sent to SVM for classification.
I know this approach might not provide satisfying result, surprisingly, it performs very poor (see below).

*

*In terms of accuracy, it is almost like random guess.

*The F1 score is extremely low . More surprisingly, true negative rate (TNR) is almost 1 but true positive rate (TPR) is almost 0 (three confusion matrices are similar and only one is included here).



I kind of understand why the accuracy is low after I did some visualizations of my document vectors using TSNE (see below). But I do not understand why the predictor will be biased to negative samples.

Updates:
I did make the dataset balanced before any processing. There are 10000 positive samples and 10000 negative samples.
 A: As you point out an accuracy of 0.5 (on average) is the lowest you could get by random labeling. A very low F1 score and an accuracy of 0.5 are therefore not logically inconsistent. I would recommend rephrasing the question title. Your question is "why does the classifier predict everything as negative?". If the class imbalance does not exist (no error in your pre-processing), that is indeed surprising. 
At first glance, your confusion matrix is indicative of a highly imbalanced data set whereby the classifier labels virtually all observations as negative. You mention you manually down or up sample to ensure equal class distribution beforehand. My guess is that there is something wrong with this pre-processing. For instance do you apply this sampling equally to train/validation and test sets? (Mind you, In real-life you do not have knowledge or control of the test set). 
Why not try with a simple classifier first and what output you get. There should be no reason a classifier is biased towards a particular label. 
