7
$\begingroup$

I have small convolution neural net for classification EEG patterns (2 class problem) with several conv layers and one dense layer. This net performs pretty well. For education purposes I used convolutional part of this pretrained net (all net except dense layer) to extract features from training samples. So this new data representation lineary seperable (because dense layer can correctly classify them) But when I used TSNE on this training set representation and got only one blob and points from different classes where mixed. enter image description here

This situation repeats independently from TSNE parameters. Does it possible to get only one cluster with mixed classes for linearly separable dataset?

Improve this question
$\endgroup$
4
  • $\begingroup$ "This net performs pretty well" - What does "pretty well" mean? What is the test-set (or cross-validated) performance? $\endgroup$
    – amoeba
    Mar 20, 2018 at 15:27
  • $\begingroup$ roc auc equal ~0.75 $\endgroup$
    – Cat-with-a-pipe
    Mar 20, 2018 at 15:48
  • $\begingroup$ On a test set? Using cross-validation? $\endgroup$
    – amoeba
    Mar 20, 2018 at 16:21
  • $\begingroup$ On validation set. But for this picture I used training set, wich is definetly seperable by this net $\endgroup$
    – Cat-with-a-pipe
    Apr 11, 2018 at 14:23

1 Answer 1

Reset to default
7
$\begingroup$

Yes.

You can use the following code to convince yourself.

N <- 1000
P <- 3

# Generates some random data
data <- matrix(data = rnorm(N*P), nrow = N, ncol = P)

# Assignate linearly separable classes
labels <- (data[,1]+data[,2]>0)+1.

# Make sure that the data can be separated
plot(data[,1],data[,2], col = labels, xlab = 'x1', ylab = 'x2')

require(Rtsne)
model <- Rtsne::Rtsne(data)

# Observe this result while varying P
plot(model$Y, col = labels, type = 'p', pch = 21, xlab = 'tSNE x_1', ylab = 'tSNE x_2')

This is what you would observe when P is 3 (only one irrelevant attribute with respect to the linear separation, we are close to reproducing the linear separation).

enter image description here

And for P is 15, the tSNE cannot reproduce the linear separation.

enter image description here

What happened ?

This is simple. The tSNE method relies on pairwise distances between points to produce clusters and is therefore totally unaware of any possible linear separability of your data.

If your points are "close" to each other, on different sides of a "border", a tSNE will consider that they belong to a same cluster.

This was exactly the point of the simulations above. When the number of dimensions is large, points look close to each other, no matter the side of the border they belong to. This is what tSNE fails to capture here.

On the other hand, when the number of irrelevant dimension is low, close points had "no other choice" but to be on the same side of the border.

Side note.

Even though you may have a nice performance with a neural network, it may not mean that your data is linearly separable (unless there is only one unit in your neural, hum, network). Indeed neural networks can recognize non linear boundaries. If you want to test how "linearly separable" a data set is, you should use linear Support Vector Machines or regressions.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ +1 but the last paragraph somewhat misses what OP says: they used t-sne on the output of the last layer (presumably before softmax) so that would indeed be linearly separable. Re-read the part of the question before the figure. $\endgroup$
    – amoeba
    Mar 21, 2018 at 10:47
  • $\begingroup$ @amoeba thank you I missed it indeed. For my arguments to apply we would need to know how many features appear on the last layer... $\endgroup$
    – RUser4512
    Mar 21, 2018 at 10:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.