1
$\begingroup$

I am doing a thesis on the generation of synthetic data for training a deep learning model and evaluating it on real data. I have a few different real datasets, and I generated multiple synthetic datasets with different parameters. I want to write a chapter with data description and would like to be able to visualize them. But I am wondering which tool would make more sense, t-SNE or PCA. What do you think?

$\endgroup$

1 Answer 1

1
$\begingroup$

Assuming you are referring to obtaining a visual representation of samples' variability/similarity, I think it really depends on the scope, but I'd probably favor t-SNE over PCA given that the data is perceptual and thus likely to contain non-linear patterns.

Have you considered plotting embeddings of your images instead of the raw data? If you are applying auto-encoders or VAEs the latent representation could be a nice input for visualization. Also, if you go down the t-SNE path, you might want to check UMAP as well (this nice post could be a good starting point). Good luck with your thesis!

$\endgroup$
6
  • 1
    $\begingroup$ Thanks for the comment! Yes, I should have mentioned in the post. I was thinking exactly about using embeddings. In this case, do you still favor t-SNE? $\endgroup$
    – Manveru
    Jul 9, 2022 at 14:12
  • 1
    $\begingroup$ I would try the following then: do the PCA on the embeddings, check the elbow plot, and keep only the components that explain say, 95% of the variance. Visualize the first few PCs to get an idea of how the samples are captured. Then do t-SNE on those first n PCs (the ones that explain the majority of the variance), and check the result. This is a known strategy that should get rid of unwanted noise, and lets you also check both dimensionality reductions in the process. $\endgroup$
    – gianMa
    Jul 9, 2022 at 19:43
  • $\begingroup$ That is a great idea, I ll check that. Thanks! $\endgroup$
    – Manveru
    Jul 9, 2022 at 20:19
  • 1
    $\begingroup$ This is something I often see in my field (high dimensional gene expression data), like in this vignette, see the " Determine the ‘dimensionality’ of the dataset" section. However, it might be only a practical way to fasten up the t-SNE/UMAP reducing the features. I don't know whether the result would be better by running on PCs instead of raw embeddings. In your case your embedding might be already reduced in dimension, so applying directly t-SNE should be feasible and my suggestion on doing PCA first might be out of scope! $\endgroup$
    – gianMa
    Jul 10, 2022 at 9:18
  • 1
    $\begingroup$ Cool, thanks for the insights. I am trying UMAP as well, I didnt know it before but it seems pretty useful + runs faster than tsne $\endgroup$
    – Manveru
    Jul 10, 2022 at 19:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.