2
$\begingroup$

I'm looking for some guidance to interpret a UMAP plot. I started with two FASTA files for two different genes. I concatenated everything into a single string of only ACGT. Then I split that into non-overlapping segments of length 400. Then I mapped each character into a value {'A':0.0, 'C':0.333, 'G:0'.666, 'T':1.0}. That gave me a numpy array of 500000 rows by 400 columns containing both genes. I fit_transformed that with UMAP and plotted it as shown. The two colors are the two genes from different organisms.

I've used UMAP and tSNE in other project and successfully understood the formations of clusters (ex MNIST). And I've done the same transform on np.random.random() uniform data and know that you just get a big round blob. I'm trying to undertand the structure of this plot and what I call the "snake".

My question is whether or not this interpretation makes sense: Knowing that UMAP puts two similar points "near" each other that would mean that "AAAA.....AAAA" should be near "AAAA....AAAT". There certainly will be variation between the 400 character segments so maybe that accounts for points from the same genome that are "close", but there's no way I have enough data to statistically fill the 4^400 possible space..... is that why I have a thin structure?

I have empirically noted that if I set the window to 100 instead of 400, the "snake" is fatter and shorter. Since I have the same amount of data, I would then be covering more of the 4^100 possible space. If I increase it to 1000, I see a longer and even thinner "snake".
UMAP Transform of FASTA Data

Improve this question
$\endgroup$
7
  • 3
    $\begingroup$ A couple things. 1) I don't understand the rationale for how you're mapping bases to real numbers (prior to UMAP). For example, under your encoding, "AAA" would be closer to "AAC" than "AAT", assuming Euclidean distance. This seems undesirable (or perhaps I'm missing an important biological fact). It might make more sense to properly treat the bases as categorical variables (e.g. using one-hot encoding in the simplest case). Or, use a distance metric designed for categorical data (biologically motivated if possible; I've heard genomics people like edit distances). $\endgroup$
    – user20160
    Commented Aug 25, 2019 at 16:28
  • 1
    $\begingroup$ 2) Is there any discernible pattern as you move along the snake? E.g. do consecutive segments along the gene map to nearby points along the snake? $\endgroup$
    – user20160
    Commented Aug 25, 2019 at 16:28
  • 2
    $\begingroup$ Tools like UMAP and tSNE try to produce embeddings that reflect structure in the input distances. What I mean is that your preprocessing method introduces a certain bias into these distances (e.g. "A" is closer to "C" than "T"). So, the resulting embedding may reflect some structure that's a consequence of the preprocessing rather than the underlying data. I'm not saying this is the reason for the snake, but it's worth considering separately. By analogy, in NLP, one would typically provide input to word embedding layers as a one-hot encoding over the vocabulary. $\endgroup$
    – user20160
    Commented Aug 25, 2019 at 18:38
  • 4
    $\begingroup$ My first guess is that your segments are in fact overlapping (this would be contrary to what you wrote) and so neighbouring DNA segments end up close by and the whole gene forms a 1D structure that appears as a "snake". CC @user20160. $\endgroup$
    – amoeba
    Commented Aug 26, 2019 at 9:27
  • 1
    $\begingroup$ @amoeba I agree with your thought. I did go back and twice verified that the sequences do not overlap. Thanks for the input everyone. $\endgroup$
    – user938512
    Commented Sep 2, 2019 at 2:50

1 Answer 1

Reset to default
1
$\begingroup$

It might be late but this is for those who encountered the same type of output. I got the snake-like embedding when I reduced the dimensionality of a time-series data with 400 features. Most of the features were correlated. Output was not ideal for clustering. The convention is to sample from the data (in my case %10 was enough), fit the sample to UMAP. Then transforming the rest of the data gives much more "nice-looking" results.

Improve this answer
$\endgroup$
1
  • $\begingroup$ +1 for correlated time-series $\endgroup$
    – Ggjj11
    Commented Oct 30 at 9:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.