Questions tagged [sequence-analysis]

Analysis of a DNA, RNA, or peptide sequence to understand its features, function, structure, or evolution.

Filter by
Sorted by
Tagged with
80
votes
14answers
26k views

What is the role of the logarithm in Shannon's entropy?

Shannon's entropy is the negative of the sum of the probabilities of each outcome multiplied by the logarithm of probabilities for each outcome. What purpose does the logarithm serve in this equation? ...
35
votes
4answers
5k views

Is LSTM (Long Short-Term Memory) dead?

From my own experience, LSTM has a long training time, and does not improve performance significantly in many real world tasks. To make the question more specific, I want to ask when LSTM will work ...
16
votes
4answers
4k views

Framing the negative binomial distribution for DNA sequencing

The negative binomial distribution has become a popular model for count data (specifically the expected number of sequencing reads within a given region of the genome from a given experiment) in ...
10
votes
2answers
136 views

How to prove cooperation from behavioural sequences

Situation: Two birds (male and female) protect their eggs in nest against an intruder. Each bird can use either attack or threat for protection, and be either present or absent. There is a pattern ...
8
votes
3answers
3k views

Sequential pattern mining on single sequence

Can someone give me a hint about a good approach to find a frequent patterns in a single sequence. For example there is the single sequence ...
8
votes
1answer
959 views

When and how to use weights for sequence analysis in social science?

Weighting in sequence analysis So far, I have scarcely found papers that address the issue of weighting for sequence analysis (using for example the optimal matching algorithm). Sequence analysis ...
8
votes
2answers
303 views

Identifying sequential patterns

I am working with sequence data which are long lists of malware win-api calls. I am trying to cast the problem of identifying 'malware behavior' into one of finding sequential patterns. I treat each ...
8
votes
1answer
133 views

How do I statistically rephrase this question

I am analyzing a dataset containing observations from n number of attempts by players in a game. If I am building a regression model to predict the outcome of each attempt given 1 or more descriptors ...
6
votes
2answers
3k views

Sums-of-Squares (total, between, within): how to compute them from a Distance Matrix?

I am having trouble understanding the concept of Sum of Squares in the context of distance matrices (Studer et al. 2010). The Sum of Squares I am familiar with is the classical $SS$ from ANOVA, ...
6
votes
2answers
648 views

Other substitution matrices for missing value state in sequence analysis with TraMineR?

We have a question about how to deal with missing values/gaps within sequences. We like to set up our own substitution-cost matrix for the Optimal Matching process. As far as we know, TraMiner allows ...
6
votes
1answer
407 views

predicting tree structure

This topic is actually rather hard to google for as 'tree' has been overloaded in this domain to refer to decision trees. I'd be interested in having a learning algorithm produce code, such as used ...
6
votes
1answer
6k views

Using Keras LSTM RNN for variable length sequence prediction

I have a set of sequences. Each sequence is the form $\{(s_1,l_1),(s_2,l_2) \ldots\}$ where $s_i$'s are real valued numbers and $l_i$s are labels from a fixed alphabet. It is important to note that ...
6
votes
2answers
4k views

similarity measure between two different ordered sequences

I know we can quantify the similarity between two sequences with the same length and same elements by rank order correlation. But how to measure similarity between two sequences of different length, ...
5
votes
2answers
943 views

Should normalization completely weed out correlation?

I have two variables: ordering & length. The former measures the ordering of a sequence (i.e. all permutations of A-B-C), and the former is the length of the sequence (i.e. A-B-C has a length of 3)...
5
votes
1answer
12k views

Optimum number of epochs and neurons for an LSTM network

I wanted to know if there's a way to select an optimum number of epochs and neurons to forecast a certain time series using LSTM, the motive being automation of the forecasting problem, i.e. the ...
5
votes
1answer
1k views

Problems with Groups in Traminer

I'm having a problem using the group function in TraMineR. I have a data set that contains SPELL data, so multiple rows per case. I also have demographic data per case, at one row per case. I merge ...
5
votes
1answer
1k views

How to estimate the centroid of clustered sequences?

I have run a sequence analaysis using the Optimal Matching algorithm. Afterwards, I have clustered the resulting distance matrice using the Ward algorithm and calculated silhouettes as measures of ...
5
votes
1answer
887 views

Statistical test to assess differences between two sequencing methods

To asses differences between two DNA sequencing methods (454 and MiSeq) I have made artificial mixtures of different bacteria in the lab (mock communities). An Example might be (numbers are ...
4
votes
2answers
312 views

Modifying the time granularity of a state sequence

I created a sequence object from my SPELL-formatted data set. The sequence length of the sequence object is 1440 (i.e., 1-min intervals for a day). Is there any easy way for TraMineR to convert the ...
4
votes
1answer
1k views

How to describe multi-channel sequence objects in TraMineR?

I have created a multi-channel object here: https://github.com/aronlindberg/VOSS-Sequencing-Toolkit/blob/master/creating_multichannel_object.R However, when I try to use various functions to ...
4
votes
2answers
1k views

Clustering customers by their orders sequence patterns

I have dataset with clients orders. Example: ...
4
votes
1answer
1k views

Which Algorithms can model a sequence and predict the next value of this sequence

I have a repeating sequence like the following, with occasionally random values (maybe noise): ABCDABCEDABCDABCD What could be an algorithm to model the sequence and predict a following value at a ...
4
votes
2answers
130 views

Position-dependent substitution cost matrix for sequence alignment

Many sequence alignment algorithms and those which measure similarity between sequences require specification of a cost matrix defined on the sequence alphabet. However, the cost incurred does not ...
4
votes
1answer
760 views

What is the relationship between substitution cost matrix and indel costs?

In TraMineR I might compute dissimilarities as such: ...
4
votes
1answer
843 views

Visualize process data

I have a dataset of order processing with 8 million rows with following columns: HistoryId - Identity column of the records ...
4
votes
1answer
30 views

Can LSTMs classify multivariate time sequences?

I've been learning about LSTMs and I commonly see them applied to the same type of task. For example, given a 1D list of values, predict which class they belong to. Or, given 8 values from 8 sensors, ...
4
votes
1answer
376 views

Predicting the observations in a POMDP with a recurrent neural network

I use neural networks for online sequence prediction. The performance of LSTM in this case, however, is not nearly as good as I expected. Maybe someone can help me understand where the problem lies. ...
4
votes
0answers
509 views

traditional state-space models and LSTMs

I am trying to understand the nature of LSTMs in relation to intuitions from traditional state-space models (e.g., Kalman filtering). The code below aims to simulate a simple univariate linear state-...
3
votes
2answers
624 views

Comparing clustering of sequences in datasets with different N?

When doing sequence analysis using a package such as TraMineR, one can calculate a clustering based on Optimal Matching (OM) distances, and then plot it as a tree. ...
3
votes
2answers
696 views

Measuring entropy for event sequences?

In TraMineR the function seqient() for calculating the entropy of a sequence object is not available for event sequences (as ...
3
votes
2answers
205 views

How to interpret p-values in output of seqecmpgroup()?

The seqecmpgroup() function returns a table that, among other things, include p-values for each identified subsequence. How should these be interpreted? I can see ...
3
votes
1answer
1k views

Comparing two sequence objects?

Using TraMineR, is it possible to compare two sequence objects to calculate the discrepancy between them? By this I mean not comparing two sequences but two sets of ...
3
votes
1answer
280 views

How to measure multichannel distances between “event” sequences?

In TraMineR, seqdistmc is used to measure multichannel distances between "state" sequences. I am wondering if there is a function to measure multichannel distances between "event" sequences.
3
votes
1answer
1k views

Entropy of state distribution, complexity index and turbulence for sequences of varying length

I have a dataset of varying sequence lengths and want to calculate descriptive measured for it. The main cause of differing lengths is that some cases are right censored, and are dealt properly when ...
3
votes
1answer
219 views

Sampling from sequence data

I work with 4 sequence datasets that I want to compare. The datasets range from roughly 150 to 2000 sequences. I've been able to use TraMineR on datasets up to 1000 sequences long, but when I try to ...
3
votes
1answer
309 views

Duration weighted Optimal Matching Algorithm using R

Besides the ado provided by Brendan Halpin for Stata, is there a way to pursue an Optimal Matching using the R-package TraMineR which weights OM’s elementary ...
3
votes
1answer
519 views

Why use K-medoids for sequence analysis?

In the package WeightedCluster there seems to be facilities for using K-medoids clustering (i.e. wcKMedoids()), but not the more ...
3
votes
1answer
444 views

Optimal matching distances

If I have two sequences: 1: A-B-C-D 1: X-A-B-C-D and I do: library(TraMineR) seqdist(sequence1, sequence2, method = "OM") ...
3
votes
2answers
922 views

Mining patterns in continuous sequence

I have data in form of $N$ sequences $s_j=(t_i, e_i)_{i\in\{1,\ldots,n_j\}}$ with $n_j$ data-points each, where $t_i$ is a time-stamp and $e_i$ is a categorial event, say $e_i\in\{A,B,C,D\}$. The $N$ ...
3
votes
1answer
120 views

Weightining using TraMineR

I have read some posts on weighting. However, I am still unclear on the sort of weights I need to use. I am using data from the Longitudinal Survey of Australian Youth (LSAY). This survey provides ...
3
votes
1answer
153 views

TraMineR: predicting class membership of new sequences

Question summary This question pertains to analyzing dissimilarities between discrete state sequences (e.g., using TraMineR), more specifically to classifying new, ...
3
votes
1answer
2k views

Pattern identification in time series data

Imagine a set of users who always perform the same set of tasks and in the same order. Say - user A - enters the store, reads the newspaper, looks at the TV, orders coffee, eats a donut and leaves. ...
3
votes
3answers
3k views

pattern recognition for sequence data

There are a lot of data sequences, I am trying to find pairs of sequences that are similar with other. Trivially, we can define some distance measure, and compare each pair of sequence in terms of ...
3
votes
1answer
2k views

Analyzing Sankey diagrams statistically?

Sankey diagrams are helpful for visualizing multiple, interacting processes: Are there any statistical tools available that can analyze the interactions of multiple processes? I'm aware of sequence ...
3
votes
1answer
921 views

Mean and Variance of a Set of Sequences

I am trying to find the probability distribution (like mean and variance) of a set of sequences which have different length. I came into different measurement models that quantify the distance between ...
3
votes
1answer
98 views

What is the difference between emerging and discriminative patterns?

I'm working on a way to make efficient classification on sequential patterns mining algorithms. Looking in the state-of-the-art I found that there is two type of algorithm to address this problem : <...
3
votes
0answers
118 views

Which distance metric to use to cluster categorical sequences (clickstreams or clickpaths)?

For my research, I want to cluster website visitors based on their clickstreams to understand different information behavior patterns (i.e., customer/visitor journeys). The data can be characterized ...
2
votes
3answers
218 views

Central limit theorem for sum of differences

I have a quick question. If we have a sequence of i.i.d random variables, $X_1,X_2,X_3,\ldots,X_n$. Then, we make sequence below: $$ S=X_1-X_2+X_3-X_4+\ldots+X_n $$ Does central limit theorem work in ...
2
votes
1answer
3k views

How can p=1 in Fisher's exact test?

I'm comparing two datasets from DNA sequencing studies, and comparing mutation rates in genes between the two datasets, which I'm doing using a two-tailed Fisher's exact test (please correct me if I'm ...
2
votes
1answer
3k views

what is “target delay” in the context of RNN/LSTM?

I'm currently reading a paper on sequence-to-sequence classification (speech-to-text) and it is stated that "there are ways to introduce future context, such as adding a delay between the ...