Other substitution matrices for missing value state in sequence analysis with TraMineR?

Question

We have a question about how to deal with missing values/gaps within sequences. We like to set up our own substitution-cost matrix for the Optimal Matching process. As far as we know, TraMiner allows creating own cost matrices - but only in case there are no missing values. If there are missing values there is a default matrix which 'charges' 2 units for all states regardless of handling with 'real' states or the missing state (cf. TraMiner User Guide, chapter 6.5.2 & 9.4.5).

Background: Right now, we evaluate a life-course dataset with career progression which lasts up to 40 years. Matthias Studer of the TraMiner-Development Team kindly gave us the advice trying the Multiple Imputation Method of Brendan Halpin (see his paper here). As Mr. Halpin pointed out with good reason, it will produce bias tendencies setting missing values as a special category and assigning own substitution cost (as the TraMiner User Guide advises by indicating the problems of this operation, cf. Ibid. chapter 9.4.5). Therefor Multiple Imputation makes sense. Unfortunately we have to deal with much bigger amount of gaps than Brendan Halpin, so it will be good to try both and compare, the substitution and the imputation method. Mr. Halpin confirmed so after having checked our data.

So far, using the default 2-units-for-all-substitution of the substitution method the clustering results shows the frequence distribution/state distribution plots are beeing clustered with tendency to the pattern of the 'missing state'. That means that the cluster types obey the 'missing state' pattern which is no good result. So we like to try other substitution cost substitution for all of the states including the missing state. But how to do so?

Answer 1

You are right, to compute "OM" dissimilarities with missing states you need substitution costs for replacing missing values. However, this is exactly what the TraMineR seqdist function expects.

The seqdist help page states: "If the OM method is selected, seqdist expects a substitution cost matrix with a row and a column entry for the missing state (symbol defined with the nr option of seqdef)."

An easy way to define such a matrix respecting the correct order of the alphabet augmented with the state state, is to first create such a matrix with for example

sm <- seqsubm(yourseq, method="CONSTANT", with.missing=TRUE)

and then replace the content of the matrix with your wanted costs before passing it to seqdist. You can also make use of the miss.cost argument to set a constant substitution cost for missing states.

As for the imputation of missing states, in addition to Brendan Halpin's nice multiple imputation solution, you could also consider exploiting the predictive capacities of Probabilistic suffix trees proposed in the just released Alexis Gabadinho's PST package.

Hope this helps.

Answer 2

thank you for answering. But as far as I see, my question isn't answered. Because I'm not even sure, whether I posted the way it should be, I try to ask again this way:

We tried this before the way Gilbert described it. Using "seqdef" (and seqsum before) there is indeed the opportunity defining 'real states' but not the gaps with defined index-costs, or am I wrong? Or the other way around: Is there a 'code' defining the gaps? It's because the custom set costs are only accounted for 'real states' (not the gaps) if you set submission costs like this: R> subm.custom <- matrix(c(0, 1, 1, 2, 1, 1, 1, 0, 1, 2, 1, 2, 1, 1, 0, 3, 1, 2, 2, 2, 3, 0, 3, 1, 1, 1, 1, 3, 0, 2, 1, 2, 2, 1, 2, 0), nrow = 6, ncol = 6, byrow = TRUE, dimnames = list(mvad.shortlab, mvad.shortlab))

Anybody can help? (if my question isn't understandable, please let me know)