I am working my way through Multitrait-Multimethod Matrix in a psychometrics class. We're only required to be able to analyze them but I'd really like to be able to construct them. I think I am able to do that (basically it's a rearangement of the correlation matrix with coefficient alpha for the diagonals). I am pretty sure I got it but want a data set and a finished MTMM example (something that looks like the format below):enter image description here

That way i can work my way through and make sure my final product looks like it's supposed to. I've attempted for a number of hours to find something but am frustrated and hoping someone has such a resource. I use R so i don't care what format the data set is in.

Thank you in advance for your help.

  • $\begingroup$ Does this help? comisef.wikidot.com/tutorial:correlation or here: uvm.edu/~dhowell/StatPages/More_Stuff/Gener_Correl_Numbers.html $\endgroup$ – Michelle Mar 10 '12 at 2:43
  • $\begingroup$ Cool display, Tyler. Can you post R code that you used to create this? $\endgroup$ – StasK Mar 10 '12 at 23:08
  • $\begingroup$ @StasK This was not created in R (though I'm sure could be) I posted it more to look at the set up of the MTMM (what goes where). $\endgroup$ – Tyler Rinker Mar 10 '12 at 23:15
  • 1
    $\begingroup$ You may want to also ask @chl what source he pulled the above picture from (in this post on the site). You never know maybe the authors made have stated how they made the picture. You could do it in a spreadsheet program such as excel as well. $\endgroup$ – Andy W Mar 11 '12 at 13:23

I worked on this some the other day when you posted your same question to stack overflow. What I will provide won't be a finished solution, but hopefully it will give you enough ideas to finish the presentation on your own.

enter image description here

This is what I could produce in SPSS, I have posted some code here using the same logic in R with ggplot2, but I'm not as familiar with ggplot2 as I am with SPSS so it is a bit off from producing something as close to floor ready as I can with SPSS.

As I said in the comment to your SO post, in a grammar of graphics style you can refer to the methods as panels (facets or small-multiples are other synonyms) and the traits as defining location along the X and Y axis. Even though they are nominal categories (so there order is arbitrary) we can still treat them the say way we do continuous variables in a scatterplot. That is, we can assign observations X and Y locations in a cartesian coordinate system defined by the categories.

So the shape your data needs to be in (in either SPSS or R) to produce this graphic is as follows (this is a read data statement for SPSS, but this should be readily transferable to a variety of languages).

data list free / Method_X Method_Y Traits_X Traits_Y (4A1) Corr (F3.2).
begin data
1 1 a a .89
1 1 a b .51
1 1 b b .89
1 1 a c .38
1 1 b c .37
1 1 c c .76
1 2 a a .57
1 2 b a .22
1 2 c a .09
1 2 a b .22
1 2 b b .57
1 2 c b .10
1 2 a c .11
1 2 b c .11
1 2 c c .46
2 2 a a .93
2 2 a b .68
2 2 b b .94
2 2 a c .59
2 2 b c .58
2 2 c c .84
1 3 a a .56
1 3 b a .22
1 3 c a .11
1 3 a b .23
1 3 b b .58
1 3 c b .12
1 3 a c .11
1 3 b c .11
1 3 c c .45
2 3 a a .67
2 3 b a .42
2 3 c a .33
2 3 a b .43
2 3 b b .66
2 3 c b .34
2 3 a c .34
2 3 b c .32
2 3 c c .58
3 3 a a .94
3 3 a b .67
3 3 b b .92
3 3 a c .58
3 3 b c .60 
3 3 c c .85
end data.

Now, for my graph I want to define one more variable (the variable used to color the blocks) and add some meta-data which propagates to the graph in SPSS.

value labels Method_X Method_Y
1 'Method 1'
2 'Method 2'
3 'Method 3'.

compute type = 0.
if method_x = method_y and traits_x = traits_y type = 1.
if method_x = method_y and traits_x <> traits_y type = 2.
if method_x <> method_y and traits_x = traits_y type = 3.
if method_x <> method_y and traits_x <> traits_y type = 4.

value labels type
1 'reliability'
2 'validity'
3 'heterotrait-monomethod'
4 'heterotrait-heteromethod'.

Now the fun part, generating the graph. SPSS's graphics language, GPL, is not as intuitive as what Hadley has written for ggplot2, but I can help break it down some. Basically everything is superfluous for our discussion here except the GUIDE statements and below (so just focus on those for now).

  /GRAPHDATASET NAME="graphdataset" VARIABLES=Traits_Y Traits_X Method_Y Method_X corr type
  SOURCE: s=userSource(id("graphdataset"))
  DATA: Traits_X=col(source(s), name("Traits_X"), unit.category())
  DATA: Traits_Y=col(source(s), name("Traits_Y"), unit.category())
  DATA: Method_Y=col(source(s), name("Method_Y"), unit.category())
  DATA: Method_X=col(source(s), name("Method_X"), unit.category())
  DATA: type=col(source(s), name("type"), unit.category())
  DATA: corr=col(source(s), name("corr"))
  GUIDE: axis(dim(1), null())
  GUIDE: axis(dim(2))
  GUIDE: axis(dim(3), opposite())
  GUIDE: axis(dim(4))
  SCALE: cat(dim(2), reverse())
  SCALE: cat(aesthetic(aesthetic.color.interior), map(("1", color.black), ("2", color.darkgrey), ("3", color.lightgrey), ("4",color.white)))
  ELEMENT: polygon(position(Traits_X*Traits_Y*Method_X*Method_Y), color.interior(type), label(corr))

The element statement in essence specifies where things go in the plot (and what gets assigned what colors and labels. In this example the variable Traits_X gets mapped to the x axis (dim(1)), Traits_Y goes to the y axis dim(2), Method_X gets mapped to the panels going horizontally dim(3), and Method_Y gets mapped to the panels running vertically dim(4). Everything else just has to do with aesthetics in the plot (what gets what color and what label goes where).

Not all chart elements are directly exposed in SPSS syntax (you often have to hack the chart template to have certain aspects produced in a particular manner), but post-hoc editing gets you along ways in this instance. The extent I am able to reproduce the above chart (without going to great extremes) is inserted at the beginning of the question.

Two things I cannot do in SPSS (without doing hacky things with inserting text boxes on my own) are superscripts/subscripts and having label text in different colors (the labels are there, just black). These are things I would just print the graph to PDF and edit some more in Inkscape or Illustrator. I know you can do subscripts and superscripts in R labels, but one thing to note is this would break the grammar I have previously provided, as the categorical Y axis change between panels.

I could do the dashed boxes fairly easily in SPSS's editor (as well as the other text), but the arrow I could not. I know you wanted a solution in R, and I'm sure most of this logic can be ported to R code (using whatever packages you want).

A note, in some of the comments it appears Tyler and me were confused about what MTMM is (at least I was). This page by David Kenny goes into more detail on what the method is and how to estimate such models.

  • $\begingroup$ thank you. I will try this later when I get home. Just one quick note this was not the same question as on SO. On SO I asked how to do it. I never really got a response there but after asking friends I think I figured it out. The question here was for a data set and a finished product to test what I think will work. I avoid cross posting unless I explicitly state I've done so. $\endgroup$ – Tyler Rinker Mar 11 '12 at 13:44
  • $\begingroup$ I'm still not quite sure what the differences between the questions are (are you saying you want someone to do an exploratory factor analysis for you?). If you figured it out yourself though you should post an answer (either here, at SO or both). $\endgroup$ – Andy W Mar 11 '12 at 14:34
  • $\begingroup$ I think I figured out how to do it in R(that's what the question on SO was about) but I can't be sure because I don't have a data set and know worked example (what this question is about) to check the method I'm using in R. Once I confirm that the method works (Using the information garnered from this question) I will go back to SO and post the R method for constructing a MTMM (answer my own question).Essentially this question here was "I think I got a way to do this but I need a known data set and outcome to make sure the method produces the MTMM and not a random matrix. $\endgroup$ – Tyler Rinker Mar 11 '12 at 17:01
  • $\begingroup$ The MTMM is just an exploratory factor analysis with the elements in the estimated correlation matrix arranged in a particular order. You didn't re-write a program to conduct EFA did you? So I thought all you needed help with was producing the plot. Any observed correlation matrix and the "known" estimated correlation matrix from an EFA should do (unfortunately it appears the correlation matrix I used was just made up by Campbell in the original article). $\endgroup$ – Andy W Mar 11 '12 at 17:14
  • 1
    $\begingroup$ (+1) The R code looks promising! I will look further into it when I have time. $\endgroup$ – chl Mar 11 '12 at 19:45

I built on @Andy W's R-code and hope my changes are useful to someone else.

I mainly changed it, so that it

  • obeys the new syntax (no more opts) in ggplot2, so no more warnings
  • adds the correlations as text
  • now correlation text size reflects its effect size
  • colour scheme shows the type of correlation (hetero/mono-trait/method).
  • put the legend in the empty upper right triangle

The function also contains my way for creating the data in the right format from a dataframe or correlation table. This depends on having your trait and method encoded in the variable name and you'd probably want to extract CFA loadings for a more solid look at the matter. In my case I first wanted to eyeball the correlations with a bit more visual structure. If you have your correlations/loadings in long format already it should be easy to adapt the function or to cast the long to wide.


I put this in a package on Github. You can get it using devtools::install_github("rubenarslan/formr"), the function is then formr:mtmm.


## function for rendering a multi trait multi method matrix
mtmm = function (
    variables, # data frame of variables that are supposed to be correlated
    reliabilities = NULL, # reliabilties: column 1: scale, column 2: rel. coefficient
    split_regex = "\\.", # regular expression to separate construct and method from the variable name. the first two matched groups are chosen
    cors = NULL
) {
    library(stringr); library(Hmisc);   library(reshape2); library(ggplot2)

        cors = cor(variables, use="pairwise.complete.obs") # select variables

    var.names = colnames(cors)

    corm = melt(cors)
    corm = corm[ corm[,'Var1']!=corm[,'Var2'] , ] # substitute the 1s with the scale reliabilities here
    if(!is.null(reliabilities)) {
        rel = reliabilities
        names(rel) = c('Var1','value')
        rel$Var2 = rel$Var1
        rel = rel[which(rel$Var1 %in% var.names), c('Var1','Var2','value')]
        corm = rbind(corm,rel)

    if(any(is.na(str_split_fixed(corm$Var1,split_regex,n = 2)))) 
		print(unique(str_split_fixed(corm$Var1,split_regex,n = 2)))
		stop ("regex broken")
	corm[, c('trait_X','method_X')] = str_split_fixed(corm$Var1,split_regex,n = 2)  # regex matching our column naming schema to extract trait and method
    corm[, c('trait_Y','method_Y')] = str_split_fixed(corm$Var2,split_regex,n = 2)

    corm[,c('var1.s','var2.s')] <- t(apply(corm[,c('Var1','Var2')], 1, sort)) # sort pairs to find dupes
        corm[ ,'trait_X']==corm[,'trait_Y'] 
        & corm[,'method_X']!=corm[,'method_Y']),'type'] = 'monotrait-heteromethod (validity)'
        corm[ ,'trait_X']!=corm[,'trait_Y'] 
        & corm[,'method_X']==corm[,'method_Y']), 'type'] = 'heterotrait-monomethod'
        corm[ ,'trait_X']!=corm[,'trait_Y'] 
        & corm[,'method_X']!=corm[,'method_Y']), 'type'] = 'heterotrait-heteromethod'
        corm[, 'trait_X']==corm[,'trait_Y'] 
        & corm[,'method_X']==corm[,'method_Y']), 'type'] = 'monotrait-monomethod (reliability)'

    corm$trait_X = factor(corm$trait_X)
    corm$trait_Y = factor(corm$trait_Y,levels=rev(levels(corm$trait_X)))
	corm$method_X = factor(corm$method_X)
	corm$method_Y = factor(corm$method_Y,levels=levels(corm$method_X))
    corm = corm[order(corm$method_X,corm$trait_X),]
    corm = corm[!duplicated(corm[,c('var1.s','var2.s')]), ] # remove dupe pairs

    #building ggplot
    mtmm_plot <- ggplot(data= corm) + # the melted correlation matrix
        geom_tile(aes(x = trait_X, y = trait_Y, fill = type)) + 
        geom_text(aes(x = trait_X, y = trait_Y, label = str_replace(round(value,2),"0\\.", ".") ,size=log(value^2))) + # the correlation text
        facet_grid(method_Y ~ method_X) + 
        ylab("")+ xlab("")+
        theme_bw(base_size = 18) + 
        theme(panel.background = element_rect(colour = NA), 
                    panel.grid.minor = element_blank(), 
                    axis.line = element_line(), 
                    strip.background = element_blank(),
                    panel.grid = element_blank(),
                    legend.position = c(1,1),
                    legend.justification = c(1, 1)
        ) + 
        scale_fill_brewer('Type') +
        scale_size("Absolute size",guide=F) +


data.mtmm = data.frame(
    'Ach.self report' = rnorm(200),'Pow.self report'= rnorm(200),'Aff.self report'= rnorm(200),
    'Ach.peer report' = rnorm(200),'Pow.peer report'= rnorm(200),'Aff.peer report'= rnorm(200),
    'Ach.diary' = rnorm(200),'Pow.diary'= rnorm(200),'Aff.diary'= rnorm(200))
reliabilities = data.frame(scale = names(data.mtmm), rel = runif(length(names(data.mtmm))))
mtmm(data.mtmm, reliabilities = reliabilities)
  • $\begingroup$ This code looks very interesting. Would it be possible for you to post your "imput" dataset somewhere so I could play with it? I can't quite visualize what you are doing with all of your which/melt statements. $\endgroup$ – Twitch_City Oct 10 '13 at 19:54
  • 1
    $\begingroup$ @Twitch_City I added some sample data and put the code into a function. You can specify a custom regular expression to control how variables are split into method and trait. Hope this helps. $\endgroup$ – Ruben Oct 11 '13 at 8:57

It looks like I forgot to link to the original resource I used to construct this picture, that was used as an illustration for an old course (I tend to prefer B&W pictures :-). I know nothing about the data, and that was not of primary interest at the time I used it (it was done with Omnigraffle for Mac).

If the question is about how to reach such figures, you can try to generate correlation matrices on your own, using the excellent psych package. (Be sure to check William Revelle's website.) However, for well-established data you could probably refer to

Brown, TA (2006). Confirmatory Factor Analysis for Applied Research. The Guilford Press.

See data for Table 6.1. Some context (pp. 214-216):

In this illustration, the researcher whishes to examine the construct validity of the DSM-IV Cluster A personality disorders, which are enduring patterns of symptoms characterized by odd or eccentric behaviors (American Psychiatric Association, 1994). Cluster A is comprised of three personality disorder constructs: (1) paranoid (an enduring pattern of distrust and suspicion such that others' motives are interpreted as malevolent); (2) schizoid (an enduring pattern of detachment from social relationships and restricted range of emotional expression); and (3) schizotypal (an enduring pattern of acute discomfort in social relationships, cognitive and perceptual distortions, and behavioral eccentricities). In a sample of 500 patients, each of these three traits is measured by three assessment methods: (1) a self-report inventory of personality disorders; (2) dimensional ratings from a structured clinical interview of personality disorders; and (3) observational ratings made by paraprofessional staff. Thus, Table 6.1 is a 3 (T) x 3 (M) matrix, arranged such that the correlations among the different traits (personality disorders: paranoid, schizotypal, schizoid) are nested within each method (assessment type: inventory, clinical interview, observer ratings).

The result should look like this:

enter image description here

If you are using R, you might be interested in looking into the mtmm() function from the psy package (which can be used to assess convergent and discriminant validity within a single measurement instrument as well), as already mentioned in earlier replies of mine: How to compute correlation between/within groups of variables?, Which package to use for convergent and discriminant validity in R?

  • $\begingroup$ Well on behalf of your students (and everyone else) I thank you for not using the original colors in the picture! $\endgroup$ – Andy W Mar 11 '12 at 22:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.