0
$\begingroup$

Can someone explain to me, how the explained / residual variance from the dissassoc command is computed? I tried to add up the variance within using the discrepancies of the different levels, but it didn't work. Let's take a small random sample (n=10).

data(mvad)
library(dplyr) 
set.seed(10)

mv = mvad %>% group_by(male) %>% sample_n(5)
mvad.seq <- seqdef(mv[,17:20])

# compute the dissimilarity matrix
mvad.ham <- seqdist(mvad.seq, method="HAM") 

# compute the discrepancy analysis 
d = dissassoc(mvad.ham, group = mv$male, R=10) 
d 
# Pseudo ANOVA table:
#         SS df      MSE
# Exp    1.7  1 1.700000
# Res    9.6  8 1.200000
# Total 11.3  9 1.255556

I understand how to compute the Total by hand:

sum(mvad.ham) / (2* 10) # = Total 11.3 

Questions:

  1. How do you compute the Exp and Res values by hand?
  2. Could you please demonstrate us how do you compute the Total Sum of Square?

I understand from (Studer et al., 2010) that the equation is:

$$ SS = \sum_{i=1}^{n} w_i(y_i - \bar{y})^2 $$

What does the $\bar{y}$ represents exactly? Could you demonstrate on this example how one can compute the $SS$ from mvad.ham manually?

$\endgroup$
4
  • 2
    $\begingroup$ @giacomoV, I voted to reject this edit. It is really the OP's job to provide an example of the problem they are having. We can't really know if this was it or something else. This thread should really just be closed. $\endgroup$ Commented Sep 30, 2016 at 17:45
  • $\begingroup$ @gung. I understand your point, but I am having exactly the same question and I dont want to ask it twice to professor Gilbert. Please do consider leave it open. I am 99% confident that it is the same question but the person didn't take the time to provide an example. I think this question is very important for the sequence community. Please again leave it open. $\endgroup$
    – giac
    Commented Sep 30, 2016 at 17:47
  • 1
    $\begingroup$ I politely disagree, @giacomoV. You should ask a new question, not hijack this one. If you ask a good question, it can get good answers (most likely better than what might happen here). Then we could close this as a duplicate of your new question. $\endgroup$ Commented Sep 30, 2016 at 17:50
  • 1
    $\begingroup$ @gung I posted my own question on the topic stats.stackexchange.com/questions/237792/… $\endgroup$
    – giac
    Commented Sep 30, 2016 at 18:10

1 Answer 1

2
$\begingroup$

We would need an example to see what you are making wrong. Nevertheless, from what you say I understand that you attempt to get the total within variance by adding up the variances of each group. It is the sums of squares $(SS = n \cdot var)$ that should be added up, not the variances.

The within SS of each group is computed the same way as the total SS. The sum of those within SS gives the residual SS.

The explained SS is then obtained as the difference between the total SS and the residual SS.

$SS_{exp} = SS_{tot} - SS_{res}$

I illustrate by building on your example:

(ss.tot <- sum(mvad.ham)/(nrow(mvad.seq)*2))
## [1] 11.3

ss.male <- sum(mvad.ham[mv$male=="yes",mv$male=="yes"])/(sum(mv$male=="yes")*2)
ss.female <- sum(mvad.ham[mv$male!="yes",mv$male!="yes"])/(sum(mv$male!="yes")*2)

(ss.res <- ss.female + ss.male)
## [1] 9.6

(ss.exp <- ss.tot - ss.res)
## [1] 1.7
$\endgroup$
3
  • $\begingroup$ I have the output: Pseudo ANOVA table: SS df MSE Exp 9372.421 9 1041.380151 Res 29832.871 9990 2.986273 Total 39205.293 9999 3.920921 I know, how i can reproduce the Total variance on my own by computing sum(example.dist)/(2*n): > (sum(org.om))/(2*10000) [1] 39205.29 But how can I reproduce the explained variance? Is there a way in R to do this like the total variance? $\endgroup$
    – user96982
    Commented Dec 3, 2015 at 14:36
  • 1
    $\begingroup$ I have completed the answer. Edit your question with your your example figures so that I can refer to them in the answer. $\endgroup$
    – Gilbert
    Commented Dec 4, 2015 at 6:31
  • $\begingroup$ @giacomo I just discovered that I didn't follow up on the provided example. This is now done. Sorry for the long delay. $\endgroup$
    – Gilbert
    Commented Feb 16, 2018 at 15:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.