6
$\begingroup$

I have two variables: ordering & length. The former measures the ordering of a sequence (i.e. all permutations of A-B-C), and the former is the length of the sequence (i.e. A-B-C has a length of 3). These are highly correlated, and I want to normalize the ordering measure by length. I was expecting this normalization to completely eradicate the correlation - but it doesn't. How can this be the case?

id      order   length  order/length
X1      4           3   1.333333333
X33     2           1   2
X566    44          6   7.333333333
X681    4           2   2
X682    46          6   7.666666667
X80     2           1   2

correlation before normalization: 0.958
correlation AFTER normalization: 0.610

The correlation has been reduced, but the variables are still highly correlated. My ambition was to partial out the component of "order" that is separate from "length", but it doesn't seem like I'm achieving that here. How can I do this? Where has my thinking gone wrong?

Improve this question
$\endgroup$
6
  • 2
    $\begingroup$ You need to include an intercept if you want to achieve zero correlation and you need to use least squares for the fitting. Instead of order/length that would give some multiple of order - 8.981366 * length (plus an arbitrary additive constant). $\endgroup$
    – whuber ♦
    Jan 23, 2014 at 20:47
  • $\begingroup$ But, the number of possible orderings in a sequence of length $n$ is $n!$. Of course, if you normalize by that, you get a correlation of -.86 between order and order/(length!). What does order actually measure? Is it the number of different permutations of some sequence that actually appear in some stream of text or data or something? $\endgroup$
    – Bill
    Jan 23, 2014 at 22:27
  • $\begingroup$ @Bill, I don't think this is central to the question, but I'm using the concept of "subsequences" from Elzinga, 2010, Complexity of Categorical Time Series. $\endgroup$
    – histelheim
    Jan 23, 2014 at 22:32
  • $\begingroup$ @whuber, could you provide a bit more context - I'm not sure I follow your comment. $\endgroup$
    – histelheim
    Jan 24, 2014 at 21:42
  • 2
    $\begingroup$ Least squares fitting of order versus length: that's how you remove correlations among variables. Because you're using Traminer, you have R, so try it: lm is your friend here. $\endgroup$
    – whuber ♦
    Jan 24, 2014 at 22:07

2 Answers 2

Reset to default
4
$\begingroup$

Instead of normalizing by the length, I would suggest to normalize by the maximum possible number of subsequences for the given length $\ell$.

The maximum is reached for a sequence made of the repetition of the alphabet. E.g., if the alphabet is {A,B,C}, the maximum for a sequence of length 6, would be the number of subsequences of A-B-C-A-B-C.

An upper bound of this maximum is $2^\ell$. It is the maximum as long as $\ell$ does not exceed the size of the alphabet.

Improve this answer
$\endgroup$
5
$\begingroup$

Admittedly, this is just an operationalization of @whuber's comment (thanks!), but I still find it helpful to document it.

normalization <- function(order, length){
  data <- as.data.frame(cbind(order, length))
  model <- lm(order ~ length, data)
  order_normalized <- order-(model$coefficients[1]+(model$coefficients[2]*length))
  (order_normalized)
}

If you input the two variables into the function, it will normalize the former by the latter using least-squares.

Improve this answer
$\endgroup$
1
  • $\begingroup$ well, would not we get the same result (normalization) simply by resid(model)? Whats being done here you just deduct the dependent variable from fitted. $\endgroup$
    – Maximilian
    Jul 11, 2018 at 6:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.