I want to test if the $\alpha$ for a scale is dependent upon a personality test, which is a continuous variable. I understand that there are many methods to test if two $\alpha$'s are significantly different for two different groups when they are given the same test. But what if the grouping variable is not dichotomous, but continuous? Should I just do a median or tertial split to artificially create separate groups and test if they have different $\alpha$'s? I've heard bad things about median split (like reducing power and whatnot), so I'm hesitant to use this method. Any suggestion?
There are certainly bad things about splitting the data; you may even have heard them from me, if you lurk here :-)
But in you case, you can't even compute alpha unless you split the group. That is, you can compute alpha for your whole sample, or any part of it, but you can't compute it for each person. So, splitting is necessary.