Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I have one sample and several features. I calculate a z-score for various features, and for various combinations of features. Is there a way to quantify the significance of the difference between two z-scores?

For example, suppose I calculate a z-score for the combination of features (male, homeowner) and also for the combination (male, homeowner, married). For one scenario, suppose the first z-score is 3.0, and the second z-score is 6.0. For a different scenario, suppose the first z-score is 25.000, and the second z-score is 25.001. I'm trying to determine whether or not (male, homeowner, married) is a better predictor than (male, homeowner). Is there a generally applicable formula to quantify the significance of the difference, perhaps taking into account the sample size?

I have no training in statistics, and am trying to implement a software program. A cookbook formula accompanied by some beginner-appropriate explanation would be ideal and greatly appreciated.

Thanks

share|improve this question
    
It would be helpful to have more detail on your outcome/dependent variable, especially whether it is a continuous or categorical variable. My other question is why are you calculating z-scores? Do you mean that you are getting a z-statistic as a hypothesis test about overall model fit? There will be answers to your question, but more detail will help people answer as there are many, many potential pitfalls for what you want to do. –  James Stanley Jan 7 '13 at 22:24
    
Currently, both dependent and independent variables are dichotomous, although I would like to also handle the case when one or the other is continuous. And yes, I am using the z-test for hypothesis testing for the selected feature as a predictor. –  Tyro Jan 7 '13 at 23:01
    
Anything? Your comment sounds promising. –  Tyro Jan 13 '13 at 23:24
    
It's probably a question of reading up on model selection methods -- I don't have time to write an answer (which would need to be quite long), but I could recommend in the first instance Frank Harrell's "Regression Modelling Strategies." –  James Stanley Jan 14 '13 at 21:52
    
From a statistical perspective, when comparing two model fits in logistic regression, you might look into likelihood ratio tests. The main validity concerns I have are around using automated selection of predictors -- see stats.stackexchange.com/questions/18638/… and linked questions for more points on this. –  James Stanley Jan 14 '13 at 21:53
show 2 more comments

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.