# Nathan VanHoudnos

less info
reputation
4
bio website stat.cmu.edu/~nmv location Pittsburgh, PA age member for 2 years, 4 months seen Nov 19 at 2:45 profile views 19

I'm a PhD student in the joint Statistics and Public Policy program at Carnegie Mellon University.

# 15 Actions

 Oct10 awarded Scholar Oct10 accepted Does there exist a model fit statistic (like AIC or BIC) that can be used for absolute instead of just relative comparisons? Oct10 accepted What are typically encountered condition numbers in social science? Oct10 answered What are typically encountered condition numbers in social science? Aug9 awarded Supporter Aug9 comment What are typically encountered condition numbers in social science? @StasK So then the short answer to my question is: No. People do not normally compile these things because they are specific to both the particular model chosen and the particular data onto which the model is fit. To talk of a "typical" number is not well posed. Aug9 comment What are typically encountered condition numbers in social science? @whuber Thanks for the comment. (1) My goal is understanding sample size via condition numbers. (2) In a simple 2-level hierarchical model, with indep. groups and equal correlation within groups, then bounding the size of the largest group bounds the condition number (e.g. adding more "schools", but not more "students per school"). (3) From my (limited) understanding, condition numbers are proxies for the expected numerical error. My naive assumption is that different fields will have orders of magnitude differences (e.g. IQ tests v. diameters of tree trunks). Do you believe that to be false? Aug9 awarded Editor Aug9 revised What are typically encountered condition numbers in social science? improved formatting, clarified that my definition of a condition number is non-standard Aug9 asked What are typically encountered condition numbers in social science? Jul22 comment Does there exist a model fit statistic (like AIC or BIC) that can be used for absolute instead of just relative comparisons? @whuber Apparently It only lets me notify one person, so please see my response above about my specific case. Your point is well taken about $R^2$ and the context of the data. Jul22 comment Does there exist a model fit statistic (like AIC or BIC) that can be used for absolute instead of just relative comparisons? @drknexus I have 3 different exams that were given to 3 different groups of people and scored by 3 different sets of raters. I fit a few models (e.g. linear, Rasch, hierarchical Bayesian) on each set of exams separately. Within the same exam data-set I can compare among the models using AIC/BIC, but I can't compare across the data-sets. For example, on exam 1 Bayesian beats Rasch, but on exam 2 Rasch beats Bayesian. Is that because of poor fit (and high AIC/BIC variance) or because of good fit and there is something different about the two exams. Jul22 awarded Student Jul22 comment Does there exist a model fit statistic (like AIC or BIC) that can be used for absolute instead of just relative comparisons? @whuber Wow, that's an awesome response to the $R^2$ question! But, its inadequacies aside, $R^2$ is used to say that their model is "good" in an "absolute" sense (e.g. "My $R^2$ is such-and-such which is better than what one normally sees..."). I'm looking for a more justified (and general) statistic than $R^2$ to accomplish the same purpose (e.g. "My MagicStatistic is such-and-such which is better...). My first naive thought was to do something like normalizing a k-fold cross validation score, but it doesn't seem like anyone has done such a thing (so its probably not a good idea). Jul22 asked Does there exist a model fit statistic (like AIC or BIC) that can be used for absolute instead of just relative comparisons?