Frank Harrell

8,416 reputation

1335

bio	website	biostat.mc.vanderbilt.edu/…
	location
	age
visits	member for	2 years, 2 months
	seen	8 hours ago
stats	profile views	2,384

	8,416 reputation	bio	website	biostat.mc.vanderbilt.edu/…	visits	member for	2 years, 2 months
1335 badges		location			seen	8 hours ago

summary answers questions tags badges favorites bounties reputation activity

736 Actions

suggestions reviews revisions comments badges posts accepts all

17h	comment	Why do Friedman and Wilcoxon post-hoc contradict each other for my data? @xmjx be sure not to make the "absence of evidence is not evidence for absence" mistake. You would need to base you "if there is a difference" assessment on confidence intervals excluding meaningful differences, not on large P-values.
19h	comment	Why do Friedman and Wilcoxon post-hoc contradict each other for my data? Note that the Friedman test in most situations has far less power than Wilcoxon-type tests. A generalization of the Wilcoxon test that is more suited to your situation is the mixed effects proportional odds model. But a quick and only slightly dirty approach is to use multiple Wilcoxon tests as you have done. On a more general note, even if the best available method resulted in a large P-value, the time was not wasted in the lab; it's just that many people are biased against "negative" studies. It is possible to learn a great deal from negative studies.
1d	comment	Test accuracy higher than training. How to interpret? It is useful to know the accuracy on the training set in order to estimate the amount of overfitting your predictive algorithm has done (by computing the difference in accuracy between training & test sets).
2d	answered	Test accuracy higher than training. How to interpret?
Jun 16	answered	Example of distribution where large sample size is necessary for central limit theorem
Jun 16	awarded	Nice Answer
Jun 15	comment	What are common statistical sins? In general it takes an enormous dataset for data splitting to be reliable. That's why stringent internal validation with the bootstrap is so attractive.
Jun 13	comment	Help me fit this non-linear multiple regression that has defied all previous efforts If you use linux I can give it to you now, otherwise expect 2 weeks.
Jun 12	comment	Help me fit this non-linear multiple regression that has defied all previous efforts Just do the binning temporarily unless we have the continuous solution. You can bin into 100 percentiles, e.g. `require(Hmisc); cut2(y, g=100, levels.mean=TRUE)`
Jun 12	answered	Help me fit this non-linear multiple regression that has defied all previous efforts
Jun 12	comment	Is it reasonable to compare a regression model with machine learing algorithms using RMSE? $R^2$ is another excellent measure, but if calculated the usual way allows for a linear recalibration of the predictions so does not penalize for predictions being off by a constant or a constant multiple.
Jun 12	answered	Is it reasonable to compare a regression model with machine learing algorithms using RMSE?
Jun 11	comment	Binary choice model and classification I would say that a probability model provides an estimate of Prob[Y=j \| X=x]. A probability is not used to estimate models. Probability estimates come from the model's estimated parameters. The notion of classification need not be involved, and requires specification of an additional function - the loss function. Those who do not specify a loss (utility) function are using one implicitly without knowing it. Probabilities can be used on their own by decision makers.
Jun 10	comment	Binary choice model and classification This has nothing to do with academia vs practice. Are you familiar with credit scoring models done in finance? Lift curves in business?
Jun 10	comment	Binary choice model and classification I do not agree that most users focus on the classification and not the probability estimates. A 0.5 threshold (and any other) is arbitrary and reflects a lack of incorporation of loss/utility/cost functions.
Jun 10	comment	R vs SAS, why is SAS prefered by private companies? Having used SAS for 23 years and S-Plus/R for 22 years I can say that a highly experienced SAS programmer can be highly productive, but that an experienced R programmer can be easily three times as productive.
Jun 9	comment	Binary choice model and classification I disagree sharply. The number one goal for the vast majority of problems is to estimate Prob[Y=1], not to do a forced choice between Y=0 and Y=1. And the entire method of maximum likelihood estimation, used to estimate the logistic regression coefficients, is based on this.
Jun 7	comment	Binary choice model and classification It is important to distinguish between the nature of Y (e.g., it is a continuous measure, an inherently dichotomous entity, or a classification of an underlying entity (usually information losing)) and the problem to be solved (prediction of the probability of a new Y or classification using X). The widely applicable method of logistic regression is a risk prediction method, which has many advantages over a "classify into a binary Y" method.
Jun 7	answered	Fisher's exact test for results of Wilcoxon-Mann-Whitney
Jun 6	comment	How to split a data set to do 10-fold cross validation Be sure to repeat the entire process 100 times to achieve satisfactory precision.