threshold cutoff value from ROC for test set evaluation, do I use the cutoff from test ROC or training ROC Let's say I am doing logistic regression.
I split my data into training and test.
I get an ROC for my training data and it has a cut-off of 0.25
I calculate my evaluation metrics, let's say just specificity and get 70%.
Then I do an ROC for my test data, and it has a cut-off of 0.4
If I use the cutoff value from training ROC (of 0.25) I get a specificity of 55%, but if I use cutoff from test ROC (of 0.4) I get specificity of 68%.
Which one do I use and why?
 A: This can be viewed as parameter estimation and it's usually done on a dedicated validation set, i.e. you split your data into three sets A, B and C then train the model on A, choose an appropriate cut-off using predictions on B and estimate the final generalization on C. 
A: Please tell us from what source you learned this procedure.  It is entirely wrong and represents bad statistical practice.  Here are some of the things that are going wrong:


*

*Logistic regression is used to estimate the probability of an event, not for classification

*Data splitting as a validation method (whether using 2 or 3 subsets) only works when you start with an enormous sample size, otherwise it is volatile

*ROC curves are not compatible with decision making - see the Diagnosis chapter of BBR

*ROC curves should not be used for obtaining a cutoff value - see the Information Loss chapter of BBR as well as http://www.fharrell.com/tags/classification and http://www.fharrell.com/post/mlconfusion/
Optimal decisions are made by taking the risk estimate from logistic regression and combining it with the utiity/loss/cost function, to maximize expected utility.  If you dichotomize the risk estimate before applying it to the utility function, the resulting decision will not be optimal.  Except for that, everything else I've mentioned here is covered in my Regression Modeling Strategies book and course notes.
