In the application of statistical methods in social science, one usually does a lot of robustness checks. If I got some publishable findings using LRT test by discrimination two theoretical models, what robustness checks can I do?
I found this paper but it seems like only for biostatistics.
I found that the previous works in political science and economics usually seem to not do robustness checks for their results obtained through LRT (I am not sure about all of them, but from a limited publication that I read they did not do robustness check). However, as a student, how to convince the referees that a simple LRT test without any robustness check is enough?
Robustness checks involve reporting alternative specifications that test the same hypothesis. Because the problem is with the hypothesis, the problem is not addressed with robustness checks.
Common robustness checks for OLS results in social sciences include adding other control variables and testing the hypothesis in a subset of the sample or another sample (out of sample test). For testing the effect of a policy using time series data, the difference in difference analysis is usually used as robustness check: we need to make sure that the trend is caused by the policy instead of a pre-existing trend. In a quasi-experiment or field experiment, we need to make sure that the treatment group and control group are "exactly the same"; therefore we do a "matching" between each treated individual and each control.
In general, a robustness check usually involves additional control, additional data, or an alternative method of testing the same hypothesis, etc.
Of course, the robustness of LRT can be confirmed by some out of sample test. I am already doing this. Some different methods of testing the same hypothesis that LRT can test would be helpful too but I think it is standard to use LRT. If it is two non-nested model, we could use Vuong test and Clarke test.
So mostly I am not sure how could we "add control power" to LRT test or any other method to test the robustness of a LRT test.
(BTW my advisor does OLS for his whole life. He has no idea about MLE).
