I was reading through some notes online, and I came upon a property that I don't know how to prove. This is the property:
E [ R(ERM(D)) - R'(ERM(D), D) ] >= 0
where D is a sample, ERM(D) is the empirical risk minimisation given D, R(ERM(D)) is the true risk of ERM, and R'(ERM(D), D) is the empirical risk of ERM on D.
Can anyone prove why this is the case?