I agree with @user918967 that you have wrongly formulated your problem, but I believe your misformulation involves ethical as well as technical deficiencies. In particular, your focus on an 'estimator' of overall survival in the face of two putatively distinct classes of patient seems perverse. Please consider how you might design your trial to acknowledge the ethical demands of adapting to the information stream that comes in as you perform the trial. For example: if it should turn out that the 'late responders' actually do worse on the treatment (compared with some available alternative), have you designed your study to detect this as soon as possible? Or would such a finding emerge only belatedly, in retrospect, after you had reached some fixed, predetermined sample size ?
As to the technical shortcomings of your problem misspecification, I believe you might find it fruitful to consider your 2 putative classes of response in light of the concept of frailty; see  and . In frailty modeling, a latent 'frailty' variable is presumed to be distributed in some way across a population, and this distribution typically morphs over time as frailer individuals 'exit' first. Your concept of 'early' and 'late' responders suggests a bimodal frailty, as if response were a monogenic phenotype. But you should consider whether you might be leaping to 'dichotomize' what is actually a more continuous phenomenon, and frailty modeling provides a formalism for such consideration.
One thing you might like to do is to preregister your study protocol on a platform like F1000Research, where you can get transparent feedback, and iteratively update your protocol (through multiple versions, if necessary) until it passes peer review.
Note 1: Your query about using "doctor's perception or [patient's] age" points very much toward frailty-type phenomena. The surgeon's 'eyeball' impression of a patient's suitability for surgery is often discussed as delivering an implicit assessment of frailty that may not be readily accessible to clinical measurement.
Note 2: As with "survival", the term "frailty" does not necessarily carry a fixed positive or negative connotation in every application. Thus, if you were modeling time-to-event for an event like clinical response, then of course the 'frailty' involved would be the vulnerability of the disease process to the treatment, and in this case 'frailty' would be a good thing, and 'survival' (i.e., longer duration of the disease) would be a bad thing.
: Berry, Donald A. “Bayesian Statistics and the Efficiency and Ethics of Clinical Trials.” Statistical Science 19, no. 1 (February 2004): 175–87. doi:10.1214/088342304000000044. http://projecteuclid.org/euclid.ss/1089808281
: Aalen, O. O. “Effects of Frailty in Survival Analysis.” Statistical Methods in Medical Research 3, no. 3 (October 1, 1994): 227–43. doi:10.1177/096228029400300303. http://journals.sagepub.com/doi/pdf/10.1177/096228029400300303
: Aalen, Odd O., Ørnulf Borgan, and S. Gjessing. Survival and Event History Analysis: A Process Point of View. Statistics for Biology and Health. New York, NY: Springer, 2008. (See especially Chapter 6, titled "Unobserved heterogeneity: The odd effects of frailty".)
: Jain, Renuka, Sue Duval, and Selcuk Adabag. “How Accurate Is the Eyeball Test?: A Comparison of Physician’s Subjective Assessment versus Statistical Methods in Estimating Mortality Risk after Cardiac Surgery.” Circulation. Cardiovascular Quality and Outcomes 7, no. 1 (January 2014): 151–56. doi:10.1161/CIRCOUTCOMES.113.000329. http://circoutcomes.ahajournals.org/content/7/1/151.long