There are two things you want to be mindful of:
- Potential bias/systematic error: Observers systematically under or over-estimating the difference.
- Reliability/random measurement error: Observers disagreeing randomly on any particular measure while, on average, none of them reports systematically higher or lower measures.
A paired sample t-test would in fact be appropriate to show that there was a difference of the first kind between the two observers but turning it on its head and taking the lack of significant difference as implying that they provided similar measures is not. The main reason for that is that the power of the test (and it's complement, the type II error rate) is hugely dependent on the sample size.
With a large sample, a tiny discrepancy between observers will be significant and you would be reluctant to aggregate the two measures for no good reason. With a small sample, you could be unaware of large discrepancies. This does not seem very useful. One way to actually test if the discrepancy is not too big is called “equivalence testing”. You would need to define beforehand what “too big” means for you, i.e. how much of a discrepancy you are willing to tolerate. Even so, the power of the test will depend on the sample size but at least the hypothesis being tested is formulated in a way that matches your question.
Note than none of this directly addresses the issue of reliability. It's perfectly possible for observers to provide sets of measures with the same mean and yet have individual measures with a lot of random error. This would also call the whole measurement process into question as it would suggest that observers are unable to agree on the evolution of any particular lesion or that additional observers would be necessary to obtain reliable measures. One way to look at that would be to do a scatterplot/look at the correlation between the two sets of measures.
At the end of the day, I am not sure you need a test. There is no absolute threshold for bias or reliability that could tell you that the measurement is good enough for any purpose and no reason why any small detectable discrepancy should prevent you from doing anything else. In fact, taking the average from several measures is especially useful when these measures are themselves unreliable.
What I would rather do is use plots: side-by-side stripcharts, boxplots or density plots for potential bias or differences in the distribution of measures coming from each observer, scatterplot for the reliability/noise in the individual measures.