2
$\begingroup$

I have two datasets of MRI images: a larger one of Altzheimer's paitents (AD), which is about 3 times the size of a smaller dataset of brain tumor paitents (BT). My aim is to make use of the AD data with some form of transfer learning to help with the classification of brain tumor images.

Question: Would it be viable to train a feature extractor unsupervised (likely a stacked convolutional autoencoder) using only the AD data, and then use the encoder as the first few layers of a new network which has fully-connected layers trained on the BT data?

My thinking is that it may be possible to have the autoencoder 'learn' the general features of the brain from the AD data, and then use this as a basis for a BT network. I have not seen this done before (which maybe for a good reason).

Perhaps there is some issue with overfitting/general transferability between the datasets. Would fine-tuning the encoder when the fully-connected layers are trained possibly prevent this?

$\endgroup$
2
$\begingroup$

Yes, this is a totally viable solution and probably it would work.

However in my experience, even CNNs trained on very generic data (e.g. imagenet) that has nothing to do with MRIs can be used as pretrained models and achieve very good enough results. The reason, probably, is that the features they learn to extract in their initial layers (i.e. lines, circles, corners, etc.) are general purpose and can be used for virtually any task.

This has two benefits: 1) you don't have to pretrain the model on your own; you can download weights online and 2) the models are trained on a dataset which most likely is an order of magnitude (or two) larger than the one you're trying to use (which means that it is trained better).

$\endgroup$
1
  • $\begingroup$ Thanks for your answer. Using ImageNet (or something similar) has been at the back of my mind given the number of times people have used it in the past for similar applications. You are right about the dataset size difference too, so I will for sure look into this. $\endgroup$ Jul 16 '19 at 19:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.