I have a dataset with 119 features and 228 samples. I have trained my classifier using a particular set of labels (0 and 1). Now I want to test how the model is performing to external data. There is not a single dataset out there with the same labels (0 and 1). But my supervisor really wanted to look for external validation of our model. That prompted me to look of other test datasets with the same feature set but a different label. I found one such dataset with 55 features overlapping with my train set. But the labels for the new test set are 67% correlated with my train labels. Should I test my model on this new data?
you can, but it's not really testing the generalization of your model since you are not using it for the same problem.
People sometimes do it, for example, someone might fit a model to classify healthy and depressed people and then show that it also classify healthy and anxious people but not healthy and schizophrenic people. This is then followed by speculations about why is it or bold claims about some kind of magical transdiagnostic effects or other speculations about for what purposes that model can be used.