When speaking about the similarity for fine-tuning there are several points to consider:
The classes of the 2 datasets - How similar are they? If your original dataset was of dogs and cats and now you move to a dataset of wolfs and tigers, than they are very similar and very deep features are still relevant (so you should only train 1 or 2 of the last layers). But if your new dataset contains cars and cows, you will need to train more layers because less of them are relevant.
The inner-variance of the classes - Even if your new dataset contains the same classes, you need to consider their inner-variance. A model that was trained to distinguish between frontal images of cat and dog faces will need to learn new features to recognize one of them from the side or back.
The number of classes - A model that was trained on a dataset with very few classes compared to the new one, will need to learn more fine-grained features (so you will need to train more layers).
The similarity/complexity of the task - If your model was trained for semantic segmentation and now you want to use it for classification, it won't be too hard. But going the other will be.
In general, the deeper you go in the model, the more complex and specific are the features. The first layers are usually very simple and can generalize well between datasets and tasks (usually first layers are mostly about gradients, colors and textures).
Regarding on how to decide if your datasets are similar, this is a question that is usually answered subjectively, so you will have to answer it yourself. If you want something more official, you can try to analyze the 2 datasets in terms of the output of different layers in the model, comparing the mean and variance. At some point there should be some noticeable difference (note that I just made up this technique, so it might not help).