I have a set of grayscale images, some of them are transformed of the other images. For example in 10 images, image 2 is the same as image 8 but rotated, and image 4 is the same as image 7 but translated. There might be a slight distortion but they look mostly the same. And lastly, there could be more than two copies of the same image, but it would be the exact same shape with slight distortion (nothing like cats and dogs).
I tried the image similarity using historgram but it can lead to very large erros and is not reliable. Also convolutional neural networks need to be trained beforehand. I'm looking for an unsupervised method because the data are not labeled, different datasets have different contents and the number of images are too small for a neural network.
I don't expect you to explain the whole method (I would appreciate it though), just looking to find a direction to do my own reading.