I found this interesting paper regarding deep unsupervised clustering and am looking to mimic some of the things done, however there is one thing that is not clear to me. In the paper, they use a softmax layer to get the probabilities of the latent vectors belonging to a certain cluster, however there is one thing that I seem to miss with using softmax.
Let's say that we wanted to use a sigmoid activation function for our encoding layers as described in the papers. If I understand correctly, doing this causes the outputs to be between 0 and 1, meaning that the latent vectors will be vectors with entries between 0 and 1. The problem with this is that with values between 1 and 0, the softmax function by its nature cannot give a probability, for $k=2$ (where $k$ is the number of clusters), greater than $$\frac{e}{e + \frac{1}{e}} = 0.88079707797...$$ The problem here is that it seems to me like the softmax function should be able to 'reach' values much closer to 0 and 1 as these are probabilities.
My original thought was that I need to normalize the data in some way, but I am not sure if this is necessary here or where I would normalize the data in the network.
I also know that softmax is supposed to get log probabilities and 'undo' them by exponentiating, and because I don't think my program uses logs anywhere, I think the issue may be here.
I think there also could be some fundamental misunderstanding I have with this whole problem, so if anybody could explain why this is happening and what I can do to avoid this problem I would be very thankful.