I have trained a neural network on DNA sequences data and my training set has exactly the same number of data in both classes. When I select a softmax function at the end, my accuracy remains at 47% and loss for both validation and training stays the same at around 7.6 regardless of how many batches and epochs I choose. But once I change the softmax function to sigmoid, the validation accuracy starts at 50% for the first epoch and reaches above 98% at the end which is odd cause I think at best my network should achieve an accuracy of around 80% since I know some of my data is misclassified. Why is this happening?
Using sigmoid with dummy encoded output (one binary column) vs using softmax with two one-hot encoded columns (one of the columns is equal to one, the other is zero) is mathematically equivalent and should give same results. Your likely have a bug in the code.
Softmax should work better for classification than sigmoid (with 2 output features in both cases). I'm guessing you have a bug in your code somewhere.
If you're getting better accuracy than you believe is reasonably possible, then you are probably overfitting and either have a model which is too complicated or (perhaps more likely) have too little data.