When training an autoencoder (softmax in outputlayer), for DNA sequence compression (by minimizing a cross-entropy loss function), how can you calculate the number of bits per base required for the storage of a DNA sequence file?
Let's say that we have a file, containing of
N bases/samples. After decoding, each reconstructed base will yield a probability vector
p_i of size 4. We thus have
N probability vectors:
P = [p_1...p_N]
Is it sufficient to say that the average bits per base needed, for storing
N bases is equal to:
(1/N) * sum_N( log2(1/max(p_i) )
max(p_i) is the highest probability seen in the softmax output for a given reconstructed base?
Finally, how can this average BPB be related to the number of hidden units (or codeword lengths) in the bottleneck layer of an autoencoder? If more hidden units in this layer are used, I would expect a better reconstruction (i.e. higher probabilities for correct bases) at the end of the decoder part while less compression occurs (since longer codewords are used).