Let's consider a critic $Q_\theta(s,a)$ implemented as a simple MLP with multiple hidden layers and some sort of nonlinear activation after each layer.
Instead of thinking of the critic as one block, we can decompose it into a feature extractor $\phi(s,a)$ comprising everything except the last layer, plus the final linear layer $w$, i.e. $Q_\theta(s,a) = w^T\phi(s,a)$.
Why do they use separate networks with scalar output as opposed to a single network with vector output per sample, where, as a last step, minimum is taken across this vector?
Isn't this equivalent, but more efficient?
When using double critics, we typically train them from scratch with two different random initializations, i.e. both critics will have separate feature extractors $\phi_1(s,a)$ and $\phi_2(s,a)$, as well as separate final layers $w_1$ and $w_2$.
What you're considering by having a vectorial output would mean having two output vectors [$w_1$,$w_2$] with the same, shared feature extractor $\phi(s,a)$, so it is definitely not equivalent.
The motivation for the double critics is to have two different "independent" estimators that "balance each other out" (in a very vague sense). Having two different outputs achieves this somewhat, but they are less "independent" than having two entirely different models.
I would encourage you to try it out in practice, it will definitely save compute and even if it performs a bit worse it would still be an interesting finding!
