Usual non-constructive mathematics leads to some paradoxes (e.g. the Banach-Tarski paradox), which are directly related to the axiom of choice. In non-constructive mathematics, the axiom of choice (as well as proofs by contradiction) are not accepted.
For statistics, the axiom of choice implies that for any $c$ there must be a model such that the BIC (or also the AIC) is below $c$ (see this answer, we can arbitrarily increase the likelihood without increasing $N$, the number of parameters), which kind of defeats the purpose of the BIC as a model selection criterion. This method would not work in constructive math, because the space-filling curves or bijections don't exists in constructive mathematics.
Is there a general treatment of statistics in constructive mathematics? AFAIK a lot of parts of statistics are non-constructive (e.g. Gaussian distribution, Beta distribution, etc). It would be interesting to see which parts of statistics are different when the constructive approach is taken.
It seems that my understanding of constructive mathematics is still to limited for this. So I may have been wrong about the existence of the distributions or the space-filling curves in constructive mathematics. I will edit this question, once I get a better understanding. Feel free to edit, if you can shed more light on this topic.