Given that your two metrics are 1) binary and 2) heavy tailed, you should avoid t-test which assumes normal distributions.
I think Mann-Whitney U is your best choice and should be sufficiently efficient even if your distributions were near-normal.
Regarding your second question:
What happens if one test suggests significant difference between cohorts and some other test suggests non-significant difference?
This is not uncommon if the statistical difference is borderline and the data has "messy" sample distributions. This situation requires the analyst to carefully consider all the assumptions and limitations of each statistical test, and give the most weight to the statistical test which has the least number of violations of assumptions.
Take the assumption of Normal distribution. There are various tests for normality, but that's not the end of the story. Some tests work pretty well on symmetric distributions even if there is some deviation from normality, but don't work well on skew distributions.
As a general rule of thumb, I'd suggest that you should not run any test where any of its assumptions are clearly violated.
EDIT: For the second variable, it might be feasible to transform the variable into one that is normally distributed (or at least close) as long as the transform is order-preserving. You need to have good confidence that the transform yields a normal distribution for both cohorts. If you fit the second variable to log-normal distribution, then a log function transforms it to a normal distribution. But if the distribution is Pareto (power law), then there is no transformation to a normal distribution.
EDIT: As suggested in this comment, you should definitely consider Bayesian Estimation as an alternative to t-testing and other Null Hypothesis Significance Testing (NHST).