For some estimator $\hat{\theta}$, we have the "bias-variance trade-off": $MSE(\hat{\theta}) = bias^2(\hat{\theta}) + var(\hat{\theta}).$
When I think of a trade-off, I would expect that as the variance goes down, the squared bias would go up, and vice versa. But I'm not sure this is the case here, since $MSE(\hat{\theta})$ is not fixed as $\hat{\theta}$ as varies.
Example: Consider two silly estimates for the mean of a single normally distributed data point $x \sim N(\theta, 1)$: $\delta_0(x) = 0$ and $\delta_1(x) = 1$. Both of these has the same variance of $0$. Furthermore, $bias^2(\delta_0) = \theta^2$ and $bias^2(\delta_1) = (1-\theta)^2$. Therefore, both estimators have the same bias but different squared biases (assuming $\theta \ne \frac{1}{2}$).
So in what sense is there a trade-off here?