If there is a sufficient statistic, then it contains all the information that a sample has about the distribution.
Other statistics that describe the sample are superfluous.
To expand, I might be able to come up with a better estimator for $\theta$ which takes on a strange functional form, and maybe then the observed value of the sufficient statistic will not be enough to allow me to infer about $\theta$ in the way that I wish.
If any estimation method that requires the additional statistics would be better, then it is despite of it and not because of it. We could replace that estimation method by a method that is dependent on the sufficient statistic only and have the additional statistics randomly generated and independent from the actual sample.
Example
The example is a simple case with a normal distribution. It gives a simple example of how the Fisher Neyman factorisation theorem represents a sample basically as composed of two independent groups of statistics, one that depend on the distribution parameters, and one that do not depend on the distribution parameters. The statistics from the latter group can be replaced with surrogate data without influencing the behaviour of the estimation method.
If we sample a distribution that follows a normal distribution $X_i \sim N(\mu,1)$ then we can replace a sample $X_1, X_2, \dots, X_n$ with surrogate data that depends on the original sample only via the sufficient statistic, the sample mean $\bar{X}$ $$Y_i = \bar{X} + \epsilon_i$$
where $\epsilon_i \sim MVN(0,\Sigma)$ is multivariate distributed and the covariance matrix is based on the residual maker matrix.
With this, any inference method on $X_i$ should work just as well with $Y_i$. So any inference method can be made to depend on the sufficient statistic only.
Computational: with the code below we simulate the situation above and compute the sample $X_i$ as a normal distributed sample and $Y_i$, the surrogate data, based on only the mean of the sample. Then we perform some inference that depends on more than the sufficient statistic by computing the median and see how the methods differ based on a histogram. The result is that both methods result in the same distribution. It shows how the inference with a method that is not based on the sufficient statistic can be turned into a method that is based on the sufficient statistic with equivalent performance. The performance is equivalent no matter what measure of performance you use because the sample distributions of the estimates are exactly the same.
set.seed(1)
mu = 5
n = 10
sim = function() {
### true sample
x = rnorm(n,mu,1)
### sufficient statistic
xbar = mean(x)
### computations for epsilon
### independent from original sample
vmu = rep(0,n)
vx = rep(1,n)
sigma = diag(rep(1,n)) - vx %*% solve(t(vx) %*% vx) %*% t(vx)
epsilon = MASS::mvrnorm(1,vmu,sigma)
### surrogate data
### only depending on true sample
### via sufficient statistic
y = xbar + epsilon
return(c(median(x),median(y)))
}
sims = replicate(10^5, sim())
### plot histogram
hist(sims[1,], main = "regression based on original data", xlab = expression(hat(mu)), breaks = seq(2.5,7.5,0.1))
hist(sims[2,], main = "regression based on surrogate data \n with only sufficient statistic as input", xlab = expression(hat(mu)), breaks = seq(2.5,7.5,0.1))
Does this make alternatives useless? No!
The above is not to say that methods that do not depend on the sufficient statistic are useless. Those other methods can be more robust to situations where the assumptions about the distribution are wrong (and the assumptions about the distribution being wrong has implications such as what the sufficient statistic is).
A lot of inference is also not performed with an explicit distribution in mind, sometimes people want to estimate a property of a distribution (like the mean or median) without an explicit definition of that distribution (and without a sufficient statistic).
The sufficient statistic also doesn't contain all the information. It is the sufficient statistic plus the assumed true distribution that describes the population. When this assumption is not present or wrong then the sufficient statistic is also not containing all the information that the sample contains about the parameters or statistics that describe a population distribution.
