Concentration Bounds for categorial distribution with good Dirichlet prior I would like to know if there are any standard methods for analyzing the concentration bounds (for example Hoeffding's bound) for a multinomial distribution modelled with a Dirichlet prior, with the additional information that the prior is "good" in some sense.
In other words, if I had a "really close guess" compared to the true distribution parameters, I would like to know how that translates into improved concentration guarantees of sampling.
Are there any standard textbook results of this sort?
 A: TL;DR: You cannot improve the performance of the sample mean in a meaningful way by designing the prior.

I am going to go with the special case of the Bernoulli trials that OP used in their comments. The equivalent of the Dirichlet prior in this case is the Beta distribution. So we have
$$
X_1, \dots, X_n \mid \Theta = \theta \sim \text{Ber}(\theta)
$$
and $\Theta \sim \text{Beta}(\alpha, \beta)$. The posterior density is
$$p(\theta \mid X_1,\dots,X_n) \propto \theta^{n \bar X_n + \alpha - 1} (1-\theta)^{n(1-\bar X_n) + \beta - 1}$$
where $\bar X_n = \frac1n \sum_{i=1}^n X_i$ is the sample average. The the posterior is
$$\text{Beta}(n \bar X_n + \alpha, n(1-\bar X_n) + \beta)$$
and the posterior mean is
$$
\hat \Theta := \frac{n \bar X_n + \alpha}{n + \alpha + \beta} = \lambda_n \bar X_n + (1-\lambda_n) \mu 
$$
where $\mu := \frac\alpha{\alpha + \beta}$ is the prior mean and $\lambda_n := \frac{n}{n + \alpha + \beta} \in [0,1]$. In other words the posterior mean is the convex combination of the MLE, i.e., the sample mean $\bar X_n$ and the prior mean $\mu$.
Now suppose that the data is actually generated as $X_1,\dots,X_n \sim \text{Ber}(\theta_0)$ for fixed $\theta_0$ (say 0.6 as in OP's comments). What is the best case scenario for the prior? That the prior mean matches the true parameter, i.e., $\mu = \theta_0$, in which case the posterior mean is just equal to the sample mean: $\hat \Theta = \bar X_n$.
If $\mu \neq \theta_0$, then you incur a nonzero bias. Say if we care about the MSE, we have
$$
\mathbb E (\hat \Theta - \theta_0)^2 = (1-\lambda_n)^2 (\mu-\theta_0)^2 + \lambda_n^2 \frac{\theta_0(1-\theta_0)}{n}. 
$$
The first term is the price that you pay because the prior mean is off from the truth ($\theta_0$). The second term goes down like $O(n^{-1})$ which is the classical rate. It is the same rate Hoeffding would give and it is coming from the averaging that occurs in $\bar X_n$.

How to interpret a "very certain prior"? One way to interpret that is via the variance of the prior. The smaller the variance, the more certainty you have. Take $\alpha + \beta = m$. This is sometimes called the prior sample size; see the definition of $\lambda_n$; it adds to $n$, the actual data sample size, and looks like a sample size. Then, $\alpha = m\mu$ an $\beta = m(1-\mu)$ and the variance of the prior is
$$
  \frac{\mu(1-\mu)}{m+1},
$$
(notice the similarity with the binomial mean) and $\lambda_n = n / (n+m).$ A smaller prior variance can be achieved by increasing $m$, which then causes $\lambda_n$ to get closer to 0 and increase the price you pay for the mismatch between $\mu$ and $\theta_0$. So unless $\mu  = \theta_0$, the more certain you are a priori, the worse off you will be.
