Is a constant ever inadmissible? For now, assume square loss. Let's estimate some parameter $\theta$, such as $\theta = \mu$ in $N(\mu, 1)$.
Is there ever a case where there is no such $c$ to make $\hat{\theta} = c$ an admissible estimator of $\theta$? (Bonus question: What about if we relax the assumption of square loss?)
I lean toward no, at least for square loss. If we decompose mean squared error, we get the bias-squared and variance of $\hat{\theta}$.
$$
\operatorname{MSE}(\hat{\theta}) = \mathbb{E}\big[(\hat{\theta} - \theta)\big]^2 + \operatorname{var}(\hat{\theta}). 
$$
For the constant $c$, the variance is zero, and the bias is zero when $\theta = c$, perhaps a ridiculous notion in applied statistics but completely legitimate in mathematical statistics. Thus, $\operatorname{MSE}(\hat{\theta}) = 0$.
I struggle to see how any other estimator could match $\hat{\theta} = c$ when $c = \theta$.
 A: YES: HERE IS AN EXAMPLE OF INADMISSIBLE CONSTANT ESTIMATOR
Assume a heads-heads or tails-tails coin; we just do not know which. If we guess whichever face comes up after the flip, that is a better way of guessing than always saying it is heads (or tails) regardless of the outcome of the flip.
Let $X\sim \text{Bernoulli}(p)$, with $p\in\{0, 1\}$. That is, we flip a coin that we know is heads-heads or tails-tails; we just do not know which. Let's estimate $p$. (That is, let's determine if the coin is heads-heads or tails-tails.)
My proposed admissible constant estimator would be $\delta_0(X) = 0$ or $\delta_1(X) = 1$. Both of these have $\text{MSE} = 0$ when $p = 0$ and $p = 1$, respectively, but each has $\text{MSE} = 1$ when $p=1$ and $p=0$, respectively.
However, $\delta(X) = X$ is another estimator. That is, look at the coin. If the coin comes up heads ($1$), the coin is heads-heads. If the coin comes up tails ($0$), the coin is tails-tails.
$$\text{bias}(\delta) = 0 \text{  }\text{  }\text{  } \forall  \text{  }\text{  }\text{  } p\in\{0, 1\}$$
$$ \text{var}(\delta) = p(1-p) = 0 \text{  }\text{  }\text{  } \forall  \text{  }\text{  }\text{  } p\in\{0, 1\}$$
$$\text{MSE}(\delta) = (\text{bias}(\delta))^2 + \text{var}(\delta) = 0 \text{  }\text{  }\text{  } \forall  \text{  }\text{  }\text{  } p\in\{0, 1\}$$
Thus, $\delta$ beats $\delta_0$ when $p=1$ and beats $\delta_1$ when $p=0$, and $\delta$ ties $\delta_0$ when $p=0$ and beats $\delta_1$ when $p=1$.
Since we restrict $p$ to $p\in\{0, 1\}$, $\delta_c(X) = c$ will have bias for any $c$ but $c\in\{0,1\}$, so those lose to $\delta$, despite their zero variance.
$\delta$ achieves the interesting feat of being unbiased and having zero variance for all $p\in\{0,1\}$. The constant estimators all have bias for some $p\in\{0,1\}$. Therefore, there is no $c$ such that $\delta_c(X) = c$ is admissible.
I liked what Xi'an posted in the comment and wanted to expand on it in a self-answer.
