Is there a distribution $p$ that I can sample from, such that for $\epsilon \sim p$, and for closed-form deterministic function $g_{\mu,\kappa}$,
$g_{\mu, \kappa}(\epsilon) \sim \mathrm{vonMises(\mu, \kappa)}$,
for any $\mu\in[-\pi,\pi]$ and $\kappa > 0$?
If not, is there a circular distribution other than von Mises with these properties, that has a closed-form density function?
Simulation for the von Mises distribution is generally done via some form of rejection sampling. There is no method available to transform a random variate from a different distribution to a von Mises random variate in the way you describe. A natural way would have been some form of inversion sampling, but the CDF of the von Mises distribution is not analytic, so this may not be possible.
Two distributions similar to von Mises that may be of interest to you are the Wrapped Normal and Wrapped Cauchy distribution.
For the Wrapped Normal, we can simply take $X \sim N(\mu, \sigma^2),$ then $\Theta = X ~ \text{[mod} ~ 2\pi]$ to have $\theta \sim WN(\mu, \rho),$ where $\rho = e^{-\frac{1}{2} \sigma^{2}}.$
For the Wrapped Cauchy with parameters $\mu$ and $\rho$, get a random variate $u$ from $\text{Uniform}(0, 2 \pi),$ then $$ V = cos (u)$$ $$ c = 2\rho / (1+\rho^2)$$ $$ \theta = \cos^{-1}\frac{V + c}{1 + cV} + \mu ~~ \text{[mod} ~ 2\pi].$$ Then $\theta \sim WC(\mu, \rho)$. This procedure is due to Fisher (1995).
