KL divergence between an uninformative (?) Gaussian and a Gaussian I have to calculate the KL divergence between a distribution $q$ and a prior distribution $p$, both of which are univariate Gaussians, i.e. $KL(q|p), q \sim \mathcal{N}(\mu, \sigma^2), p \sim \mathcal{N}(\mu', \sigma'^2)$. This term is part of a larger formula, which is justified in some way not relevant to this question.
Now, to be honest, I don't want to put any asusmptions on $p$ except that it is Gaussian. My intuition is to just say that if I do that, I can just say $p=q$ and thus $KL(p|q) = 0$.
I wonder if there is a way to phrase this intuition into proper math. I read about non-informative priors, but found nothing about calculating KL divergences in this scenario.
Update:
I try to make the question more clear.
I have two distributions. One is the prior, the other driven by data. My prior is not tied to any parameter ranges (e.g. in form of a conjugate prior), but only specified in its functional form (i.e. disitrbution family). In typical Bayesian frameworks, such a thing is called an informative prior. Does the same concept exist for KL based objective functions?
 A: The answer to your previous question about this topic was
$$KL(p, q) = \log \frac{\sigma_2}{\sigma_1} + \frac{\sigma_1^2 + (\mu_1 - \mu_2)^2}{2 \sigma_2^2} - \frac{1}{2}$$
where you were using the notation $\sigma_1=\sigma, \sigma_2=\sigma^\prime$.  To make the mildest possible assumption on $p$ you must take the supremum of $KL(p,q)$ as the parameters of $p$ range through all possible values; that is, as $\mu_2$ ranges through all real numbers and $\sigma_2$ ranges through all positive numbers.  But note that this expression can be made arbitrarily large.
Rigorously, let $N \gg 0$ be any positive real number.  Then by setting $\sigma_2 = \sigma_1 \exp(N+1/2)$,
$$KL(p,q) = \log\frac{\sigma_1\exp(N+ \frac{1}{2})}{\sigma_1} + \cdots - \frac{1}{2} = N + \cdots \gt N$$
where the omitted term "$\cdots$" is obviously positive.  Therefore the supremum is $+\infty$.
Similar analysis (by finding the minimum of $0$ at $\sigma_2=\sigma_1$ and $\mu_2=\mu_1$ and observing that the divergence is a continuous function of its arguments $\sigma_2$ and $\mu_2$) shows that $KL(p,q)$ can attain all real values in the interval $[0,\infty)$.  Without making any further assumptions on $p$, that is all that can be said.

Graph of $KL(p,q)$ for $p\sim \text{Normal}(\mu_2,\sigma_2)$ and $q\sim \text{Normal}(0,1)$.  The vertical and standard deviation scales are logarithmic.
