Conjugate prior distribution for a specific (normal) curved exponential family Is there a conjugate prior distribution in the model $\text{N}(\theta, \sigma^2=\theta^2)$, that is a normal distribution where the mean and standard deviation are equal ($\theta>0$)? (This question is related to Conjugate for a special normal distribution)
 A: There is a conjugate distribution for $\theta$ in the model $X_1, X_2, \dotsc, X_n$ iid $\text{N}(\theta, \theta^2)$ with $\theta>0$, but it is not a commonly named distribution. First find the likelihood of this "normal parabola" model, as it is called:
$$
L(\theta) = \prod_i^n \frac1{\sqrt{2\pi}\theta}e^{-\frac12\sum_i^n \left(\frac{x_i-\theta}{\theta} \right)^2}
$$
and (leaving out constants) this is proportional to
$$
L(\theta) \propto \theta^{-n} e^{-\theta^{-2}\sum_i x_i^2/2 + \theta^{-1}\sum_i x_i}
$$
Then we need a prior distribution on $\theta$ that can "absorb" these three factors. Using a form of bayes' theorem,
$$
\pi(\theta | x) \propto f(x | \theta) \pi(\theta) \\
= \theta^{-n} e^{-\theta^{-2}\sum_i x_i^2/2} e^{\theta^{-1}\sum_i x_i}\pi(\theta)
$$
So we need a prior density having some corresponding factors. We can try with a form which is a particular generalization of a generalized inverse gamma density
$$
   \pi(\theta) = K^{-1} \cdot \theta^{-\alpha-1}e^{-\frac{\beta}{\theta}-\frac{\gamma}{\theta^2}},\qquad \theta>0
$$
which is convergent for $\alpha>0,\beta\in\mathbb{R}, \gamma>0$. The proportionality factor $K$ has a very complicated form which will be given at the end.  This distribution will have a finite expectation for $a>1$ and a finite variance for $a>2$. 
Using this we find that 
$$
\pi(\theta | x) \propto \theta^{-(n+\alpha)-1} e^{-(\beta-\sum_i x_i)/\theta } e^{-(\gamma+\sum_i x_i^2/2)/\theta^2}
$$
that is, the posterior has the same form as the prior. 
I calculated the  constant $K$ with maple:
int( (theta^(-a-1))*exp(-b/theta - c/theta^2), theta=0..infinity ) assuming a>0,c>0,b,real;

which gives the following result:
$$
   1/2\,{\frac {{c}^{-a/2}}{\sqrt {\pi}} \cdot \\ \left( 1/4\,{\frac {{\pi}^{2}b}{
\sqrt {c}\cos \left( 1/2\,\pi\,a \right) \Gamma \left( 2-a/2 \right) }
 \left( 1/2\,{\frac {{b}^{2}}{c}}-1+a \right) {\it LaguerreL} \left( 1
/2-a/2,1/2, \\ 1/4\,{\frac {{b}^{2}}{c}} \right) }-1/8\,{\frac {{\pi}^{2}{
b}^{3}}{{c}^{3/2}\cos \left( 1/2\,\pi\,a \right) \Gamma \left( 2-a/2
 \right) }{\it LaguerreL} \left( 1/2-a/2,3/2,1/4\,{\frac {{b}^{2}}{c}}
 \right) }+ \\ 1/2\,{\frac {{\pi}^{2}}{\sin \left( 1/2\,\pi\,a \right) 
\Gamma \left( 3/2-a/2 \right) } \left( 1/2\,{\frac {{b}^{2}}{c}}+1
 \right) {\it LaguerreL} \left( -a/2,1/2,1/4\,{\frac {{b}^{2}}{c}}
 \right) }- \\ 1/4\,{\frac {{\pi}^{2}{b}^{2}}{c\sin \left( 1/2\,\pi\,a
 \right) \Gamma \left( 3/2-a/2 \right) }{\it LaguerreL} \left( -a/2,3/
2,1/4\,{\frac {{b}^{2}}{c}} \right) } \right) }
$$
For the ${\it LaguerreL}$ function see https://en.wikipedia.org/wiki/Laguerre_polynomials. 
A: I wrote a paper in 1991 (!) about this problem, which got published in Statistics and Probability Letters. And which recoups Kjetil's solution as the generalised inverse Normal distribution, which generalises the distribution of the inverse of a Normal random variable (not to be confused with the inverse Gaussian distribution). The density is expressed as
$$K(\alpha,\mu,\tau)\,|z|^{-\alpha}\exp\{-(z^{-1}-\mu)^2/2\tau^2\}\qquad\alpha>1,\ \tau>0$$
The paper also contains a representation of the normalising constant using confluent hypergeometric functions
$$K(\alpha,\mu,\tau)=\tau^{\alpha-1}e^{-\mu^2/2\tau^2} 2^{(\alpha-1)/2}
\Gamma((\alpha-1)/2){}_1F_1((\alpha-1)/2,1/2,\mu^2/2\tau^2)$$
