Putting prior on a function of parameters Suppose that we have a likelihood for a conditional distribution $p(y|X,\theta)$. For clarity purposes we can consider linear regression with homescadastic errors. It is clear to me how one will put a prior on the parameters, i.e. $\pi(\theta)$.
Instead of doing that can we put a prior $\pi(\mu)$ over $\mu = g(X,\theta)$ where $g$ is an inverse link function for the mean of $p(y|\mu)$ and estimate $\theta$? Does it make sense? Is there a name for this procedure?
 A: Another perspective on this: Yes, of course you can do that, the only question in practice is how you do it. 
There's several options:


*

*You can work out the changes in variables etc. and explicitly define the implied priors on the untransformed variables. This is often impractical / requires a lot of work.

*If you can define your model in terms of the function of the variables, you can directly apply the prior and just work with the transformed variables.

*You can use software that lets you specify such a prior and deals with the rest automatically (well, for non-linear transformations, you may have to figure out the determinant of the Jacobian of the transform). One example is Stan, in which you can e.g. specify something like the following:
parameters {
  vector[4] beta;
}
model {
  sum(beta) ~ normal(0,1);
  ...
The last approach is what I would use in practice. The only downside is that it may be hard to understand the implied prior on the original parameters, but to explore that, you can sample without data (and look at the resulting prior on the parameters, as well as the prior predictive distribution).
As a side remark, this is also very useful for overparameterized models for introducing additional (soft) constraints. E.g. if you have a categorical outcome and 3 parameters $\theta_i$ and your model is $Y_j \sim \text{Categorical}(\text{softmax}(\boldsymbol{\theta}))$, then the $\theta_i$ are only identifiable up to a constant. That actually tends to result in sampling problems, that you can get rid off in this manner.
A: As I understand your question, I don't know weither this approach has a name ; expect maybe those of change-of-variable.
Indeed, by defining a prior on $g(\theta)$, you are simply defining a prior on $\theta$ (assuming $g$ is monotonic, or bijective in multidimensional setting).
More formally, using change of variable relationship gives:
$$
p(\theta|y)  \propto p(y|\theta) \cdot p_{\theta}(\theta) \\
\propto  p(y|\theta) \cdot p_{\theta}(g^{-1}(\mu)) \\
\propto  p(y|g^{-1}(\mu)) \cdot p_{\mu}(\mu) \cdot \frac{\partial}{\partial \mu}(g^{-1}(\mu))
$$
As you can see, the equivalence between the two formulations (one in $\theta$ and one in $\mu$) involves a third term that must be accounted for to pass from one formulation to another. 
Finally, consider two points : first, if $g$ is not bijective, the inverse function is not defined and thus you have no way to go back from $\mu$ to $\theta$. Second, regarding your initial formulation -that included the regressors $X$- all I wrote above is still consistent if your replace $\theta$ by $\theta^{'}=[\theta,X]$.
A: When you can not compute a posterior for $\mu$ because of missing information in the likelihood
Below is a counterexample for a case where the likelihood function $p(y\vert \mu)$ is not uniquely defined by $\mu$, but depends in a more complex way on the vector $\theta$.

Say you have the likelihoodfunction:
$$Y \vert \theta_1+\theta_2 \sim N(\theta_1+\theta_2,1) \quad \text{with} \quad\theta_1 + \theta_2 \sim N(0,1)$$
Now this is like
$$Y \vert \mu \sim N(\mu,1) \quad \text{with} \quad\mu_{prior} \sim N(0,1)$$
and after observing value $Y$ the posterior becomes:
$$\mu_{posterior \vert Y} \sim N(1/2 Y,1/2)$$

In the above you can factor out the different $\theta$ from the likelihood function in a single function $\pi (X,\theta_1, ... , \theta_n) = \pi^\prime(g (X, \theta_1, ... , \theta_n)) = \pi ^\prime(\mu)$. That works fine. But say we have now instead some more complicated likelihood function:
$$Y \vert \theta_1+\theta_2 \sim N(\theta_1+\theta_2,\theta_1^2) \quad \text{with} \quad\theta_1 + \theta_2 \sim N(0,1)$$
Now this is like
$$Y \vert \mu, \theta_1 \sim N(\mu,\theta_1) \quad \text{with} \quad\mu_{prior} \sim N(0,1)$$
Which includes an extra (meta-)parameter $\theta_1$ in the likelihood function and we can not (independent from that meta-parameter) express $\mathbb{P}(Y,\mu)$ from which we could compute the posterior for $\mu$.
After observing a value $Y$ the posterior for $\mu$ becomes*:
$$\mu_{posterior \vert Y} \sim N( cY,d)$$ with $c = \frac{1}{\theta_1^2+1}$ and $d = \frac{\theta_1^2}{\theta_1^2+1}$
So you can not solve the posterior of $\mu$ for that problem because you have this metaparameter on which it depends as well.
When you can not compute $\theta$ based on $\mu$
This problem is a bit like Is it possible to derive joint probabilities from marginals with assumptions about the conditionals? Your distribution for $\mu=g(\theta)$ is turning a joint distribution for multiple variables (if $\theta$ is a vector) into a distribution for a single variable (some 'sort' of marginal distribution). You can not invert this
Say in the first example above. You might be able to find the distribution for $\mu = \theta_1 + \theta_2$ like $$\mu_{posterior \vert Y} \sim N(1/2 Y,1/2)$$ But it is not possible to turn this into a joint distribution for $\theta_1$ and $\theta_2$ without more information.

*This can be derived by the joint distribution $f(y,\mu\vert \theta_1) = \frac{1}{2 \pi \theta_1} e^{-\frac{1}{2} \left(\frac{(y-\mu)^2}{\theta_1^2}+ \mu^2\right)}$
