Can quadratic constraints be handled by Bayesian methods? I want to solve a regression method whose parameters are under quadratic constraint a'*a=1. Is there any method in Bayesian statistics to handle this constraint? Thank you in advance. 
 A: I don't know of a success story, but I can think of an approach based on reparametrisation and variational inference. It is a long story short and quite complicated.
Let's say your parameters are  $\theta$. You can then obtain a variational lower bound on the marginal log-likelihood 
$$
\mathcal{L} = \log \sum_\theta p(x|\theta) p(\theta) \\
= \log \sum {q(\theta) \over q(\theta)} p(x|\theta) p(\theta) \\
\geq \sum q(\theta) \log  {p(x|\theta) \over q(\theta)} p(\theta)\\
= \mathbb{E}[\log p(x|\hat\theta)]_{\hat\theta \sim q} - KL(q(\theta)||p(\theta))
$$
Now, if you can i) sample from $q$ and ii) calculate the KL between $q$ and the prior, you can train this objective with stochastic gradient-based techniques, i.e. stochastic gradient descent or, even better, rmsprop or adadelta.
What you now need is to make sure that $q$ is only a distribution over values satisfying the constraint. You can achieve that either by


*

*Reparametrising $a$ in a way that each $\theta$ represents a "valid" $a$,

*Projecting each sample to the set of orthogonal matrices.


The first can be achieved by performing an Eigendecomposition $A = UDU^T$ where you a) have to make sure that all elements of D are either $1$ or $-1$. You will also have to orthogonalize $U$.
The latter can be achieved by SVD and setting the singular values to 1.
A: The paper The Kernel Gibbs Sampler by Graepel and Herbrich (2001) describes how to do Gibbs sampling over a sphere.  They had a special likelihood function that gave piecewise constant conditionals, but a similar approach should work for any likelihood, using e.g. a slice sampler to sample along a great circle.
