I'm new to bayesian inference. I've just discovered that improper priors can't be specified in WinBUGS/OpenBUGS. I was wondering if this is common or not in bayesian inference.
Are there same cases in which it's possible to implement a improper prior? Suppose you have a improper uniform prior: $U(0,\infty )$. Do I always have to approximate this by a proper uniform $U(0,A)$ with $A$ large?
As noted in comments, Bayesian inference is distinct from BUGS or any other software implementation, and one cannot sample from an improper prior. (Pseudo-random number generators generally work by mapping a random number in $(0,1)$ to a cumulative distribution function, which is clearly not possible with an improper distribution.)
But, improper priors can lead to proper posteriors, and can lead to conditional densities from which we can sample. Thus, improper priors can be implemented when:
The first is model-specific and requires human judgment, thus making it an obstacle to implementing improper priors in general-purpose sampling software.
Bayesian Analysis for the Social Sciences gives a very nice explanation. Summarized, the theory behind the Gibbs sampler assumes than an invariant density actually exists, and with improper priors it may not. More, though
[...] a joint density is characterized by its conditional densities, the converse is not necessarily true: i.e. it is possible for the to be a set of mutually consistent conditional densities without the existence of a proper joint density[.]
More from an example on the following page:
With the improper, reference priors given above, it is known that this posterior density is improper (Hill 1965). Nonetheless, a Gibbs sampler can be implemented for this problem, since the requisite conditional densities are well-defined and easy to sample from. Moreover, the problem here is particularly pernicious in that nothing in the behavior of the Gibbs sampler for this problem would suggest that the posterior density is improper, and the results are meaningless.
Likewise, Bayesian Data Analysis recommends caution when dealing with diffuse priors in hierarchal models (p. 107-8):
If little is known about $\phi$, we can assign a diffuse prior distribution, but we must be careful when using an improper prior density to check that the resulting posterior distribution is proper, and we should assess whether our conclusions are sensitive to this simplifying assumption. [...] As in nonhierarchal models, it is often practical to start with a simple, relatively non-informative prior distribution on $\phi$ and seek to add more prior information if there remains too much variation in the posterior distribution.
