Why do we use Gamma($\epsilon, \epsilon$) as non-informative prior for precision and Normal prior for betas in Linear Regression Suppose my regression model is 
$$Y_i = \beta_0 + \beta_1X_{i1} + \epsilon_i $$
In most books I am seeing that the prior used for precision $\tau = 1/\sigma^2 $ is $Gamma(\epsilon, \epsilon)$. However I am unable to understand how is that found given that the beta's are unknown too? Similarly I am not getting how is Normal prior used for the beta's. 
 A: A simple linear regression model is 
$$ Y_i = \beta_0 + \beta_1 X_{i1} + \epsilon_i, \epsilon_i \stackrel{ind}{\sim} N(0,\sigma^2). $$
A frequently used prior is 
$$ p(\beta_0,\beta_1,\sigma^2) \propto 1/\sigma^2 $$
Notice that this is a joint prior for $\beta_0$, $\beta_1$ and $\sigma^2$. Marginally the priors for $\beta_0$ and $\beta_1$ are uniform over the real line and the marginal prior for $\sigma^2$ is the limit as $\epsilon \to 0$ of an inverse gamma distribution with shape and scale both $\epsilon$ (or equivalently a gamma distribution on the precision $\tau=1/\sigma^2$ with the shape and rate both $\epsilon$). 
You question appears to be why do we use this prior and I can think of two reasons: 1) it corresponds to the prior derived by some methods of deriving default priors (I believe a particular reference prior approach and the maximal data information approach) and 2) the resulting posterior exactly matches a non-Bayesian solution, i.e. inference through the sampling distributions of the MLE. 
