SE community, I hope to get some insights into the following problem. Given a simple linear regression model $$Y=X\beta+\epsilon\text{ , where } Y\in\mathbb{R}^T,X\in\mathbb{R}^{T \times N}.$$ Under a Gaussian likelihood function with homoscedastic error terms the conditional distribution of the dependent variable takes the form $$Y|\beta,h \sim N(X\beta,h^{-1}I).$$ I assign a conditional (uninformative) conjugate prior for $\beta$ and $h$ $$\beta|h \sim N(0,cI), h\sim G(s^{-2},v)$$ were $c\rightarrow\infty, v\rightarrow0$. It is a standard result that the marginal posterior distribution of $\beta$ is multivariate t with $$\beta|D\sim t_N (\hat{\beta},\hat{\Sigma},T).$$ What happens if $(X'X)$ is singular? In standard regression I would go for the generalized Moore-Penrose Pseudoinverse $(X'X)^+$ instead of using $(X'X)^{-1}$. However, in this case the posterior variance $\hat{\Sigma}:=c(X'X)^{-1}$ would be singular as well and I doubt that the $t$-Distribution is still well-defined. Is this correct?
And even further distracting to me: Assume I am not really interested in the posterior distribution of $\beta$ but just a linear combination $z:=A\beta$ where $A\in\mathbb{R}^{N-1 \times N}$, and $|A\hat{\Sigma}A'|\neq 0$. I would be able to sample from that distribution although its construction is based on something that is not really defined (the distribution of $\beta$). Is there a way to handle this? Or is there an essential mistake in my question that makes my whole point obsolete?