Equal weight between prior probabilities

Question

While constructing a model hierarchy for Bayesian analysis, I have two parameters:

• $\theta_0 \sim \textrm{Uniform}(80, 90)$
• $\theta_1 \sim \textrm{Normal}(0.093, 0.002)$

I take the $\ln$ of the pdf for the parameter's value, sum them: $\sum \ln(\rm pdf)$, and obtain the prior probability. This is then also added log-likelihood of the output of the model and the likelihood of this model is obtained (as one iteration in my MCMC).

However, do these priors need to be rescaled somehow? It seems wrong that $\ln(\textrm{pdf}(\theta_1)) > \ln(\textrm{pdf}(\theta_0))$ will always be true.

As an example case, any value for $\theta_0$ will have a pdf of $0.1$ and $\ln(\rm pdf)$ of $-2.3$ while for $\theta_1$ a value close to the mean will have a pdf of $200$ and $\ln(\rm pdf)$ of $5.7$.

I do not mean to presuppose that the importance of $\theta_1 > \theta_0$ in the model.

Similarly, the likelihood values for the model seem under represented -- the $\ln(\rm pdf)$ for predicted values' errors falling under the t-distribution is much smaller than my priors.

Answer 1

However, do these priors need to be rescaled somehow? It seems wrong that $\ln(\textrm{pdf}(\theta_1)) > \ln(\textrm{pdf}(\theta_0))$ will always be true.

Why does it seem wrong? TLDR; It doesn't matter.

Abstracting from Bayes theorem, let's simplify it. Imagine that you have two independent random variables $X \sim \mathcal{N}(1, 2)$ and $X \sim \mathcal{G}(1,2)$.

You want to look at their joint distribution and since they are independent, you multiply their probability densities to obtain joint density. Obviously, they have "different scales", but so what? Marginal density of $X$ is still highest at it's highest point, marginal density of $Y$ is still highest at it's highest point. Jointly they have highest point where they are jointly highest. It doesn't matter that they had different "scales".

Moreover, by pure algebra, if you take your Bayes theorem, e.g.

$$ f(\theta_1,\theta_2\mid X) \propto f(X\mid\theta_1,\theta_2) \, f(\theta_1)\, f(\theta_2) $$

and, as you suggest, you introduce some "scaling constants" $c_1,c_2,c_3$ so that each density has the same maximum height (or whatever you want), then you can re-arrange the arguments since multiplication has commutative property and receive the same outcome

$$ c_1 f(X\mid\theta_1,\theta_2) \, c_2 f(\theta_1)\, c_3 f(\theta_2) = f(X\mid\theta_1,\theta_2) \, f(\theta_1)\, f(\theta_2) \, c_1 \, c_2 \, c_3 = \\ f(X\mid\theta_1,\theta_2) \,c_3\, f(\theta_2) \, c_1 \, c_2 \, f(\theta_1) = \dots $$

Answer 2

The density for a normal distribution converges to 0 in the limit to both positive and negative infinity. So it is not true that $\ln(\textrm{pdf}(\theta_1)) > \ln(\textrm{pdf}(\theta_0))$ for all $\theta_1$ and $\theta_0$. The reason that the density is larger for this normal distribution near its mean compared to the uniform distribution is because the variance of the normal distribution is small.