Skip to main content
added 176 characters in body
Source Link
Néstor
  • 3.8k
  • 30
  • 38

This is a very interesting question, indeed!

The fact that a given distribution violates some physical limits, is why I love Bayes' Theorem so much. It is true that the normal distribution performs very well in a lot of situations, but (if I understood your question correctly) what annoys you is that it actually violates some physical constrainsconstraints (e.g., that a person can't be taller than 30 feet). The question is, then, how do we correct for this? Is it possible to take into account this BEFORE we do the measurement? The answer is yes: we can account for this.

What you are actually trying to accomplish is to put some a-priori certain limits to a distribution, and this can be included via te a-priori distribution. For example, if we want to incorporate this limit for the height of a person, we could do it via the a-posteriori distribution of heights $h$, i.e., using Bayes' theorem,

$$p(h|D,I)=\frac{p(D|h,I)p(h|I)}{p(D|I)}.$$

Here $D$ is the data, $h$ is the parameter (the height in this case) and $I$ is any a-priori information that you have concerning the distribution of the parameter $h$. As you may recall, $p(D|h,I)$ is the likelihood, where you actually define the sampling distribution of your data (maybe you are measuring persons' height assuming gaussian errors). On the other hand, $p(h|I)$ is the a-priori distribution of the parameter $h$ and is exactly what you are looking for: here you impose some constraints on the value of $h$. For example, if you know that a person by physical constraints cannot be smaller than $h_{low}$ and cannot be larger than $h_{high}$, then this enters via the a-priori distribution: $$p(h|I)=f(h),\ \ \ h_{low}<h<h_{high}$$ and is $0$ otherwise. However, the real problem is how to define a "good" a-priori distribution. You might think of the uniform distribution, i.e., assume $$p(h|I)=\frac{1}{h_{high}-h_{low}},\ \ \ h_{low}<h<h_{high}$$ but this can affect in a serious way your distribution because not all ranges of values in a logaritmic scale are equal (this means that the probability of obtaining, e.g., $1<h<10$ is lower than obtaining $10<h<100$ and, therefore, the distribution is scale dependent!). When you measure a scale parameter (like height or width), a more reliable a-priori distribution is the Jeffrey's prior, i.e., $$p(h|I)=\frac{1}{h\ln(h_{high}/h_{min})},\ \ \ h_{low}<h<h_{high}$$

This is a very interesting question, indeed!

The fact that a given distribution violates some physical limits, is why I love Bayes' Theorem so much. It is true that the normal distribution performs very well in a lot of situations, but (if I understood your question correctly) what annoys you is that it violates some physical constrains. The question is, then, how do we correct for this?

What you are actually trying to accomplish is to put some a-priori certain limits to a distribution, and this can be included via te a-priori distribution. For example, if we want to incorporate this limit for the height of a person, we could do it via the a-posteriori distribution of heights $h$, i.e., using Bayes' theorem,

$$p(h|D,I)=\frac{p(D|h,I)p(h|I)}{p(D|I)}.$$

Here $D$ is the data, $h$ is the parameter (the height in this case) and $I$ is any a-priori information that you have concerning the distribution of the parameter $h$. As you may recall, $p(D|h,I)$ is the likelihood, where you actually define the sampling distribution of your data (maybe you are measuring persons' height assuming gaussian errors). On the other hand, $p(h|I)$ is the a-priori distribution of the parameter $h$ and is exactly what you are looking for: here you impose some constraints on the value of $h$. For example, if you know that a person by physical constraints cannot be smaller than $h_{low}$ and cannot be larger than $h_{high}$, then this enters via the a-priori distribution: $$p(h|I)=f(h),\ \ \ h_{low}<h<h_{high}$$ and is $0$ otherwise. However, the real problem is how to define a "good" a-priori distribution. You might think of the uniform distribution, i.e., assume $$p(h|I)=\frac{1}{h_{high}-h_{low}},\ \ \ h_{low}<h<h_{high}$$ but this can affect in a serious way your distribution because not all ranges of values in a logaritmic scale are equal (this means that the probability of obtaining, e.g., $1<h<10$ is lower than obtaining $10<h<100$ and, therefore, the distribution is scale dependent!). When you measure a scale parameter (like height or width), a more reliable a-priori distribution is the Jeffrey's prior, i.e., $$p(h|I)=\frac{1}{h\ln(h_{high}/h_{min})},\ \ \ h_{low}<h<h_{high}$$

This is a very interesting question, indeed!

The fact that a given distribution violates some physical limits, is why I love Bayes' Theorem so much. It is true that the normal distribution performs very well in a lot of situations, but (if I understood your question correctly) what annoys you is that it actually violates some physical constraints (e.g., that a person can't be taller than 30 feet). The question is, then, how do we correct for this? Is it possible to take into account this BEFORE we do the measurement? The answer is yes: we can account for this.

What you are actually trying to accomplish is to put some a-priori certain limits to a distribution, and this can be included via te a-priori distribution. For example, if we want to incorporate this limit for the height of a person, we could do it via the a-posteriori distribution of heights $h$, i.e., using Bayes' theorem,

$$p(h|D,I)=\frac{p(D|h,I)p(h|I)}{p(D|I)}.$$

Here $D$ is the data, $h$ is the parameter (the height in this case) and $I$ is any a-priori information that you have concerning the distribution of the parameter $h$. As you may recall, $p(D|h,I)$ is the likelihood, where you actually define the sampling distribution of your data (maybe you are measuring persons' height assuming gaussian errors). On the other hand, $p(h|I)$ is the a-priori distribution of the parameter $h$ and is exactly what you are looking for: here you impose some constraints on the value of $h$. For example, if you know that a person by physical constraints cannot be smaller than $h_{low}$ and cannot be larger than $h_{high}$, then this enters via the a-priori distribution: $$p(h|I)=f(h),\ \ \ h_{low}<h<h_{high}$$ and is $0$ otherwise. However, the real problem is how to define a "good" a-priori distribution. You might think of the uniform distribution, i.e., assume $$p(h|I)=\frac{1}{h_{high}-h_{low}},\ \ \ h_{low}<h<h_{high}$$ but this can affect in a serious way your distribution because not all ranges of values in a logaritmic scale are equal (this means that the probability of obtaining, e.g., $1<h<10$ is lower than obtaining $10<h<100$ and, therefore, the distribution is scale dependent!). When you measure a scale parameter (like height or width), a more reliable a-priori distribution is the Jeffrey's prior, i.e., $$p(h|I)=\frac{1}{h\ln(h_{high}/h_{min})},\ \ \ h_{low}<h<h_{high}$$

Source Link
Néstor
  • 3.8k
  • 30
  • 38

This is a very interesting question, indeed!

The fact that a given distribution violates some physical limits, is why I love Bayes' Theorem so much. It is true that the normal distribution performs very well in a lot of situations, but (if I understood your question correctly) what annoys you is that it violates some physical constrains. The question is, then, how do we correct for this?

What you are actually trying to accomplish is to put some a-priori certain limits to a distribution, and this can be included via te a-priori distribution. For example, if we want to incorporate this limit for the height of a person, we could do it via the a-posteriori distribution of heights $h$, i.e., using Bayes' theorem,

$$p(h|D,I)=\frac{p(D|h,I)p(h|I)}{p(D|I)}.$$

Here $D$ is the data, $h$ is the parameter (the height in this case) and $I$ is any a-priori information that you have concerning the distribution of the parameter $h$. As you may recall, $p(D|h,I)$ is the likelihood, where you actually define the sampling distribution of your data (maybe you are measuring persons' height assuming gaussian errors). On the other hand, $p(h|I)$ is the a-priori distribution of the parameter $h$ and is exactly what you are looking for: here you impose some constraints on the value of $h$. For example, if you know that a person by physical constraints cannot be smaller than $h_{low}$ and cannot be larger than $h_{high}$, then this enters via the a-priori distribution: $$p(h|I)=f(h),\ \ \ h_{low}<h<h_{high}$$ and is $0$ otherwise. However, the real problem is how to define a "good" a-priori distribution. You might think of the uniform distribution, i.e., assume $$p(h|I)=\frac{1}{h_{high}-h_{low}},\ \ \ h_{low}<h<h_{high}$$ but this can affect in a serious way your distribution because not all ranges of values in a logaritmic scale are equal (this means that the probability of obtaining, e.g., $1<h<10$ is lower than obtaining $10<h<100$ and, therefore, the distribution is scale dependent!). When you measure a scale parameter (like height or width), a more reliable a-priori distribution is the Jeffrey's prior, i.e., $$p(h|I)=\frac{1}{h\ln(h_{high}/h_{min})},\ \ \ h_{low}<h<h_{high}$$