1
$\begingroup$

I'm studying pattern recognition and I'm at the part about Kernel density estimators. During the introduction of the subject, the book I'm studying (Pattern Recognition & Machine Learning by Bishop) takes for granted something I'm not sure I can understand.

Say we have an unknown pdf $p(x)$ in some D-dimensional space and let us consider some small region $R$ containing $x$. Then, if we make the assumption that $R$ is small enough so that the pdf is roughly constant over the region, we have $$P \approx p(x)V$$ where $V$ is the volume of $R$.

I'm completely unaware of how this formula was derived or how the volume $V$ appeared there. Any help woud be greatly appreciated. Thank you.

Improve this question
$\endgroup$
2
  • $\begingroup$ See stats.stackexchange.com/questions/4220. Your equation is one definition of a probability density. $\endgroup$
    – whuber ♦
    Commented Nov 23, 2019 at 22:31
  • 1
    $\begingroup$ Thanks whuber. I was aware of this definition but as I said in the comments I was confused from the volume term. I think I got it now though. $\endgroup$
    – thenac
    Commented Nov 23, 2019 at 23:10

2 Answers 2

Reset to default
2
$\begingroup$

It's simple integral approximation. First, think in 1D. The area under a curve $f(x)$ in a very small x-axis segment (e.g. $[x,x+\Delta x]$) is $\approx f(x)\Delta x$; because $f(x)$ is nearly constant across this region. Similarly, in 2D, integral of $f(x,y)$ over a small region is $\approx f(x)\Delta x\Delta y$. In multiple dimensions, all the multiplicands near $f$ is called as volume, i.e. $f(x_1,...,x_n)\underbrace{\Delta x_1...\Delta x_n}_V$.

Improve this answer
$\endgroup$
7
  • $\begingroup$ This is what I thought at first, but I was confused because as you say, when my variable $x$ is a vector of 2 dimensions $x,y$ then indeed the approximation would be $f(x)Δ(x)Δ(y)$. But, isnt this the volume? Since $f(x)$ is the "height" and you multiply it with the other 2 dimensions, say width and depth. Therefore I was confused because I thought, it should be $P = V$ and not $p(x)V$ since the volume requires to multiply $p(x)$ with all the rest dimensions. But its probably an n-dimensional thing that they called volume this product? $\endgroup$
    – thenac
    Commented Nov 23, 2019 at 19:26
  • $\begingroup$ @Thomas, You misinterpret the term volume. In 1D, length of line segment $R$ is the volume. In 2D, area of $R$ is the volume etc. The volume here is not the area/volume under $f(x)$. Informally, think as if it is referred as hypervolume of region $R$. $\endgroup$
    – gunes
    Commented Nov 23, 2019 at 19:33
  • $\begingroup$ When you say it like that it makes sense. We get off subject here but, isn't volume the space that a 3d object takes up? And it is $A*h$ where A = area and h = height. Is the concept we use in this question somewhat different from the one I mention here in the comment? Of course we generalize it for more than 3 dimensions. $\endgroup$
    – thenac
    Commented Nov 23, 2019 at 19:42
  • 1
    $\begingroup$ The term volume is used for all dimensions, e.g. en.wikipedia.org/wiki/Volume_of_an_n-ball $\endgroup$
    – gunes
    Commented Nov 23, 2019 at 19:43
  • 1
    $\begingroup$ Ok thing I got this! Gotta let it sink in :) Thanks for all the help, very much appreciated. Have a good day. $\endgroup$
    – thenac
    Commented Nov 23, 2019 at 19:54
1
$\begingroup$

Basically, this is how probability density is defined. Probability density is the probability per foot. It is normalized by the area, so that over all the area it integrates to unity. More formally, quoting All of statistics by Larry A. Wasserman:

2.11 Definition. A random variable $X$ is continuous if there exists a function $f_X$ such that $f_X(x) \ge 0$ for all $x$, $\int_{-\infty}^\infty f_X(x)\, dx = 1$ and for every $a \le b$,

$$ \mathbb{P}(a < X < b) = \int_a^b f_X(x) \,dx \tag{2.2} $$

The function $f_X$ is called the probability density function (PDF).

Improve this answer
$\endgroup$
1
  • $\begingroup$ I'm aware of the definition and it was my first thought. However I was confused for the reason I posted in the above comment, if you'd like to take a look ! $\endgroup$
    – thenac
    Commented Nov 23, 2019 at 19:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.