When a probability density function is defined to be finite? In "Pattern recognition and machine learning" by Cristopher Bishop, Chapter 2.3.6 (pag. 100) says that

The gamma distribution has a finite integral if $a>0$, and the
  distribution itself is finite if $a \ge 1$

When a probability density function is defined to be finite? Can you make me an example of a finite pdf, and one of an infinite pdf?
 A: I suppose you mean the function $f(x) = \frac{\beta^\alpha}{\Gamma(\alpha)}x^{\alpha - 1}e^{-\beta x},$ for $x > 0,$ and $0$ otherwise, as stated in Wikipedia. There it is stated that parameters $\alpha >0$ and $\beta > 0,$ assuring that $f(x)$ is a density function. 
If that is true,
then the integral $J = \int_0^\infty f(x)\,dx$ converges and the constant
factor $\frac{\beta^\alpha}{\Gamma(\alpha)}$ is chosen so that $J = 1.$
That together with the fact that $f(x) \ge 0,$  makes
$f(x)$ the density function of a probability distribution.
Furthermore, if $\alpha \ge 1,$ then $f(x) \le M$ for some finite $M$ and for all $x\in (0,\infty).$
By contrast, for $0 < \alpha < 1,$ the function $f(x)$ has a vertical asymptote at $0.$
By way of illustration we show density functions of the distributions
$\mathsf{Beta}(\alpha = 1/2, \beta = 1)$ on the left, $\mathsf{Beta}(\alpha = 1, \beta = 1)$ in the center, and $\mathsf{Beta}(\alpha = 2, \beta = 1)$ at the right.

Note: The parameter $\alpha$ is called a shape parameter because
it determines the fundamental shape of the density curve. As you can see from the right-hand plot above, the density function has one inflection point for $\alpha = 2.$ Can you find a value of $\alpha$ above which the density function has two inflection points (and thus a 'left tail')?
Addendum: Plots for three additional values of $\alpha:$

