Concerning the infinite variance of the least-squares estimator for the regression coefficients in case $p>n$, the book is right, in the following sense. If $p\leqslant n$ and the true model of the data is the one you are considering, i.e.
$$
Y_i = \beta_0 + \beta_1 x_{1i} + \ldots + \beta_{p-1} x_{p-1,i} + \varepsilon_i
$$
for $i=1,\ldots,n$ and all $\varepsilon_i$ are independent and normally distributed with mean zero and variance $\sigma^2$, then it can be shown that the column vector of least-squares estimates for $\beta_j$ ($j=0,\ldots,p-1$) has a multivariate normal distribution. That is,
$$
\hat{\boldsymbol{\beta}} \sim \text{N}(\boldsymbol{\beta}, \sigma^2(\boldsymbol{x}'\boldsymbol{x})^{-1})
$$
with $\boldsymbol{\beta}=\left[\matrix{\beta_0\\ \beta_1\\ \vdots \\ \beta_{p-1}}\right]$, $\hat{\boldsymbol{\beta}}=\left[\matrix{\hat{\beta_0}\\ \hat{\beta_1}\\ \vdots \\ \hat{\beta_{p-1}}}\right]$ and the design matrix $\boldsymbol{x} = \left[\matrix{1 & x_{11} & \cdots & x_{p-1,1}\\ \vdots & \vdots & & \vdots\\ 1 & x_{n1} & \cdots & x_{p-1,n}}\right]$.
So the coefficient estimates have the true coefficients as their mean, which is good news because that makes these estimates $\hat{\boldsymbol{\beta}}$ unbiased.
However, the main point here is that the Gram matrix $\boldsymbol{x}'\boldsymbol{x}$ is not always invertible, in which case its inverse does not exist, or, as the book says "is infinite". That is just like the number $1/0$ is "infinite" or does not exist. In that situation, the estimates $\hat{\boldsymbol{\beta}}$ are not unique, there are many values which minimise the sum of squares of the residuals.
If $p>n$ the columns of the design matrix $\boldsymbol{x}$ necessarily become linearly dependent and the Gram matrix $\boldsymbol{x}'\boldsymbol{x}$ will not have full rank. That means its inverse does not exist (or is infinite), and neither does the variance of $\hat{\boldsymbol{\beta}}$.
