What does the pmf of a discrete random variable look like if it can take on the value $\infty$? What does the pmf of a discrete random variable look like if it can take on the value $\infty$? 
Consider a stopping time $\tau$ of a Markov chain, which is a random variable that takes values in the set $\{1,2,3,\ldots\}$. Often it is easy to find the individual probabilities $P(\tau = k)$ for specific $k \in \mathbb{Z}^+$. 
I have always written this without really pausing to think, but is it always correct to say 
$$
P(\tau < \infty) = \sum_{k=1}^{\infty}P(\tau = k)?
$$
If so, how do we come up with $P(\tau = \infty)$? Is it always equal to $1 - \sum_{k=1}^{\infty}P(\tau = k)$? Are there any measure-theoretic concerns here?
 A: It helps to go back to basic definitions. A probability space is the triple $(\Omega, \mathcal{F}, P)$ where $\Omega$ is the sample space, $\mathcal{F}$ is the set of subsets of $\Omega$, called "events", on which $P$ is defined (normally the Borel $\sigma$-algebra), and $P$ is a probability measure.
A random variable is a function $f: \Omega \mapsto E$, where $E$ is some measurable space. $E$ is almost always the real numbers $\mathbb{R}$, but it doesn't have to be. It could be the complex numbers $\overline{\mathbb{C}}$, or the Riemann sphere \, or the extended real numbers $\overline{\mathbb{R}}$, or the projectively extended real line $\widehat{\mathbb{R}}$, or any other measurable space. I mention these in particular because they all have at least one point called $\infty$ adjoined to the set in order to compactify it. 
So, the first thing you have to do is specify that the range $E$ of your random variable is one of these sets. (If you don't specify, everyone will assume $E = \mathbb{R}$, and will therefore naturally assume that $\infty \notin E$.)
I can't actually guess what you were going for from your question:  you only mention one infinity (instead of $-\infty$ and $+\infty$) which seems to imply the projectively extended real line, but then you use the notation $< \infty$ which isn't defined for $\widehat{\mathbb{R}}$. ($\widehat{\mathbb{R}}$ is a circle, so comparison operations don't make sense.) You definitely want to clear that up, which you can do by simply giving the domain and range of your random variable, e.g. $X : \Omega \mapsto \overline{\mathbb{R}}$.
Now, because you're using a discrete random variable, the situation is actually very simple. You must define the measure $P$ on a set of discrete points, and the sum over the entire support of the random variable must sum to one. I would use set summation notation instead of indexing over the natural numbers, e.g. something like:
$$ \sum_{k \in \text{supp}(X)} P(τ=k) = 1$$
The support $\text{supp}(X)$ may be $\mathbb{N} \cup \{\infty\}$, for example, which would give us three equivalent ways to write the sum:
$$ 
     \sum_{k \in \mathbb{N} \cup \{\infty\}} P(τ=k) =
     \sum_{k \in \mathbb{N}} P(τ=k) + P(τ=\infty) =
     \sum_{k=1}^{\infty} P(τ=k) + P(τ=\infty) = 1
$$
I find the third form to be very confusing in this case and would avoid it. Whether or not the form you use in your original question is valid depends on which extended real line you are using; if it is $\overline{\mathbb{R}}$ then I think it is technically correct but I would use the set summation notation with $k \in \mathbb{N}$ to avoid confusion. 
Finally, I would also reiterate that statistics over the extended real line isn't very practical and would advise an alternative approach in most cases. For example, two dependent random variables $X$ and $Y$, where $X : \Omega \mapsto \mathbb{R}$ and $Y : \Omega \mapsto \{0, 1\}$ where $Y$ indicates if $X$ is "finite" or not. This is the way censured, truncated, or missing data is often represented, for instance.
A: You can work with random variables that take on values in the extended real numbers $\bar{\mathbb{R}} = \mathbb{R} \cup \{ -\infty, \infty \} $.  This set is just like the real numbers, except that there are two special elements for "negative infinity" and "positve infinity" that are defined using standard limiting behaviours for real numbers.  It is not problematic to work with random variables that can taken on these values, and in some contexts (e.g., survivial analysis and the analysis of Markov chains) it is extremely useful.  (For an example in survival analysis, see here).
In your case you have a random variable that is a positive integer, so you can work on the extended natural numbers $\bar{\mathbb{N}} = \mathbb{N} \cup \{ \infty \}$.  Your random variable will obey the standard laws of probability over that set, including the norming axiom and the countable additivity axiom.  Using these axioms you have:
$$1 = \mathbb{P}(\tau \in \bar{\mathbb{N}}) = \sum_{k \in \bar{\mathbb{N}}} \mathbb{P}(\tau = i) = \sum_{k=1}^\infty \mathbb{P}(\tau = k) + \mathbb{P}(\tau = \infty).$$
Re-arranging gives the common-sense rule:
$$\mathbb{P}(\tau = \infty) = 1- \sum_{k=1}^\infty \mathbb{P}(\tau = k).$$
There are no measure-theoretic problems here.  The set $\bar{\mathbb{N}}$ is countable (i.e., it is cardinally equivalent to the integers) so the underlying measure theory is the same as it would be for an integer random variable that can't take on infinite values.
