In a Markov chain, a state $j$ is transient if $f_{jj}<1$ ($f_{jj}$ is probability of ever visiting state $j$ starting from state $j$ ). Suppose, I have an irreducible transient DTMC (means all states are transient). Now, I want to prove that for any $i,j$ in $S$ ($S$ is DTMC state space), $f_{ij}$ (i.e probability of ever reaching state $j$ starting from state $i$) is less than 1. It is clear that $f_{ii}<1$ and $f_{jj}<1$. But, how to prove that $f_{ij}<1$ for any $i,j$.
Thanks Prasenjit
Your claim is false: there exist transient Markov chains such that $f_{ij}=1$ for some (but not all) states $i$ and $j$.
For example, assume that the state space is the union of the discrete halfline $\mathbb{Z}_+$ and of a discrete circle $\mathbb{Z}/N\mathbb{Z}$ with $N\ge3$, the halfline and the circle meeting at $0$. Write $c(k)$ for the $k$th state on the circle, counted clockwise and starting from $0$, thus $c(0)=c(N)=0$ but $c(k)$ for $1\le k\le N-1$ is not on the halfline $\mathbb{Z}_+$.
The transitions are as follows. If one is at $i$ in $\mathbb{Z}_+$ with $i\ne0$, one moves to $i+1$ or to $i-1$ with probability $p$ or $1-p$, respectively. If one is at $0$, one moves to $1$ or to $c(1)$, both with positive probability. If one is at $c(k)$ with $1\le k\le N-1$, one moves to $c(k+1)$ with probability $1$.
In words, while on the halfline, one performs a biased random walk and while on the circle, one moves on the circle clockwise and deterministically until one is back at $0$.
For every $p>1/2$, this Markov chain is transient. Nevertheles, for every $k$ and $\ell$ such that $1\le k<\ell\le N-1$, starting from $c(k)$, one hits $c(\ell)$ with full probability hence $f_{c(k)c(\ell)}=1$.
