Maybe I'm missing something, but isn't the prior, $p(\theta) \propto \theta^{-1}$ merely a $\mbox{beta}(a=0,b=1)$ distribution? (If $a=0$ is uncomfortable, then set $a=\epsilon$ for small $\epsilon$.) Therefore the usual beta-Bernoulli update rule should apply. That is, with a Bernoulli likelihood function (which this is, for $N$ independent trials), and a conjugate beta prior (for which the present problem is a special case), the posterior is a again a beta distribution: $\mbox{beta}(a+k,b+N-k)$. Here we start with $a=\epsilon$ and $b=1$, so the posterior is $\mbox{beta}(\epsilon+k,1+N-k)$.

Maybe I'm missing something, but isn't the prior, $p(\theta) \propto \theta^{-1}$ merely a $\mbox{beta}(a=0,b=1)$ distribution? (If $a=0$ is uncomfortable, then set $a=\epsilon$ for small $\epsilon$.) Therefore the usual beta-Bernoulli update rule should apply. That is, with a Bernoulli likelihood function (which this is, for $N$ independent trials), and a conjugate beta prior (for which the present problem is a special case), the posterior is again a beta distribution: $\mbox{beta}(a+k,b+N-k)$. Here we start with $a=\epsilon$ and $b=1$, so the posterior is $\mbox{beta}(\epsilon+k,1+N-k)$.

P.S. For a derivation of the general beta-Bernoulli update rule, see p. 132 of DBDA2E.