# Handling overflow warnings in pymc

## Abstract

I am getting numerical overflow warnings in pymc that are stalling the sampler. I'll first specify what the context is then ask more directed questions about the solution. The context portion can probably be skipped; I included it in case some feature of it is possibly relevant to the solution.

## Context

I've got a mixed effects bivariate logistic AR(1) model that I am fitting to time series binary data in pymc 2.3. The model specification is as follows, where $\mathbf{l}$ and $\mathbf{t}$ are observed and $\mathcal{N}$ is parameterized by precision instead of variance. The random intercepts are $\mathbf{U}$ (subjects) and $\mathbf{v}$ (items) and the random slopes are $\mathbf{d}$. The fixed effects are encoded in $\mathbf{c}$ (conditions) and $f$ maps a subject index to its corresponding fixed effect.

$$\tau_{11}, \tau_{12}, \tau_2, \lambda \sim \mathrm{Exp}(1)$$ $$u_{s1} | \tau_{11} \sim \mathcal{N}(0, \tau_{11})$$ $$u_{s2} | \tau_{12} \sim \mathcal{N}(0, \tau_{12})$$ $$d_s | \lambda \sim \mathrm{Exp}(\lambda)$$ $$v_i | \tau_2 \sim \mathcal{N}(0, \tau_2)$$ $$c_j \sim \mathcal{N}(0, 10^{-6})$$ $$l_{si1} | u_s, v_i \sim \mathrm{Bern}(\mathrm{logit}^{-1}(u_{s1} + v_i + c_{f(s)}))$$ $$l_{sij} | u_s, v_i, d_s \sim \mathrm{Bern}(\mathrm{logit}^{-1}(u_{s1} + v_i + c_{f(s)} + d_sl_{si(j-1)}))$$ $$t_{si1} | u_s, v_i \sim \mathrm{Bern}(\mathrm{logit}^{-1}(u_{s2} + v_i + c_{f(s)}))$$ $$t_{sij} | u_s, v_i, d_s \sim \mathrm{Bern}(\mathrm{logit}^{-1}(u_{s2} + v_i + c_{f(s)} + d_st_{si(j-1)}))$$

## Problem

I can run the pymc 2.3 sampler for this model at about 10 samples a second over approximately 120000 (bivariate binary) data points. The sampler will go for a few hundred thousand samples before hitting aRuntimeWarning: overflow encountered in double_scalars. The sampler then stalls indefinitely.

I have encountered this specific warning when hand-rolling samplers in numpy/scipy. In general, it happens when the MCMC proposal is particularly bad thus making the likelihood extremely small and the negative log-likelihood extremely large. I assume this is happening here since the traceback is telling me StepMethods.py is throwing this warning.

My tack in these cases is to catch the warning and generate a new proposal, since I can be nearly certain that these proposals would never be accepted. I'm surprised that pymc doesn't do something like this. (Or maybe it does?) My question: is there some parameter I should be flipping to force pymc to repropose in these cases or do I need to muck with the proposal generation code? (Or is this even a problem arising from particularly bad proposals?)

One idea would be to use the AdaptiveMetropolis sampler and specify the initial (co)variance of the variables to a low estimate. That could keep the sampler from proposing especially unlikely steps... https://pymc-devs.github.io/pymc/modelfitting.html#the-adaptivemetropolis-class