Vowpal wabbit and SGD divergence

I have the following vowpal wabbit log. To me it looks quite counter-intuitive: the objective function (l1-regularized hinge loss) seems to go down then suddenly spiking up. I am aware that gradient descent can diverge if the learning rate is too high. What bugs me is this behavior: it tends to converge, then finally diverges. Is this possible for convex functions (it is convex, right?) If the learning rate is not too high initially, and it's decreasing as well as the gradient must, how can it suddenly become too high? What am I missing? Can the vw learning rate decay have something to do with it?

And btw, what's probably wrong with my settings/trainset according to the log? Thanks for your attention.

using l1 regularization = 7.498e-05
final_regressor = /home/mm/models/52.model
Num weight bits = 28
learning rate = 2
initial_t = 0
power_t = 0.5
decay_learning_rate = 1
creating cache_file = /tmp/trtmp.cache
num sources = 1
average    since         example     example  current  current  current
loss       last          counter      weight    label  predict features
0.504495   0.504495           23         2.0   1.0000  -1.0000       16
0.250552   0.000000           68         4.0  -1.0000  -1.0000       15
0.241113   0.232359          143         8.3   1.0000  -1.0000       14
0.311833   0.380056          250        16.9   1.0000  -1.0000       14
0.340020   0.368193          436        33.8  -1.0000  -1.0000        3
0.346579   0.353136          915        67.6  -1.0000  -1.0000       25
0.301540   0.256500         1830       135.2  -1.0000  -1.0000       11
0.253466   0.205517         3625       270.7   1.0000  -1.0000       17
0.225351   0.197240         7251       541.5  -1.0000  -1.0000        2
0.193044   0.160737        14823      1083.1  -1.0000  -1.0000        4
0.171241   0.149439        30096      2166.2  -1.0000  -1.0000       13
0.156050   0.140859        60235      4332.4  -1.0000  -1.0000        3
0.129873   0.103696       120171      8664.8  -1.0000  -1.0000       14
0.110798   0.091725       240159     17330.0   1.0000   1.0000       16
0.104312   0.097826       479570     34660.0  -1.0000  -1.0000        2
0.116619   0.128926       959076     69320.0  -1.0000  -1.0000       18
0.142526   0.168433      1918859    138640.0  -1.0000  -1.0000       14

finished run
number of examples per pass = 62866
passes used = 50
weighted example sum = 227125
weighted label sum = -45425.1
average loss = 0.164373
best constant = -0.2
total feature number = 39394300

• 'learning rate = 2' <- Looks huge indeed – bijey Nov 21 '14 at 9:12