My POV would be reviewing a paper in psychology or forecasting on its statistical merits. I'll mostly second Nico's very good remarks.
How much effort should we put in to
understand the application area?
Quite a lot, actually. I wouldn't trust myself to comment on more than the most basic statistical problems without having understood the area. Fortunately, this is often not very hard in many branches of psychology.
How much time should I spend on a
I'll go out on a limb and state a specific time: I'll spend anything between two and eight hours on a review, sometimes more. If I find that I'm spending more than a day on a paper, it probably means that I'm really not qualified to understand it, so I'll recommend the journal find someone else (and try to suggest some people).
How picky are you when looking at
Very picky indeed. The figures are going to be what people remember of a paper and what ends up in lecture presentations without much context, so these really need to be done well.
How do you cope with the data not
In psychology, the data are usually not shared - measuring 50 people by MRI is very expensive, and the authors will want to use these data for further papers, so I kind of understand their reluctance to just give out the data. So anyone who does share their data gets a big bonus in my book, but not sharing is understandable.
In forecasting, many datasets are publicly available. In this case I usually recommend that the authors share their code (and do so myself).
Do you try and rerun the analysis
Without the data, there is only so much one can learn from this. I'll play around with simulated data if something is very surprising about the paper's results; otherwise one can often tell appropriate from inappropriate methods without the data (once one understands the area, see above).
What's the maximum number of papers
your would review in a year?
There is really little to add to whuber's point above - assuming that every paper with on average n coauthors I (co-)submit gets 3 reviews, one should really aim at reviewing at least 3/(n+1) papers for each own submission (counting submissions rather than own papers which may be rejected and resubmitted). And of course, the number of submissions as well as the number of coauthors varies strongly with the discipline.