Are there examples of R packages changing dramatically between version such that the results of a statistical function were significantly different? Im trying to understand how people use R packages and was wondering if there are documented cases where R packages have produced different answers.
Clarification: The motivation behind this question comes from an effort I'm involved in where the goal to understand the importance of provenance in the analytical methods and how it facilitates reproducible research. While R is big in the science community at present, and R packages are versioned in CRAN, without detailed information [especially version numbers], someone trying to reproduce a body of work in the future might come to a different conclusion that the original work (even with the original data). 
Example: Paper by John Doe says "we used R 2.3.1 and package glmulti to fit our models". 10 years from now, someone might use a new version of glmulti (no one knows what version was used in the original) which might produce a much different conclusion. My question: Are there examples of such a thing happening already? Version 2 or R package produces a much different result that version 1. 
 A: This will vary package to package, but the general answer is yes. Outputs can vary, and even basic usage too (input/ouput args). This is why, when I do an important analysis, I always like to document what versions were used with version() and sessionInfo(). Even if things change, old versions are retained on CRAN, so you can get the old versions if you need them.
A: I've had issues with package glmnet over versions. If I remember well, it was going from version 1.5 to 1.6, but I may be off a little.
The package creators/maintainers changed the order of the classes of their objects (so it became c("lognet", "glmnet") instead of c("glmnet", "lognet") - or it may have been the other way around). Of course, they also changed all their S3 functions to properly handle this (e.g. predict.lognet).
The problem with this was: an object you had created with an old version of glmnet was not compatible with the new functions (since dispatching worked the other way around). Most people wouldn't be in that spot (who saves a glmnet object for later use?), but I was.
Mind you: this is a very powerful package, developed by extremely intelligent people, so it could happen to anyone :-)
A: Just a quick point:


*

*The R package ecosystem is very big, and it's really up to the individual author, whether they intend to maintain backwards compatibility.

*I personally haven't had any issues with base R packages changing in ways that led to issues of backwards compatibility. In general, this is one reason why I prefer to use base R packages.

A: In my experience, most of the changes create the usual computing/programming type of issues. Functions get deprecated, arguments are different, etc. For example, it already happened to me that code would stop working because some function required an extra argument. This can be annoying but the problem is obvious and usually not too difficult to resolve.
Some packages could be better maintained in that respect but the usual “rules” of free software apply: You have to see that it is produced by volunteers – often statisticians who have other duties and not full-time professional software developers – and if quality and dependability are important to you, you should avoid anything with a version number like 0.x and only use mature packages with an active community of users and developers.
I have never encountered an update that would result in insidious changes of statistical results (e.g. switching the default method in some function, changing the degree of freedom and p value while producing superficially similar results). I guess it must happen though, at least through bug corrections (but I have read somewhere that Microsoft actually added code in its new products to emulate some older bugs to avoid breaking compatibility with programs that depend on these bugs). Maybe, some maintainers can give us some insights into the way these things are handled for their packages.
