# Is the R language reliable for the field of economics?

I am a graduate student in economics who recently converted to R from other very well-known statistical packages (I was using SPSS mainly). My little problem at the moment is that I am the only R user in my class. My classmates use Stata and Gauss and one of my professors even said that R is perfect for engineering, but not for economics. He said that many packages are built by people who know a lot about programming, but not much about economics and therefore are not reliable. He also mentioned the fact that since no money is actually involved in building an R package, there is therefore no incentive to do it correctly (unlike in Stata for example) and that he used R for a time and got some "ridiculous" results in his attempts to estimate some stuff. Moreover, he complained about he random number generator in R which he said was "messy".

I've been using R for just a little more than a month and I must say I have fallen in love with it. All this stuff I am hearing from my professor is just discouraging me.

So my question is: "Is R reliable for the field of economics?".

-
"many packages are built by people who know a lot about programming, but not much about economics". Package authors are practically always scientists or academics first and programmers a (very) distant (last) second. Actually, I think it would be a challenge to find a package authored by a "programmer". –  Brandon Bertelsen Apr 4 '12 at 3:09
Can I invite you and our other stats economists over to economics.stackexchange.com too - we'll welcome your questions and your answers on economics there –  EnergyNumbers Apr 4 '12 at 5:00
It seems there is also a bit of professional chauvinism in your professor's comments. How is being an economist a guarantee of reliability? I had rather trust software developers (it's actually a profession, not something anybody can do well without particular experience or training) and statisticians to produce reliable statistical software. –  Gaël Laurans Apr 4 '12 at 6:17
"Perfect for engineering but not for economics" because of reliability? He'd rather a skyscraper collapsed than an economy? The man's an idiot. Quit that school and find one not staffed by idiots. –  Spacedman Apr 4 '12 at 7:02
Commercial software can be good or bad. Open source software can be good or bad. What matters is whether or not the software you use is any good. Don't decide that based on prejudice and dogma. Use real evidence. –  David Heffernan Apr 4 '12 at 7:19

Let me share a contrasting view point. I'm an economist. I was trained in econometrics using SAS. I work in financial services and just tonight I updated R based models which we will use tomorrow to put millions of dollars at risk.

Your professor is just plain wrong. But the mistake he's making is VERY common and is worth discussing. What your professor seems to be doing is commingling the idea of the R software (the GNU implementation of the S language) vs. packages (or other code) implemented in R. I can write crap implementations of a linear regression using SAS IML. As a matter of fact, I've done that very thing. Does that mean SAS is crap? Of course not. SAS is crap because their pricing is non-transparent, ridiculously expensive, and their in house consultants over promise, under deliver, and charge a premium for the pleasure. But I digress...

The openness of R is a double edged sword: Openness allows any Tom, Dick, or Harry to write a crap implementation of any algorithm they think up while smoking pot in the basement of the economics building. The same openness allows practicing economists to share code openly and improve on each other's code. The licensing rules with R mean that I can write parallelization code for running R in parallel on Amazon's cloud and not have to worry about licensing fees for a 30 node cluster. This is a HUGE win for simulation based analysis which is a big part of what I do.

Your professor's comment that "many packages are built by people who know a lot about programming, but not much about economics" is, no doubt, correct. But there are 3716 packages on CRAN. You can be damn sure many of them were not written by economists. In the same way that you can be sure many of the 105,089 modules in CPAN were not written by economists.

Choose your software carefully. Make sure you understand and have tested the tools you're using. Also make sure you understand the true economics behind which ever implementation you chose. Getting locked into a closed software solution is more costly than just the licensing fees.

-
Thank you for your response. So are you suggesting that I try to learn something else? What would you then suggest I learn? –  SavedByJESUS Apr 4 '12 at 1:06
Nope, I use R almost completely. Are there some packages that your professor or other economists use a lot? I don't use any econ specific packages. I use plyr, matrix, and many other packages, but none are designed uniquely for economists. –  JD Long Apr 4 '12 at 1:37
So were you referring to "packages" and you said that I shouldn't get locked into a closed software solution? –  SavedByJESUS Apr 4 '12 at 1:44
"Openness allows any Tom, Dick, or Harry to write a crap implementation of any algorithm" - this isn't anything to do with openness, it's purely a result of having a public API, which many closed source products have (i.e. any proprietary software that has a plugin interface). Good answer though. –  naught101 Apr 4 '12 at 2:01
To continue with naught101 comment -- this is the concept behind Stata. It has a commercial core engine and 2142 packages in the Statistical Software Components archive written by third parties, a number comparable to that of R packages on CRAN. Most of these packages are not reliable -- I think I can tell that much as a professional Stata programmer. But the more complex ones usually are better, since it requires one to know some programming to really put together 1000+ lines of Stata code package that does something meaningful most of the time... –  StasK Oct 6 '12 at 2:47

It is not more or less reliable than other software. Base and recommended R is probably less prone to errors than contributed packages might be, but it depends on the authors.

But R's biggest advantage is that you can check yourself whether it is! It is free software, not like Stata or SPSS or similar. Hence even if it was unreliable, it would be detected eventually. That may not be the case for proprietary software. And you can even help make it more reliable.

For the rest of your professor's comments, he's clearly wrong and a person spreading FUD. But allow me to say that unreliable software should be the least of economist's concerns judging by the models and assumptions used and predictions made in this field.

Stick with R if you like it and maybe you and the professor can even contribute to developing good software for economics. Here's a possibly interesting starting point http://cran.r-project.org/web/views/Econometrics.html and http://cran.r-project.org/web/views/TimeSeries.html

-
Thank you very much! I really want to stick with R. I think it's a great software. Also, I've always been a big fan of open source. –  SavedByJESUS Apr 4 '12 at 0:59
+1 for the third paragraph. –  naught101 Apr 4 '12 at 2:02
+1 for stating an unfortunate truth of our time. "unreliable software should be the least of economist's concerns judging by the models and assumptions used and predictions made in this field." –  digitalmaps Apr 4 '12 at 3:21

Your professor makes some bold claims. I suspect that the problem was unfamiliarity with R language, not the actual results produced. I work in a company which does a lot of econometric modeling and we do everything in R. I also converted my economist colleague into using R.

Concerning field of economics in my personal experience, reliability issue might go the other way around. For example EVIEWS version 5 had some strange bugs when working with panel data. And it reported usual Durbin-Watson statistic for pooled OLS, which in panel-data setting is plain wrong. R package for working with panel data has its issues too, but the money argument here plays strongly in R favor.

Recently I was in course on non-stationary panel time series methods. The lecturer used RATS software. When demonstrating some code he advised clicking on some icon which cleans the workspace several times, just in case. Talk about reliability.

-

I am an economist and I have been working in research for 4 years now, mostly doing applied econometrics. There are plenty of econometrics packages out there, and there is room for all of them. In my view, in economics, Stata is used for almost everything but time series, Rats, Eviews and Ox are used for time series, Matlab and Gauss are used for more low level programming.

The advantage of R is that it is capable of doing almost everything the other programs do, and it's free and open. It requires some more programming and has less canned procedures, but it gets things done at the end. I use Stata most of the time, but if I had to choose one software to do everything, I would choose R.

R is pretty reliable on most econometrics problems, but I can provide examples of some routines written for R that are not reliable. I have had problems with 3SLS and demand system estimation routines. Numerical optimization routines are not as robust as in Stata or Gauss. On the other hand, R is much better at problems like quantile regression. Still, with a good working knowledge of R, you can find out what is the problem in R's user written routines, fix it, and continue working. So I don't think the lack of reliability in some specific routines is a compelling reason not to use R at all.

My advice would be to continue using R but to have experience on other program that is widely used in your field , e.g. Stata for microeconometrics or Rats for time series.

-

I'd be very careful of anyone who claims a fact but never backs it up with anything substantial.

You can easily turn his arguments around.

For example, people getting paid to write code could have LESS incentive to get it right because there is an expectation that their code will be correct, whereas the typical basement dweller wants to make a commit that will impress the project leaders. Maybe he couldn't care less about how much extra time he spends doing it for free if it means quality work gets done.

If the random number generator is 'messy' (which is a vague term; easily replacing a real fact to back up his argument), then he should be able to prove it or show you someone who can.

If he gets incoherent results from a package, he should be able to point out the steps he took to get that result. If it's really a bug and you have good programming skills, you can even try and fix it for him!

I realize my answer doesn't answer your question directly (sorry). Simply from the way he words his points, you can see there is no meat behind it. If there is, feel free to edit it in your question for people here to discuss it further!

-
When I was teaching graduate level statistics, I was telling my students: "I don't care what package you use, and you can use anything for your homework, as I expect you to provide substantive explanations, and will take points off if I see tr23y5m variable names in your submissions. I can support your learning very well in Stata, and reasonably well, in R. With SAS, you are on your own, as you claim you have taken a course in it. With SPSS or Minitab, God bless you". I imagine that the reasonable employers would think the same. What matters is your productivity in terms of the project outcomes. If you can achieve the goal in R with 40 hours of work, fine; if you can achieve it in C++ in 40 hours of work, fine; if you know how to do this in R in 40 hours, but your supervisor wants you to do this in SAS, and you have to spend 60 hours just to learn some basics and where the semicolons go, that can only be wise in the context of the large picture of the rest of the code being in SAS... and then the manager was not very wise in having hired an R programmer.
Update: As AndyW commented below, you can write terrible code in any language. The question of cost then becomes, which language is easier to debug. To me this looks like a combination of how accurate and informative the output is, and how easy and transparent the syntax itself is, and I don't have a good answer for that, of course. For instance, Python enforces code indenting, which is a good idea. Stata and R code can be folded over the brackets, and that's not going to work with SAS. Use of subroutines is a two-edged sword: the use of *apply() with ad-hoc functions in R is obviously very efficient, but harder to debug. By a similar token, Stata locals can mask nearly anything, and defaulting to an empty string, while useful, may also lead to difficult-to-catch errors.