64
$\begingroup$

I am a graduate student in economics who recently converted to R from other very well-known statistical packages (I was using SPSS mainly). My little problem at the moment is that I am the only R user in my class. My classmates use Stata and Gauss and one of my professors even said that R is perfect for engineering, but not for economics. He said that many packages are built by people who know a lot about programming, but not much about economics and therefore are not reliable. He also mentioned the fact that since no money is actually involved in building an R package, there is therefore no incentive to do it correctly (unlike in Stata for example) and that he used R for a time and got some "ridiculous" results in his attempts to estimate some stuff. Moreover, he complained about he random number generator in R which he said was "messy".

I've been using R for just a little more than a month and I must say I have fallen in love with it. All this stuff I am hearing from my professor is just discouraging me.

So my question is: "Is R reliable for the field of economics?".

$\endgroup$
  • 30
    $\begingroup$ "many packages are built by people who know a lot about programming, but not much about economics". Package authors are practically always scientists or academics first and programmers a (very) distant (last) second. Actually, I think it would be a challenge to find a package authored by a "programmer". $\endgroup$ – Brandon Bertelsen Apr 4 '12 at 3:09
  • 9
    $\begingroup$ Can I invite you and our other stats economists over to economics.stackexchange.com too - we'll welcome your questions and your answers on economics there $\endgroup$ – EnergyNumbers Apr 4 '12 at 5:00
  • 18
    $\begingroup$ It seems there is also a bit of professional chauvinism in your professor's comments. How is being an economist a guarantee of reliability? I had rather trust software developers (it's actually a profession, not something anybody can do well without particular experience or training) and statisticians to produce reliable statistical software. $\endgroup$ – Gala Apr 4 '12 at 6:17
  • 23
    $\begingroup$ "Perfect for engineering but not for economics" because of reliability? He'd rather a skyscraper collapsed than an economy? The man's an idiot. Quit that school and find one not staffed by idiots. $\endgroup$ – Spacedman Apr 4 '12 at 7:02
  • 10
    $\begingroup$ Commercial software can be good or bad. Open source software can be good or bad. What matters is whether or not the software you use is any good. Don't decide that based on prejudice and dogma. Use real evidence. $\endgroup$ – David Heffernan Apr 4 '12 at 7:19
83
$\begingroup$

Let me share a contrasting view point. I'm an economist. I was trained in econometrics using SAS. I work in financial services and just tonight I updated R based models which we will use tomorrow to put millions of dollars at risk.

Your professor is just plain wrong. But the mistake he's making is VERY common and is worth discussing. What your professor seems to be doing is commingling the idea of the R software (the GNU implementation of the S language) vs. packages (or other code) implemented in R. I can write crap implementations of a linear regression using SAS IML. As a matter of fact, I've done that very thing. Does that mean SAS is crap? Of course not. SAS is crap because their pricing is non-transparent, ridiculously expensive, and their in house consultants over promise, under deliver, and charge a premium for the pleasure. But I digress...

The openness of R is a double edged sword: Openness allows any Tom, Dick, or Harry to write a crap implementation of any algorithm they think up while smoking pot in the basement of the economics building. The same openness allows practicing economists to share code openly and improve on each other's code. The licensing rules with R mean that I can write parallelization code for running R in parallel on Amazon's cloud and not have to worry about licensing fees for a 30 node cluster. This is a HUGE win for simulation based analysis which is a big part of what I do.

Your professor's comment that "many packages are built by people who know a lot about programming, but not much about economics" is, no doubt, correct. But there are 3716 packages on CRAN. You can be damn sure many of them were not written by economists. In the same way that you can be sure many of the 105,089 modules in CPAN were not written by economists.

Choose your software carefully. Make sure you understand and have tested the tools you're using. Also make sure you understand the true economics behind which ever implementation you chose. Getting locked into a closed software solution is more costly than just the licensing fees.

$\endgroup$
  • $\begingroup$ Thank you for your response. So are you suggesting that I try to learn something else? What would you then suggest I learn? $\endgroup$ – SavedByJESUS Apr 4 '12 at 1:06
  • 5
    $\begingroup$ Nope, I use R almost completely. Are there some packages that your professor or other economists use a lot? I don't use any econ specific packages. I use plyr, matrix, and many other packages, but none are designed uniquely for economists. $\endgroup$ – JD Long Apr 4 '12 at 1:37
  • $\begingroup$ So were you referring to "packages" and you said that I shouldn't get locked into a closed software solution? $\endgroup$ – SavedByJESUS Apr 4 '12 at 1:44
  • 3
    $\begingroup$ "Openness allows any Tom, Dick, or Harry to write a crap implementation of any algorithm" - this isn't anything to do with openness, it's purely a result of having a public API, which many closed source products have (i.e. any proprietary software that has a plugin interface). Good answer though. $\endgroup$ – naught101 Apr 4 '12 at 2:01
  • 3
    $\begingroup$ "There is not now, nor has there ever been, nor will there ever be, any programming language in which it is the least bit difficult to write bad code." $\endgroup$ – ardave Mar 18 '16 at 20:53
33
$\begingroup$

It is not more or less reliable than other software. Base and recommended R is probably less prone to errors than contributed packages might be, but it depends on the authors.

But R's biggest advantage is that you can check yourself whether it is! It is free software, not like Stata or SPSS or similar. Hence even if it was unreliable, it would be detected eventually. That may not be the case for proprietary software. And you can even help make it more reliable.

For the rest of your professor's comments, he's clearly wrong and a person spreading FUD. But allow me to say that unreliable software should be the least of economist's concerns judging by the models and assumptions used and predictions made in this field.

Stick with R if you like it and maybe you and the professor can even contribute to developing good software for economics. Here's a possibly interesting starting point http://cran.r-project.org/web/views/Econometrics.html and http://cran.r-project.org/web/views/TimeSeries.html

$\endgroup$
  • $\begingroup$ Thank you very much! I really want to stick with R. I think it's a great software. Also, I've always been a big fan of open source. $\endgroup$ – SavedByJESUS Apr 4 '12 at 0:59
  • 16
    $\begingroup$ +1 for stating an unfortunate truth of our time. "unreliable software should be the least of economist's concerns judging by the models and assumptions used and predictions made in this field." $\endgroup$ – digitalmaps Apr 4 '12 at 3:21
  • 1
    $\begingroup$ I appreciate the wry tone here, but some disagreement is possible. Errors in proprietary software can be shown up in various ways, e.g. if there is clear disagreement with results in R that appear totally correct. There is some FUD here about proprietary software that is just as inappropriate as silly FUD about R. Checkability in R is checkability in principle for virtually all R users; it's a feature that it exists, but saying that you can check the code for yourself is a little rhetorical too. Note that for Stata much of the code is visible to users; it's just the executable that isn't. $\endgroup$ – Nick Cox Mar 27 '15 at 12:58
23
$\begingroup$

Your professor makes some bold claims. I suspect that the problem was unfamiliarity with R language, not the actual results produced. I work in a company which does a lot of econometric modeling and we do everything in R. I also converted my economist colleague into using R.

Concerning field of economics in my personal experience, reliability issue might go the other way around. For example EVIEWS version 5 had some strange bugs when working with panel data. And it reported usual Durbin-Watson statistic for pooled OLS, which in panel-data setting is plain wrong. R package for working with panel data has its issues too, but the money argument here plays strongly in R favor.

Recently I was in course on non-stationary panel time series methods. The lecturer used RATS software. When demonstrating some code he advised clicking on some icon which cleans the workspace several times, just in case. Talk about reliability.

$\endgroup$
20
$\begingroup$

I am an economist and I have been working in research for 4 years now, mostly doing applied econometrics. There are plenty of econometrics packages out there, and there is room for all of them. In my view, in economics, Stata is used for almost everything but time series, Rats, Eviews and Ox are used for time series, Matlab and Gauss are used for more low level programming.

The advantage of R is that it is capable of doing almost everything the other programs do, and it's free and open. It requires some more programming and has less canned procedures, but it gets things done at the end. I use Stata most of the time, but if I had to choose one software to do everything, I would choose R.

R is pretty reliable on most econometrics problems, but I can provide examples of some routines written for R that are not reliable. I have had problems with 3SLS and demand system estimation routines. Numerical optimization routines are not as robust as in Stata or Gauss. On the other hand, R is much better at problems like quantile regression. Still, with a good working knowledge of R, you can find out what is the problem in R's user written routines, fix it, and continue working. So I don't think the lack of reliability in some specific routines is a compelling reason not to use R at all.

My advice would be to continue using R but to have experience on other program that is widely used in your field , e.g. Stata for microeconometrics or Rats for time series.

$\endgroup$
13
$\begingroup$

When I was teaching graduate level statistics, I was telling my students: "I don't care what package you use, and you can use anything for your homework, as I expect you to provide substantive explanations, and will take points off if I see tr23y5m variable names in your submissions. I can support your learning very well in Stata, and reasonably well, in R. With SAS, you are on your own, as you claim you have taken a course in it. With SPSS or Minitab, God bless you". I imagine that the reasonable employers would think the same. What matters is your productivity in terms of the project outcomes. If you can achieve the goal in R with 40 hours of work, fine; if you can achieve it in C++ in 40 hours of work, fine; if you know how to do this in R in 40 hours, but your supervisor wants you to do this in SAS, and you have to spend 60 hours just to learn some basics and where the semicolons go, that can only be wise in the context of the large picture of the rest of the code being in SAS... and then the manager was not very wise in having hired an R programmer.

From this perspective of the total cost, "free" R is a hugely overblown myth. Any serious project requires custom code, if just for the data input and formatting the output, and that's a non-zero cost of professional time. If this data input and formatting requires 10 hours of SAS code and 20 hours of R code, R is a more expensive software at the margin, as an economist would say, i.e., in terms of the additional cost to produce a given piece of functionality. If a big project requires 200 hours of R programmer's time and 100 hours of Stata programmer's time to provide identical functionality, Stata is cheaper overall, even accounting for the ~$1K license that you need to buy. It would be interesting to see such direct comparisons; I was involved in re-writing a huge mess of 2Mb of SPSS code that was said to have been accumulated over about 10 person-years into ~150K of Stata code that ran about as fast, may be a tad faster; that was about 1 person-year project. I don't know if this 10:1 efficiency ratio is typical for SPSS:Stata comparisons, but I won't be surprised if it were. For me, working with R is always a large expense because of the search costs: I have to determine which of the five packages with similar names does what I need to do, and gauge whether it does it reliably enough for me to use it in my work. It often means that it is cheaper for me to write my own Stata code in less time that I would be spending figuring out how to make R work in a given task. It should be understood that this is my personal idiosyncrasy; most people on this site are better useRs than I am.

Funny that your prof would prefer Stata or GAUSS over R because "R was not written by economists". Neither were Stata or GAUSS; they are written by computer scientists using computer scientists' tools. If your prof gets ideas about programming from CodeAcademy.com, that's better than nothing, but professional grade software development is as different from typing in CodeAcademy.com text box as driving a freight truck is different from biking. (Stata was started by a labor econometrician converted computer scientist though, but he has not been doing this labor econometrics thing for about 25 years by now.)

Update: As AndyW commented below, you can write terrible code in any language. The question of cost then becomes, which language is easier to debug. To me this looks like a combination of how accurate and informative the output is, and how easy and transparent the syntax itself is, and I don't have a good answer for that, of course. For instance, Python enforces code indenting, which is a good idea. Stata and R code can be folded over the brackets, and that's not going to work with SAS. Use of subroutines is a two-edged sword: the use of *apply() with ad-hoc functions in R is obviously very efficient, but harder to debug. By a similar token, Stata locals can mask nearly anything, and defaulting to an empty string, while useful, may also lead to difficult-to-catch errors.

$\endgroup$
  • 1
    $\begingroup$ This is a fine answer, but IMO over-generalizations aren't very helpful (you can write bad or good code in practically any language, Stata isn't magical at enforcing good coding standards). I have a hard time seeing how 2mb of SPSS code is efficient SPSS code to begin with (with the newer versions with syntax highlighting it would be ill advised to even open such a file in the editor). Seems there is a good chance it was time well spent re-writing in any language. $\endgroup$ – Andy W Oct 9 '12 at 15:14
  • 1
    $\begingroup$ I don't write SPSS code at all, and my understanding that the code I worked with was not written as code, either, but rather saved from the point and click interface, and then may be brushed up a little bit. I would imagine that SPSS would put all the defaults and such with the interface-to-code conversion, so it was not a terribly efficient code to begin with. $\endgroup$ – StasK Oct 10 '12 at 16:12
12
$\begingroup$

I'd be very careful of anyone who claims a fact but never backs it up with anything substantial.

You can easily turn his arguments around.

For example, people getting paid to write code could have LESS incentive to get it right because there is an expectation that their code will be correct, whereas the typical basement dweller wants to make a commit that will impress the project leaders. Maybe he couldn't care less about how much extra time he spends doing it for free if it means quality work gets done.

If the random number generator is 'messy' (which is a vague term; easily replacing a real fact to back up his argument), then he should be able to prove it or show you someone who can.

If he gets incoherent results from a package, he should be able to point out the steps he took to get that result. If it's really a bug and you have good programming skills, you can even try and fix it for him!

I realize my answer doesn't answer your question directly (sorry). Simply from the way he words his points, you can see there is no meat behind it. If there is, feel free to edit it in your question for people here to discuss it further!

$\endgroup$
5
$\begingroup$

In the ReplicationWiki (that I work on) you can see that R was one of the software packages used most often for some 2000 empirical studies published in some well established journals already in the years 2000-2013. It seems that it was more used in more recent years. Stata was used by far most often (>900 times), followed by MATLAB (280), SAS (60), GAUSS (60), Excel (50), R (30), FORTRAN (30), Mathematica (19), EViews (18), z-Tree (16), dynare (15), RATS (12), C (8), C++ (6), python (5, more recent studies), SPSS (5) and some others. Often times more than one package is used.

$\endgroup$
  • $\begingroup$ This is interesting evidence about software usage. But it doesn't bear directly on the question other than by providing clear indications that R is widely used (the inference that it is widely trusted too is germane). $\endgroup$ – Nick Cox Feb 13 '16 at 10:14
2
$\begingroup$

I have been using R for half a decade and also use SAS, SPSS, Calc, WEKA and a couple of other tools. I never enjoyed with any tool as much as it was through R. Basically R is for those who think independently and try something on their own learning. When it comes to statistics it is all about methods. Users might not be knowing as how methods were defined and modeled in commercial software and they might be correct or wrong. R is for those who would like to define methods and use those methods that befits for their needs. It is all about freedom. This freedom is not there with commercial software despite of spending money and buying them. Knowledge is the property of community (society) nobody can claim authorship on the same. Research is all about finding solutions for problems. As far as R is concerned one need not worry about methods for the users are free to define and revamp. For instance, if there exists any model specific problem or erratically defined methods that can be fixed by either fixing or developing a new code. By doing so a researcher not only develops knowledge but also evolves.

The advantage of R is that one need not be a computer programmer. Statistical methods are all about writing functions just with control statements and loops (to start with, The higher level things comes later). R has very easy programming environment for newbies.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.