I will try to formulate an answer touching the main points where the two languages come into play for data science / statistics / data analysis and the like, as someone who uses both.
The workflow in data analysis generally consists of the following steps:
- Fetching the data from some sort of source (most likely a SQL/noSQL database or .csv files).
- Parsing the data in a decent and reasonable format (data frame) so that one can do operations and think thereupon.
- Applying some functions to the data (grouping, deleting, merging, renaming).
- Applying some sort of model to the data (regression, clustering, a neural network or any other more or less complicated theory).
- Deploying / presenting your results to a more-or-less technical audience.
Fetching data
99% of the time, the process of fetching the data comes down to querying some sort of SQL or Impala database: both Python and R have specific clients or libraries that do the job in no time and equally well (RImpala
, RmySQL
for R and MySQLdb
for Python work smoothly, not really much to add). When it comes to reading external .csv files, the data.table
package for R provides the function fread
that reads in huge and complicated .csv files with any custom parsing option in no time, and transforms the result directly into data frames with column names and row numbers.
Organising the data frames
We want the data to be stored in some sort of table so that we can access any single entry, row or column with ease.
The R package data.table
provides unbeatable ways to label, rename, delete and access the data. The standard syntax is very much SQL-like as dt[i, j, fun_by]
, where that is intended to be dt[where_condition, select_column, grouped_by (or the like)]
; custom user-defined functions can be put in there as well as in the j
clause, so that you are completely free to manipulate the data and apply any complicated or fancy function on groups or subsets (like take the i-th row, k-th element and sum it to the (k-2)-th element of the (i-1)-th row if and only if the standard deviation of the entire column is what-it-is, grouped by the last column altogether). Have a look at the benchmarks and at this other amazing question on SO. Sorting, deleting and re-naming of columns and rows do what they have to do, and the standard vectorised R methods apply, sapply, lapply, ifelse
perform vectorised operations on columns and data frames altogether, without looping through each element (remember that whenever you are using loops in R you are doing it badly wrong).
Python's counterweapon is the pandas
library. It finally provides the structure pd.DataFrame
(that standard Python lacks, for some reason still unknown to me) that treats the data for what they are, namely frames of data (instead of some numpy array, numpy list, numpy matrix
or whatever). Operations like grouping, re-naming, sorting and the like can be easily achieved and here, too, the user can apply any custom function to a grouped dataset or subset of the frame using Python apply
or lambda
. I personally dislike the grammar df[df.iloc(...)]
to access the entries, but that is just personal taste and no problem at all. Benchmarks for grouping operations are still slightly worse than R data.table
but unless you want to save 0.02 seconds for compilation there is no big difference in performance.
Strings
The R way to treat strings is to use the stringr
package that allows any text manipulation, anagram, regular expression, trailing white spaces or similar with ease. It can also be used in combination with JSON libraries that unpack JSON dictionaries and unlist their elements, so that one has a final data frame where the column names and the elements are what they have to be, without any non-UTF8 character or white space in there.
Python's Pandas .str.
does the same job of playing with regular expressions, trailing or else as good as its competitor, so even here no big difference in taste.
Applying models
Here is where, in my opinion, differences between the two languages arise.
R has, as of today, an unbeatable set of libraries that allow the user to essentially do anything they want in one to two lines of code. Standard functional or polynomial regressions are performed in one-liners and produce outputs whose coefficients are easily readable, accompanied by their corresponding confidence intervals and p-values distributions. Likewise for clustering, likewise for random forest models, likewise for dendograms, principal component analysis, singular value decompositions, logistic fits and many more. The output for each of the above most likely comes with a specific plotting class that generates visualisations of what you have just done, with colours and bubbles for coefficients and parameters. Hypotheses tests, statistical tests, Shapiro, Kruskal-Wallis or the like can be performed in one line of code by means of appropriate libraries.
Python is trying to keep up with SciPy
and scikit-learn
. Most of the standard analysis and models are available as well, but they are slightly longer to code and less-intuitive to read (in my opinion). More complicated machineries are missing, although some can be traced back to some combinations of the already existing libraries. One thing that I prefer doing in Python rather than in R is bag-of-word text analysis with bi-grams, tri-grams and higher orders.
Presenting the results
Both languages have beautiful plotting tools, R ggplot2
above all and the corresponding Python equivalent. Not really much to compete, they do the job safe and sound, although I believe that if you are presenting the results you may have to use other tools—there are fancy colourful design tools out there and neither Python nor R are meant to astonish the audience with fancy red-and-green drag and drops. R has lately published a lot of improvements on its shiny app
features, that basically allow it to produce interactive outputs. I never wanted to learn it, but I know it is there and people use it well.
Side note
As a side note I would like to emphasise that the major difference between the two languages is that Python is a general purpose programming langauge, made by and for computer science, portability, deployments and so on and so forth. It is awesome at what it does and is straightforward to learn; there is nobody who does not like python. But it is a programming language to do programming.
R, on the other hand, was invented by and for mathematicians, physicists, statisticians and data scientists. If you come from that background everything makes perfect sense because it perfectly mirrors and reproduces the concepts used in statistics and mathematics. But if, instead, you come from a computer science background and want to simulate Java or C in R you are going to be disappointed; it does not have "objects" in the standard sense (well, it does, but not what one typically thinks they are...), it does not have classes in the standard sense (well, it does, but not what one typically thinks they are...), it does not have "pointers" or all other computer science structures - but just because it does not need them. Last but not the least: documentation and packages are straightforward to create and read (if you are using Rstudio); there is a large and passionate community out there, and it takes literally five seconds to Google "how to do insert-random-problem in R" whose first entry redirects you to a solution to the problem (done by someone else) with corresponding code, in no time.
Most industrial companies have their infrastructure built in Python (or a Python-friendly environment) that allows easy integration of Python code (just import myAnalysis
anywhere and you are basically done). However, any modern technology or server or platform easily runs background R code without any problem as well.