# Tag Info

96

I think the key will be whether or not libraries start being developed for Julia. It's all well and good to see toy examples (even if they are complicated toys) showing that Julia blows R out of the water at tasks R is bad at. But poorly done loops and hand coded algorithms are not why many of the people I know who use R use R. They use it because for ...

56

I agree with a lot of the other comments. "Hope"? Sure. I think Julia has learned a lot from what R and Python/NumPy/Pandas and other systems have done right and wrong over the years. If I were smarter than I am, and wanted to write a new programming language that would be the substrate for a statistical development environment in the future, it would look ...

35

For me, one very important thing for a data analysis language is to have query/relational algebra functionality with reasonable defaults and interactively-oriented design, and ideally this should be a built-in of the language. IMO, no FOSS language that I've used does this effectively, not even R. data.frame is very clunky to work with interactively - for ...

26

I can see Julia replacing Matlab, which would be a huge service for humanity. To replace R, you'd need to consider all of the things that Neil G, Harlan, and others have mentioned, plus one big factor that I don't believe has been addressed: easy installation of the application and its libraries. Right now, you can download a binary of R for Mac, Windows, ...

24

The Julia language is pretty new; it's time in the spot light can be measured in weeks (even though its development time can of course be measured in years). Now those weeks in the spot light were very exciting weeks---see for example the recent talk at Stanford where "it had just started"---but what you ask for in terms of broader infrastructure and package ...

22

You can do this with R, which may be a bit of overkill... EDIT 2: [OOPS, looks like someone else hit with Rscript while I was retyping this.] I found an easier way. Installed with R should be Rscript, which is meant to do what you're trying to do. For example, if I have a file bar which has a list of numbers, one per line: cat bar | Rscript -e 'summary (as....

21

Try "st": $seq 1 10 | st N min max sum mean stddev 10 1 10 55 5.5 3.02765$ seq 1 10 | st --transpose N 10 min 1 max 10 sum 55 mean 5.5 stddev 3.02765 You can also see the five number summary: $seq 1 10 | st --summary min q1 median q3 max 1 3.5 5.5 7.5 10 You can download it here: https:... 19 Bruce Tate here, author of Seven Languages in Seven Weeks. Here are a few thoughts. I am working on Julia for the followup book. The following is just my opinion after a few weeks of play. There are two fundamental forces at play. First, all languages have a lifespan. R will be replaced some day. We don't know when. New languages have an extremely ... 18 R works in-memory - so your data do need to fit into memory for the majority of functions. The compiler package, if I am thinking of the thing you are thinking of (Luke Tierney's compiler package supplied with R), is not the same thing as a compiled language in the traditional sense (C, Fortran). It is a byte compiler for R in the sense of Java bytecode ... 15 Every time I see a new language, I ask myself why an existing language can't be improved instead. Python's big advantages are a rich set of modules (not just statistics, but plotting libraries, output to pdf, etc.) language constructs that you end up needing in the long run (objected-oriented constructs you need in a big project; decorators, closures, etc. ... 15 I have switched to Julia, and here are my pragmatic reasons: It does glue code really well. I have a lot of legacy code in MATLAB, and MATLAB.jl took 5 minutes to install, works perfectly, and has a succinct syntax that makes it natural to use MATLAB functions. Julia also has the same for R, Python, C, Fortran, and many other languages. Julia does ... 13 I have used SAS for 15 years, and have started using R seriously the past 6 months, with some tinkering around in it for a couple of years ahead of that. From a programming perspective, R does data manipulations directly, there is no equivalent to DATA or PROC SQLprocedures because they're not needed (the latter being more efficient in SAS when there is a ... 13 It's a little tough to provide specific recommendations, particularly without knowing too much about your budget and goals. However: A lot of data analysis can now be done on...nearly anything. If you plan on doing a lot of$t$-tests, ANOVAs, or regression modeling, I think you would be hard-pressed to find a system that was too slow, even with relatively ... 11 I think "learn X over Y" isn't the right way to formulate the question. In fact, you can learn (at least basics of) both and decide on the right tool depending on concrete task at hand. And since Julia inherited most of its syntax and concepts from other languages, it shoud be really easy to grasp it (as well as Python, though I'm not sure the same may be ... 10 R provides a command called Rscript. If you have only a few numbers that you can paste on the command line, use this one liner: Rscript -e 'summary(as.numeric(commandArgs(TRUE)))' 3 4 5 9 7 which results in Min. 1st Qu. Median Mean 3rd Qu. Max. 3.0 4.0 5.0 5.6 7.0 9.0 If you want to read from the standard input use this: ... 10 R is a programming language. It works not in datasteps. It does whatever you want it to do, for it is but a programming language, a slave for your desires, expressed in a language of curly brackets and colons. Think of it like Fortran or C, but with implicit vectorisation so you don't have to loop over arrays, and dynamic memory management so you don't have ... 9 Julia will not take over R very soon. Check out Microsoft R open. https://mran.revolutionanalytics.com/open/ This is an enhanced version of R that automatically uses all the cores of your computer. It is the same R, same language, same packages. When you install it, RStudio will also use it in the console. The speed of MRO is even faster than Julia. I do a ... 8 I think it's unlikely that Julia will ever replace R, for a lot of the reasons previously mentioned. Julia is a Matlab replacement, not a R replacement; they have different goals. Even after Julia has a fully-fleshed out statistics library, no one would ever teach an Intro to Statistics class in it. However, an area in which it could be incredible is as a ... 8 The following probably does not deserve to be an answer, but it is too important to be buried as a comment to someone else's response... I have not heard much said about memory consumption, just speed. R's entire semantics being pass-by-value can be painful, and this has been one criticism of the language (which is a separate issue from how many great ... 8 I am a Julia newbie, and am R competent. The reasons I find Julia interesting so far are performance and compatibility oriented. GPU tools. I'd like to use CUSPARSE for a statistical application. CRAN results indicate there's not much out there. Julia has bindings available which seem to work smoothly so far. using CUSPARSE N = 1000 M = 1000 hA = sprand(N, ... 8 There are two different algorithms used here. The naive approach is to list all the thresholds of the ROC curve (typically$O(N)$with continuous variables), and calculate sensitivity and specificity on each of them (again$O(N)$). Because of the two$O(N)$, this has a worst-case of$O(N^2)$. But it can be surprisingly efficient when the predictor takes ... 8 If the matter is numerical stability, you could look at the log of the hazard function: $$log(h(t; \theta)) = log(f(t;\theta)) - log(1-F(t;\theta))$$ You could use the log / log.p = TRUE flag in R for log values and the lower.tail flag for obtaining$log(1 - F(t;\theta))$values: dweibull(100,1,1, log = T) # -100 pweibull(100, 1, 1, log.p = TRUE, lower.... 7 I understand that by default SAS can work with models that are bigger than memory, but this is not the case with R, unless you specifically use packages like biglm or ff. However, if you are doing array work in R that can be vectorised it will be very quick - maybe half the speed of a C program in some cases, but if you are doing something that can't be ... 7 Julia 1.0 has just come out with a very usable IDE (Juno). It came out a bit late to the party as Python has already dominated Machine Learning, while R continues to dominate every other kind of statistical analysis. That being said, Julia is already rising to prominence in the area of finance and trading algorithms as fast development time AND execution are ... 7 The mistake you made was to ignore the covariances between the estimators$\hat{\beta}_0$,$\hat{\beta}_1$and$\hat{\beta}_2$. You want a 95% confidence interval for $$E(Y|X=0.25) = \beta_0 + 0.25\beta_1 + 0.25^2\beta_2 = a\beta,$$ with$a = [1\ \ 0.25\ \ 0.25^2]$and$\beta = [\beta_0\ \ \beta_1\ \ \beta_2]^T$. Under the usual assumptions (normally ... 5 I am interested by the promise of better speed and easy parallelisation using different architectures. For that reason I will certainly watch Julia development but I am unlikely to use it until it can handle generalised linear mixed models, the has a good generic bootstrap package, a simple model language for building design matrices the capability ... 5 Using QR decomposition (which ought to be available if you already have calculated the regression): Let$X$have$n$rows and$p$columns and be of full column rank.$H=X(X'X)^{-1}X'=QR(R'Q'QR)^{-1}R'Q'=QR(R'R)^{-1}R'Q'$But if$R_1$is the first$p$rows of$R$then$R'R=R_1'R_1=QR(R_1'R_1)^{-1}R'Q'$Now let$Q=(Q_1,Q_2)$where$Q_1\$ is the ...

4

The luxury of NA's in R does not come without performance penalties. If Julia supports NA's with a smaller performance penalty then it becomes interesting to a segment of the stats community, but NA's also impose considerable extra work when using compiled code with R. Many of the packages in R rely on routines written in legacy languages (C, Fortran, or ...

4

I will be up front, I have no experience with R, but I work with plenty of people that think it is an excellent tool for statistical analysis. My background is in data warehousing, and due to Julia's easily distributed, but more standard programming model, I think it could be a very interesting substitute for the transform portion of traditional ETL tools ...

Only top voted, non community-wiki answers of a minimum length are eligible