0
$\begingroup$

Disclaimer

My ramble posed below may be considered off-topic for CV, but I thought I would at least try to post it here as I am curious as to others' thoughts/opinions, because I have not encountered a similar question elsewhere, and because the post involves times-series analysis. This all said, I would fully understand if the question is eventually closed. Alternatively, it may be migrated to a more appropriate sister site.

The Context

Many of us here on CV likely have developed R packages for statistical visualization/analysis, etc.

As a result, perhaps unsurprisingly, a natural question many of us ponder is: "how many times is my package actually downloaded?"

If we place our packages on GitHub (or some other similar version control environment), the CRAN downloads badge can provide some insight.

My Thoughts

A while back (circa 2015) Rob Hyndman had this exact same question regarding his "forecast" R package, which he wrote about on his Hyndsight blog: https://robjhyndman.com/hyndsight/fpp-downloads/. In it, Rob observed that 'forecast' was being dowloaded on average 24000 times per month.

Unlike Rob's "forecast" package, which has been on CRAN since 2009, my own package has only been on CRAN for 10 months. Yet, according to GitHub, my package has already seen over 3000 total downloads (>600 downloads/month). Further, unlike 'forecast', my package does a very specific thing. Given this, I thought it would be neat to apply forecasting to predict future downloads of CRAN packages.

It seems to me that all time series based on R package downloads would exhibit both trend and seasonality. This information in turn can give package developers/maintainers an idea of the best time to submit new packages or updates of existing packages to CRAN. A forecast of such time series can give maintainers an idea of how popular their packages are. It would be of no surprise that general packages, like those of the Tidyverse, would be downloaded en masse in the future.

Hyndman et al.'s newly minted 'tsibble', 'fable' and 'fasster' R packages (which all seem to be very powerful) can be employed to help shed light on future downloads of R packages.

The Details

For ease, I use base R capabilities below to forecast my own package downloads.

library(cranlogs)

y <- cran_downloads("HACSim", from = "2019-05-09")
y <- y[-277, ] # remove last value (0 downloads today, *i.e.*, not yet recorded by CRAN)

plot(y$date, y$count, type = "l", xlab = "Date", ylab = "Daily Number of Downloads", main = "Daily Number of HACSim Downloads")

enter image description here

Examination of ACF and PACF plots suggest an AR(10) process model is appropriate for my package count data, since autocorrelation in the PACF plot drops below 0 at lag 10 before the ACF plot does:

acf(y$count)
pacf(y$count)

enter image description here

As such, I used both the stats::ar.ols() and forecast::auto.arima() functions to estimate model parameters, which selected AR(7) and ARIMA(5, 1, 2) process models as follows:

(fit1 <- stats::ar.ols(y$count, order.max = 10))

Call:
ar.ols(x = y$count, order.max = 10) 

Coefficients:
      1        2        3        4        5        6        7  
 0.3711   0.0866  -0.0720   0.0976   0.0235  -0.0192   0.2358  

Intercept: -0.09344 (0.4495) 

(fit2 <- forecast::auto.arima(y$count))

Series: y$count 
ARIMA(5,1,2) 

Coefficients:
         ar1      ar2      ar3      ar4      ar5      ma1     ma2
      0.3952  -0.3625  -0.2665  -0.1098  -0.2637  -0.9904  0.4379
s.e.  0.1620   0.1461   0.0970   0.0851   0.1122   0.1589  0.1466

sigma^2 estimated as 85.17:  log likelihood=-1002.14
AIC=2020.28   AICc=2020.82   BIC=2049.25

I then computed forecasts for the next 21 days:

(pred1 <- forecast(fit1, h = 21)) # AR(7)
(pred2 <- forecast(fit2, h = 21)) # ARIMA(5, 1, 2)

and tested accuracy

(acc1 <- accuracy(pred1)) # AR(7)

                       ME     RMSE      MAE  MPE MAPE      MASE         ACF1
Training set 1.244434e-15 7.365306 4.903228 -Inf  Inf 0.7650033 -0.008316723

(acc2 <- accuracy(pred2)) # ARIMA(5, 1, 2)

                     ME     RMSE      MAE  MPE MAPE      MASE      ACF1
Training set -0.6544371 9.094689 5.560972 -Inf  Inf 0.8676248 0.1228663

which suggests fit1 is superior to fit2.

The visual forecasts for both models are as follows:

enter image description here

The average total number of downloads predicted over the next 21 days for each model is:

sum(pred1$mean) # AR(7)

[1] 296.0835

sum(pred2$mean) # ARIMA(5, 1, 2)

[1] 350.2503

My Questions

Clearly, such an analysis seems like a good idea for maintainers to perform, even if download numbers returned from CRAN logs may not be so accurate (as pointed out by Hadley Wickham in the comments to Rob's post).

Are any package maintainers doing such analyses out of curiosity, or for their own sanity to ensure their packages remain popular, and that precious time was not inadvertently wasted in the development process?

Has the CRAN team ever considered automating such analyses in some way to optimize say, priorities for package updates?

I would very much like to hear your thoughts on this, especially from the seasoned statisticians, package developers and package maintainers!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.