The Apache Arrow project has a handy guide for cutting down R package installation time on Linux: https://cran.r-project.org/web/packages/arrow/vignettes/install.html
But the RSPM suggestion didn’t work for me:
install.packages("arrow", repos = "https://packagemanager.rstudio.com/cran/__linux__/focal/latest")
Installing package into '/home/ubuntu/R/x86_64-pc-linux-gnu-library/4.1'
(as 'lib' is unspecified)
trying URL 'https://packagemanager.rstudio.com/cran/__linux__/focal/latest/src/contrib/arrow_7.0.0.tar.gz'
Content type 'binary/octet-stream' length 4572465 bytes (4.4 MB)
==================================================
downloaded 4.4 MB
* installing *source* package 'arrow' ...
** package 'arrow' successfully unpacked and MD5 sums checked
** using staged installation
*** Found local C++ source: 'tools/cpp'
*** Building libarrow from source
For a faster, more complete installation, set the environment variable NOT_CRAN=true before installing
See install vignette for details:
https://cran.r-project.org/web/packages/arrow/vignettes/install.html
**** arrow
PKG_CFLAGS=-I/tmp/RtmphYTlJ7/R.INSTALL14fae676c8045/arrow/libarrow/arrow-7.0.0/include -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -DARROW_R_WITH_JSON
PKG_LIBS=-L/tmp/RtmphYTlJ7/R.INSTALL14fae676c8045/arrow/libarrow/arrow-7.0.0/lib -larrow_dataset -lparquet -larrow -larrow /usr/lib/x86_64-linux-gnu/libbz2.so -pthread -larrow_bundled_dependencies -lz -llz4 -lzstd -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -lssl -lcrypto -lcurl
...
This is definitely not a binary package! What’s more, this is pretty consistent with the experience I’ve always had with the RSPM: I’ve never had it successfully serve me a binary package, it has always sent me something that needs compilation. The epic compilation time of {arrow} motivated me to look into this though, and this is what I found: https://community.rstudio.com/t/unable-to-install-binary-packages-from-packagemanager-rstudio-com-on-linux/82161
I needed to add this line to my .Rprofile
:
options(HTTPUserAgent = sprintf("R/%s R (%s)", getRversion(), paste(getRversion(), R.version$platform, R.version$arch, R.version$os)))
And now I get:
install.packages('arrow')
Installing package into ‘/home/ubuntu/R/x86_64-pc-linux-gnu-library/4.1’
(as ‘lib’ is unspecified)
trying URL 'https://packagemanager.rstudio.com/all/__linux__/focal/latest/src/contrib/arrow_7.0.0.tar.gz'
Content type 'binary/octet-stream' length 29595575 bytes (28.2 MB)
==================================================
downloaded 28.2 MB
* installing *binary* package ‘arrow’ ...
* DONE (arrow)
The downloaded source packages are in
‘/tmp/RtmpbZPQja/downloaded_packages’
I’ve heard a lot of people talking about Linux binaries via RPSM over the last couple of years, but never mentioned this HTTPUserAgent
issue. I suspect there are a lot of people who think they are getting binary installs on their servers that are still doing compilation! Definitely worth checking!