Problem with install_version across multiple repositories

[colearendt]

It seems like install_version just checks one of them? Maybe the last one? 0.1.0.9000 is the latest version for internal/latest, and install.packages works, so I would have expected install_version to work too.

Further, I would expect that install_version should work for older versions in both repositories.

options("repos" = c(
  RSPM_DEV = "https://demo.rstudiopm.com/internal/latest", 
  RSPM_PROD = "https://demo.rstudiopm.com/cran/latest")
  )

remotes::install_version("sass", "0.1.0.9000")
#> Trying https://demo.rstudiopm.com/internal/latest
#> Trying https://demo.rstudiopm.com/cran/latest
#> Error in download_version_url(package, version, repos, type): version '0.1.0.9000' is invalid for package 'sass'

remotes::install_version("sass", "0.1.0.9000", repos = "https://demo.rstudiopm.com/internal/latest")
#> Downloading package from url: https://demo.rstudiopm.com/internal/latest/src/contrib/sass_0.1.0.9000.tar.gz
#> Your system is ready to build packages!
#> Skipping 1 packages not available: digest
#> Installing package into '/home/rstudio-admin/dev-proj/renv/library/R-3.5/x86_64-pc-linux-gnu'
#> (as 'lib' is unspecified)

install.packages("sass")
#> Installing package into '/home/rstudio-admin/dev-proj/renv/library/R-3.5/x86_64-pc-linux-gnu'
#> (as 'lib' is unspecified)

^{Created on 2019-08-21 by the reprex package (v0.3.0)}

[jimhester]

Try this PR #305, it seems like it implements what you are trying to do.

Really the code currently in remotes was never meant to do what you are attempting, it assumes only one repository will have an archive, because historically the main CRAN repository was the only one which had one.

[colearendt]

Very interesting! Thanks! It looks very similar. Unfortunately, I don't think it does... although maybe I'm missing something:

> packageDescription("remotes")
...
GithubRepo: remotes
GithubUsername: kenahoo
GithubRef: install_version-multi2
GithubSHA1: ffc24bf15bbcc96fd0976865eccb8879b8fc425d
NeedsCompilation: no
Packaged: 2019-08-21 13:57:51 UTC; rstudio-admin
Author: Jim Hester [aut, cre], Gábor Csárdi [aut], Hadley Wickham [aut], Winston Chang
          [aut], RStudio [cph], Martin Morgan [aut], Dan Tenenbaum [aut], Ken Williams
          [ctb], Mango Solutions [cph]
Maintainer: Jim Hester <james.hester@rstudio.com>
Built: R 3.5.3; ; 2019-08-21 13:57:51 UTC; unix

-- File: /home/rstudio-admin/dev-proj/renv/library/R-3.5/x86_64-pc-linux-gnu/remotes/Meta/package.rds 
> remotes::install_version("sass", "0.1.0.9000")
Trying https://demo.rstudiopm.com/internal/latest
Trying https://demo.rstudiopm.com/cran/latest
Error in download_version_url(package, version, repos, type) : 
  version '0.1.0.9000' is invalid for package 'sass'

[kenahoo]

@colearendt I downloaded the archive.rds files for those two repositories:

• https://demo.rstudiopm.com/internal/latest/src/contrib/Meta/archive.rds
• https://demo.rstudiopm.com/cran/latest/src/contrib/Meta/archive.rds

The former doesn't have sass in it at all, and the latter only has a single version in it right now, sass_0.1.1.tar.gz.

And the latest version in the latter repository seems to be 0.1.2.1. So I'm not sure where 0.1.0.9000 is being housed right now, do you still see it?

[colearendt]

Interesting! Is Meta/archive.rds a standard for package information? Or is that only old / non-current versions of packages? I'm unfamiliar with that one! If you check the PACKAGES file, you can see the sass package there with that version:

$ curl https://demo.rstudiopm.com/internal/latest/src/contrib/PACKAGES | grep 'sass' -A 2
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3834    0  3834    0     0  26983      0 --:--:-- --:--:-- --:--:-- 27000
Imports: checkmate (>= 1.8.5), commonmark (>= 1.5), dplyr (>= 0.7.6), ggplot2 (>= 3.0.0), glue (>= 1.2.0), htmltools (>= 0.3.6), magrittr (>= 1.5), rlang (>= 0.2.1), scales (>= 1.0.0), sass (>= 0.1.0), stringr (>= 1.3.1), tibble (>= 1.4.2), tidyr (>= 0.8.1), tidyselect (>= 0.2.4)
Suggests: knitr, testthat, rmarkdown, rvest, shiny, xml2
License: MIT + file LICENSE
--
Package: sass
Version: 0.1.0.9000
Imports: digest

Further, the package itself is here:

curl -I https://demo.rstudiopm.com/internal/latest/src/contrib/sass_0.1.0.9000.tar.gz
HTTP/1.1 200 OK
Date: Wed, 21 Aug 2019 18:39:27 GMT
Content-Type: application/x-gzip
Content-Length: 499955
Connection: keep-alive
Last-Modified: Tue, 04 Dec 2018 02:51:45 GMT
Server: RStudio Package Manager v1.0.10-2
X-Content-Type-Options: nosniff
X-Frame-Options: DENY

[kenahoo]

Aha, @colearendt , you're quite correct. I checked both the archive.rds files, and the PACKAGES file from the "cran" repo, but I forgot to check the "internal" PACKAGES file. I'm not sure why my patched version didn't find it, but I think I can find time to investigate it...

[kenahoo]

This case wasn't covered by the existing unit tests, and indeed it failed. I wrote a unit test for this and some other motivating cases, and it should work now as expected. Thanks for finding the issue, @colearendt .

[colearendt]

Thanks so much for the fix! I will say I may be doing something crazy, but it still seems to be failing for me. Reprex:

options("repos" = c(
  RSPM_DEV = "https://demo.rstudiopm.com/internal/latest", 
  RSPM_PROD = "https://demo.rstudiopm.com/cran/latest")
  )

remotes::install_version("sass", "0.1.0.9000")
#> Trying https://demo.rstudiopm.com/internal/latest
#> Trying https://demo.rstudiopm.com/cran/latest
#> Error in download_version_url(package, version, repos, type): version '0.1.0.9000' is invalid for package 'sass'

packageDescription("remotes")
#> Package: remotes
#> Title: R Package Installation from Remote Repositories, Including
#>        'GitHub'
#> Version: 2.1.0.9000
#> Authors@R: c( person("Jim", "Hester", ,
#>        "james.hester@rstudio.com", role = c("aut", "cre")),
#>        person("Gábor", "Csárdi", , "csardi.gabor@gmail.com", role
#>        = c("aut")), person("Hadley", "Wickham", role = c("aut")),
#>        person("Winston", "Chang", role = "aut"), person("RStudio",
#>        role = "cph"), person("Martin", "Morgan", role = "aut"),
#>        person("Dan", "Tenenbaum", role = "aut"), person("Ken",
#>        "Williams", , "kenahoo@gmail.com", role = "ctb"),
#>        person("Mango Solutions", role = "cph") )
#> Description: Download and install R packages stored in 'GitHub',
#>        'BitBucket', or plain 'subversion' or 'git' repositories.
#>        This package provides the 'install_*' functions in
#>        'devtools'.  Indeed most of the code was copied over from
#>        'devtools'.
#> License: GPL (>= 2)
#> URL: https://github.com/r-lib/remotes#readme
#> BugReports: https://github.com/r-lib/remotes/issues
#> Imports: methods, stats, tools, utils
#> Suggests: brew, callr, codetools, curl, covr, git2r (>= 0.23.0),
#>        knitr, mockery, pkgbuild (>= 1.0.1), pingr, rmarkdown,
#>        rprojroot, testthat, withr
#> Depends: R (>= 3.0.0)
#> VignetteBuilder: knitr
#> RoxygenNote: 6.1.1
#> SystemRequirements: Subversion for install_svn, git for
#>        install_git
#> Encoding: UTF-8
#> Roxygen: list(markdown = TRUE)
#> RemoteType: github
#> RemoteHost: api.github.com
#> RemoteRepo: remotes
#> RemoteUsername: kenahoo
#> RemoteRef: install_version-multi2
#> RemoteSha: 92401a3bdea96803892eb61beedf3f8b18a62331
#> GithubRepo: remotes
#> GithubUsername: kenahoo
#> GithubRef: install_version-multi2
#> GithubSHA1: 92401a3bdea96803892eb61beedf3f8b18a62331
#> NeedsCompilation: no
#> Packaged: 2019-08-22 01:38:30 UTC; rstudio-admin
#> Author: Jim Hester [aut, cre], Gábor Csárdi [aut], Hadley Wickham
#>        [aut], Winston Chang [aut], RStudio [cph], Martin Morgan
#>        [aut], Dan Tenenbaum [aut], Ken Williams [ctb], Mango
#>        Solutions [cph]
#> Maintainer: Jim Hester <james.hester@rstudio.com>
#> Built: R 3.5.3; ; 2019-08-22 01:38:31 UTC; unix
#> 
#> -- File: /home/rstudio-admin/dev-proj/renv/library/R-3.5/x86_64-pc-linux-gnu/remotes/Meta/package.rds

^{Created on 2019-08-22 by the reprex package (v0.3.0)}

[kenahoo]

Right again! Looks like there's one more problem - available.packages() has by default a filter that removes packages that appeared in a previous entry of repos. It's easy for us to turn off that filter, though, we can change

available.packages(contriburl)

to

available.packages(contriburl, filters = c("R_version", "OS_type", "subarch"))

inside the download_version_url() function. This seems to solve the problem for me (finally):

> download_version_url("sass", "0.1.0.9000", repos=getOption('repos'), type='source')
[1] "https://demo.rstudiopm.com/internal/latest/src/contrib/sass_0.1.0.9000.tar.gz"

Not sure how to properly unit test it, I guess the unit test would amount to making sure we're not using the "duplicates" filter for available.packages(), and that when we don't use it, it indeed doesn't screen out duplicates...

Anyway, I pushed the change, does it work now, I hope?

[colearendt]

Nice!! Working perfectly - well done! Much appreciated! It's unclear to me whether removing "duplicates" there could potentially cause problems (i.e. if multiple repos both presented the same version), but I also know very little about this code base! We hit an issue like that in rsconnect recently.

[rikoprogrammer]

Error in download_version_url(package, version, repos, type) :
version '0.12.2' is invalid for package 'admiral'

[rikoprogrammer]

Session info ───────────────────────────────────────────────
setting value
version R version 4.4.2 (2024-10-31 ucrt)
os Windows 11 x64 (build 26100)
system x86_64, mingw32
ui RStudio
language (EN)
collate English_United States.utf8
ctype English_United States.utf8
tz Africa/Nairobi
date 2025-03-15
rstudio 2024.12.0+467 Kousa Dogwood (desktop)
pandoc NA
quarto ERROR: Unknown command "TMPDIR=C:/Users/Admin/AppData/Local/Temp/RtmpmygU4J/file5a6444e235c9". Did you mean command "install"? @ C:\PROGRA1\RStudio\RESOUR1\app\bin\quarto\bin\quarto.exe

─ Packages ───────────────────────────────────────────────────
package * version date (UTC) lib source
admiral * 1.2.0 2025-01-15 [1] CRAN (R 4.4.3)
admiraldev 1.2.0 2025-01-15 [1] CRAN (R 4.4.2)
backports 1.5.0 2024-05-23 [1] CRAN (R 4.4.0)
cachem 1.1.0 2024-05-16 [1] CRAN (R 4.4.2)
cellranger 1.1.0 2016-07-27 [1] CRAN (R 4.4.2)
checkmate 2.3.2 2024-07-29 [1] CRAN (R 4.4.2)
cli 3.6.3 2024-06-21 [1] CRAN (R 4.4.2)
devtools * 2.4.5 2022-10-11 [1] CRAN (R 4.4.3)
digest 0.6.37 2024-08-19 [1] CRAN (R 4.4.2)
dplyr * 1.1.3 2023-09-03 [1] CRAN (R 4.4.2)
ellipsis 0.3.2 2021-04-29 [1] CRAN (R 4.4.2)
fastmap 1.2.0 2024-05-15 [1] CRAN (R 4.4.2)
forcats 1.0.0 2023-01-29 [1] CRAN (R 4.4.2)
fs 1.6.5 2024-10-30 [1] CRAN (R 4.4.2)
generics 0.1.3 2022-07-05 [1] CRAN (R 4.4.2)
glue 1.8.0 2024-09-30 [1] CRAN (R 4.4.2)
haven * 2.5.3 2023-06-30 [1] CRAN (R 4.4.2)
hms 1.1.3 2023-03-21 [1] CRAN (R 4.4.2)
htmltools 0.5.8.1 2024-04-04 [1] CRAN (R 4.4.2)
htmlwidgets 1.6.4 2023-12-06 [1] CRAN (R 4.4.2)
httpuv 1.6.15 2024-03-26 [1] CRAN (R 4.4.2)
later 1.4.1 2024-11-27 [1] CRAN (R 4.4.2)
lifecycle 1.0.4 2023-11-07 [1] CRAN (R 4.4.2)
lubridate * 1.9.4 2024-12-08 [1] CRAN (R 4.4.2)
magrittr 2.0.3 2022-03-30 [1] CRAN (R 4.4.2)
memoise 2.0.1 2021-11-26 [1] CRAN (R 4.4.2)
metacore * 0.1.2 2023-03-02 [1] CRAN (R 4.4.2)
mime 0.12 2021-09-28 [1] CRAN (R 4.4.0)
miniUI 0.1.1.1 2018-05-18 [1] CRAN (R 4.4.3)
pharmaversesdtm * 1.2.0 2025-01-23 [1] CRAN (R 4.4.3)
pillar 1.10.1 2025-01-07 [1] CRAN (R 4.4.2)
pkgbuild 1.4.6 2025-01-16 [1] CRAN (R 4.4.2)
pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.4.2)
pkgload 1.4.0 2024-06-28 [1] CRAN (R 4.4.2)
profvis 0.4.0 2024-09-20 [1] CRAN (R 4.4.3)
promises 1.3.2 2024-11-28 [1] CRAN (R 4.4.2)
purrr 1.0.4 2025-02-05 [1] CRAN (R 4.4.2)
R6 2.6.1 2025-02-15 [1] CRAN (R 4.4.2)
Rcpp 1.0.14 2025-01-12 [1] CRAN (R 4.4.2)
readr 2.1.5 2024-01-10 [1] CRAN (R 4.4.2)
readxl 1.4.3 2023-07-06 [1] CRAN (R 4.4.2)
remotes 2.5.0 2024-03-17 [1] CRAN (R 4.4.2)
rlang 1.1.5 2025-01-17 [1] CRAN (R 4.4.2)
rstudioapi 0.17.1 2024-10-22 [1] CRAN (R 4.4.2)
sessioninfo 1.2.3 2025-02-05 [1] CRAN (R 4.4.3)
shiny 1.10.0 2024-12-14 [1] CRAN (R 4.4.2)
stringi 1.8.4 2024-05-06 [1] CRAN (R 4.4.0)
stringr * 1.5.1 2023-11-14 [1] CRAN (R 4.4.2)
tibble * 3.2.1 2023-03-20 [1] CRAN (R 4.4.2)
tidyr * 1.3.1 2024-01-24 [1] CRAN (R 4.4.2)
tidyselect 1.2.1 2024-03-11 [1] CRAN (R 4.4.2)
timechange 0.3.0 2024-01-18 [1] CRAN (R 4.4.2)
tzdb 0.4.0 2023-05-12 [1] CRAN (R 4.4.2)
urlchecker 1.0.1 2021-11-30 [1] CRAN (R 4.4.3)
usethis * 3.1.0 2024-11-26 [1] CRAN (R 4.4.3)
vctrs 0.6.5 2023-12-01 [1] CRAN (R 4.4.2)
withr 3.0.2 2024-10-28 [1] CRAN (R 4.4.2)
xml2 1.3.6 2023-12-04 [1] CRAN (R 4.4.2)
xportr * 0.4.2 2025-01-07 [1] CRAN (R 4.4.3)
xtable 1.8-4 2019-04-21 [1] CRAN (R 4.4.2)

[1] C:/Users/Admin/AppData/Local/R/win-library/4.4
[2] C:/Program Files/R/R-4.4.2/library

• ── Packages attached to the search path.

[rikoprogrammer]

facing issues with admiral package