wm_records_name() & wm_name2id_() produces "Error: (400) Bad Request" on some systems when there are spaces in query

[zachary-foster]

While debugging an issue with taxize, I found that wm_records_name fails on some systems (a clean Windows install in this case). It might have something to do with how spaces in the URL call to the Worrms API are formatted, although I am not sure. Here is a reprex I was using to debug the error:

library(worrms)
wm_records_name("Gadus morhua")
#> Error: (400) Bad Request - Gadus morhua

^{Created on 2022-03-29 by the reprex package (v2.0.1)}

The error happens in the internal function wm_GET. It produces this URL:

https://www.marinespecies.org/rest/AphiaRecordsByName/Gadus morhua?like=true&marine_only=true&offset=1

Which results in a "(400) Bad Request" error when called with crul::HttpClient, but works when pasting into an internet browser (Firefox in my case). The browser replaces the space in "Gadus morhua" with "%20" automatically, making the URL:

https://www.marinespecies.org/rest/AphiaRecordsByName/Gadus%20morhua?like=true&marine_only=true&offset=1

This error did not occur on my Linux system (PopOS).

Session Info

R version 4.1.3 (2022-03-10)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19042)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252    LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                           LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] worrms_0.4.2

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.8.3      fansi_1.0.2       utf8_1.2.2        crayon_1.5.0      crul_1.2.0        R6_2.5.1          jsonlite_1.8.0    lifecycle_1.0.1  
 [9] magrittr_2.0.2    pillar_1.7.0      rlang_1.0.2       cli_3.2.0         curl_4.3.2        data.table_1.14.2 vctrs_0.3.8       ellipsis_0.3.2   
[17] urltools_1.7.3    tools_4.1.3       glue_1.6.2        triebeard_0.3.0   compiler_4.1.3    pkgconfig_2.0.3   tibble_3.1.6      httpcode_0.3.0  

[zachary-foster]

I just noticed wm_records_name("Gadus+morhua") does work, so it is probably an issue with how the space is being encoded. It might be an issue with crul rather than worrms actually.

[salvafern]

Hi @zachary-foster, I have been struggling with this same issue in ropensci-archive/mregions#60 It seems indeed a problem feature of httr/crul (see r-lib/httr#335) as you may have noticed.

I don't think this will change anytime soon, so we rather find workarounds for our packages

[sckott]

I think changing worrms internals to use curl::curl_escape() is needed just to make sure that encoding of spaces happens

[zachary-foster]

Thanks for the insights @salvafern and @sckott!

[reblake]

I think I'm having a similar problem with the worrms function wm_records_taxamatch(), which is called by the obistools function match_taxa(). I get

> wm_records_taxamatch("Gadus morhua")
Error: (400) Bad Request - AphiaRecordsByMatchNames

I tried replacing spaces in the scientificName column with + as mentioned above, but that didn't work for me.

> wm_records_taxamatch("Gadus+morhua")
Error: (400) Bad Request - AphiaRecordsByMatchNames

─ Session info ───────────────────────────────────────────────────────────────────
 setting  value
 version  R version 4.3.2 (2023-10-31 ucrt)
 os       Windows 10 x64 (build 19045)
 system   x86_64, mingw32
 ui       RStudio
 language (EN)
 collate  English_United States.utf8
 ctype    English_United States.utf8
 tz       America/New_York
 date     2023-12-21
 rstudio  2023.12.0+369 Ocean Storm (desktop)
 pandoc   3.1.1 @ C:/Program Files/RStudio/resources/app/bin/quarto/bin/tools/ (via rmarkdown)
─ Packages ─────────────
obistools         * 0.0.11  2023-12-18 [1] Github (iobis/obistools@a7e0661) 
worrms            * 0.4.3   2023-06-20 [1] CRAN (R 4.2.3)

[bart-v]

@reblake this was a temp. issue, indeed fixed now, unrelated to this "space" issue