Test Suite Failures and Warnings

[clnsmth]

Issue:

Running the ecocomDP test suite results in one critical error and numerous warnings, indicating potential issues with data validation, file handling, and dependency updates. This issue documents these problems and proposes solutions to improve package robustness.
Test Summary:

• FAIL: 1
• WARN: 11
• SKIP: 1
• PASS: 781

Critical Error: test_convert_to_dwca.R

The test suite fails with an error in test_convert_to_dwca.R when attempting to load data from a URL.

Error Message:

Error (test_convert_to_dwca.R:15:3): Creates tables, meta, and valid EML
Error in `load(url(objurl))`: the input does not start with a magic number compatible with loading from a connection

Analysis:
This error suggests that the object at objurl is not a valid R data file. The load() function is likely receiving an HTML error page (e.g., 404 Not Found) or another non-R format, which it cannot parse.
Recommendation:

• Validate URL Content: Before calling load(), verify the URL is accessible and the content type is appropriate for an R object.
• Implement Robust Error Handling: Wrap the connection attempt in a tryCatch() block to manage failures gracefully and provide a more informative error message.

Warnings

1. test_create_eml.R: Temporary File Conflict

Warning Message:

Warning (test_create_eml.R:12:3): Creates valid EML
'/var/folders/nq/lyx69z3d0z511bp5rwgq0yww0000gn/T//RtmpCp08t0/data' already exists

Analysis:
The test creates a temporary directory that already exists, likely from a previous, incomplete test run.
Recommendation:

• Ensure Cleanup: Use a teardown() expression or an on.exit() call within the test to ensure temporary files and directories are removed after the test completes.

2. test_flatten_data.R: Deprecated tidyselect Syntax

Warning Message:

Warning (test_flatten_data.R:20:5): Column presence
Using an external vector in selections was deprecated in tidyselect 1.1.0.
ℹ Please use `all_of()` or `any_of()` instead.
# Was:
data %>% select(last_col)

Analysis:
The code uses an outdated tidyselect pattern that will be removed in future versions.
Recommendation:

• Update Syntax: Replace select(last_col) with select(all_of(last_col)) to align with current tidyselect best practices.

3. test_read_data.R: Data Validation Issues

Warning Messages:
Multiple warnings indicate validation failures for several datasets (edi.193.5, edi.193.3, edi.262.1, edi.359.1) when reading from both .rds and .csv files.
Warning (test_read_data.R:73:3): Reads from 1 local .rds
Validation issues found for edi.193.5
...and others

Analysis:
The validate_data() function is flagging inconsistencies in the test datasets. These warnings clutter the test output and may hide more severe issues.
Recommendation:

• Inspect Test Data: Review the specified datasets to identify and correct the validation inconsistencies.
• Expect Warnings in Tests: If the validation failures are intentional for testing purposes, wrap the function calls in testthat::expect_warning() to explicitly acknowledge them and prevent them from appearing as unhandled warnings in the test results.

[clnsmth]

Update:

I've confirmed that the root cause of the Error in load(url(objurl)) is an HTTP 429 Too Many Requests error from the API. This is triggered by the rapid succession of API calls during the test suite execution.

Solution:

1. API Maintainer Consultation: contact the API maintainer to clarify their rate-limiting policy.

2. Client-Side Robustness: Implement a retry mechanism with exponential backoff within the package to automatically handle 429 errors. This will resolve the immediate test failure and make the package more resilient for users. Evaluate httr2 for this implementation due to its built-in retry functionality.