Repository navigation
Conversation
get_dataloader's loaders set the dataset's shuffle flag at the start of each pass, so after another model trains with a shuffled loader on the same dataset, the flag stays True. XGBoostModel.fit then warned that an unshuffled loader shuffles. Read the loader's own setting when it has one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PhbzTePoLMBYwjMLG8R9ek
Test CoverageTotal: 67.87% (16694 / 24597 lines)
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
XGBoostModel.fitwarns when its training loader shuffles, because row order changes row subsampling. The check readsloader.dataset.shuffle. Loaders fromget_dataloader(_SeededDataLoader) set that flag on the shared dataset at the start of each pass. So after any shuffled loader on the same dataset has run, for example another model's training loader, the flag staysTrue.fitwithget_dataloader(ds, ..., shuffle=False)then warns that the loader shuffles, which is false.This showed up in the new Tutorial 00, which trains an RNN and XGBoost on the same split. The notebook currently works around it by training XGBoost first.
Fix
Use the loader's own
shuffle_samplessetting when it has one. That is what applies when the loader iterates. Other loaders keep the old check: aRandomSampler, or the dataset's flag.Test
test_shuffle_warning_follows_the_fit_loader: after iterating a shuffled loader on the dataset, fitting with an unshuffled loader logs no warning, and fitting with a shuffled loader still warns. It fails without the fix. All 22 tests intests/core/test_xgboost_model.pypass, ruff is clean, andtools/check_pr_rules.pypasses.🤖 Generated with Claude Code
https://claude.ai/code/session_01PhbzTePoLMBYwjMLG8R9ek