[Backend Tester] Add test name filter #12625
Conversation
|
Stack from ghstack (oldest at bottom): |
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/12625
Note: Links to docs will display an error until the docs builds have been completed. ❗ 1 Active SEVsThere are 1 currently active SEVs. If your PR is affected, please view them below: ⏳ No Failures, 9 PendingAs of commit b35e7b1 with merge base d5a7f33 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
bot
added
the
CLA Signed
| if test_filter.backends is not None and flow.backend not in test_filter.backends: | ||
| return False | ||
|
|
||
| if test_filter.name_regex is not None and not test_filter.name_regex.search( |
There was a problem hiding this comment.
Feels like we are reimplementing features from unittest or pytest :p
There was a problem hiding this comment.
Yeah, I've tried to avoid this, but unfortunately, it doesn't look like the unittest package structure is very extensible in the way I need. There aren't a lot of hooks to control reporting / filtering / discovery without writing custom driver code. I'm open to suggestions, but this seemed to be the lowest friction path with unittest. Switching to pytest might be an option, but I'm hoping that I don't need to do much more non-differentiated work like this.
GregoryComer
added
the
release notes: none
Add a --filter argument to the test runner. Minor refactoring on the discovery logic. You can now run `python -m executorch.backends.test.suite.runner operators --backend xnnpack --filter add`, for example.
Add a --filter argument to the test runner. Minor refactoring on the discovery logic.
You can now run
python -m executorch.backends.test.suite.runner operators --backend xnnpack --filter add, for example.