Upgrade pytest to 9.0.0, remove pytest-subtests, and migrate to new TOML config #3439

Copilot · 2025-11-11T01:11:41Z

Pytest 9.0.0 introduced native support for unittest's self.subTest() and stricter serialization requirements for subTest parameters when using pytest-xdist parallel execution. Non-serializable objects (mocks, datetime instances, exceptions) passed to subTest() now cause DumpError failures.

Changes

Upgraded pytest: 8.4.2 → 9.0.0 in pyproject.toml
Removed pytest-subtests: Removed pytest-subtests==0.14.1 from dev dependencies (pytest 9.0 has native support)
Migrated to new TOML config format: Changed from [tool.pytest.ini_options] to [tool.pytest] with list-based addopts
Fixed subTest serialization: Replaced non-serializable parameters with simple identifiers across 20+ test files

Pattern Applied

Before:

for case in test_cases:
    with self.subTest(error=case["error"], channel=MockTextChannel()):
        # test logic

After:

for i, case in enumerate(test_cases):
    with self.subTest(test_case=i, has_channel=True):
        # test logic

Configuration Migration

Before:

[tool.pytest.ini_options]
addopts = "-p no:nose"

After:

[tool.pytest]
addopts = ["-p", "no:nose"]

Additional Fixes

Fixed HTTPException initialization in test_utils.py (requires message argument)

All 442 tests pass with xdist parallel execution.

Original prompt

Update pytest to 9.0.0
Migrate to using TOML configuration for pytest: https://docs.pytest.org/en/stable/reference/customize.html#config-file-formats
Additionally, fix the tests so these DumpErrors related to our mocks do not occur:

[gw1] linux -- Python 3.13.9 /home/runner/work/bot/bot/.venv/bin/python3

self = <tests.bot.exts.backend.test_error_handler.IndividualErrorHandlerTests testMethod=test_handle_api_error>
log_mock = <MagicMock name='log' id='140094977313488'>

    @patch("bot.exts.backend.error_handler.log")
    async def test_handle_api_error(self, log_mock):
        """Should `ctx.send` on HTTP error codes, and log at correct level."""
        test_cases = (
            {
                "error": ResponseCodeError(AsyncMock(status=400)),
                "log_level": "error"
            },
            {
                "error": ResponseCodeError(AsyncMock(status=404)),
                "log_level": "debug"
            },
            {
                "error": ResponseCodeError(AsyncMock(status=550)),
                "log_level": "warning"
            },
            {
                "error": ResponseCodeError(AsyncMock(status=1000)),
                "log_level": "warning"
            }
        )
    
        for case in test_cases:
>           with self.subTest(error=case["error"], log_level=case["log_level"]):
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

tests/bot/exts/backend/test_error_handler.py:486: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../_temp/uv-python-dir/cpython-3.13.9-linux-x86_64-gnu/lib/python3.13/contextlib.py:148: in __exit__
    next(self.gen)
../../_temp/uv-python-dir/cpython-3.13.9-linux-x86_64-gnu/lib/python3.13/contextlib.py:148: in __exit__
    next(self.gen)
.venv/lib/python3.13/site-packages/_pytest/unittest.py:438: in addSubTest
    self.ihook.pytest_runtest_logreport(report=sub_report)
.venv/lib/python3.13/site-packages/pluggy/_hooks.py:512: in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/pluggy/_manager.py:120: in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/xdist/remote.py:255: in pytest_runtest_logreport
    self.sendevent("testreport", data=data)
.venv/lib/python3.13/site-packages/xdist/remote.py:98: in sendevent
    self.channel.send((name, kwargs))
.venv/lib/python3.13/site-packages/execnet/gateway_base.py:912: in send
    self.gateway._send(Message.CHANNEL_DATA, self.id, dumps_internal(item))
                                                      ^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/execnet/gateway_base.py:1629: in dumps_internal
    return _Serializer().save(obj)  # type: ignore[return-value]
           ^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/execnet/gateway_base.py:1647: in save
    self._save(obj)
.venv/lib/python3.13/site-packages/execnet/gateway_base.py:1667: in _save
    dispatch(self, obj)
.venv/lib/python3.13/site-packages/execnet/gateway_base.py:1744: in save_tuple
    self._save(item)
.venv/lib/python3.13/site-packages/execnet/gateway_base.py:1667: in _save
    dispatch(self, obj)
.venv/lib/python3.13/site-packages/execnet/gateway_base.py:1740: in save_dict
    self._write_setitem(key, value)
.venv/lib/python3.13/site-packages/execnet/gateway_base.py:1734: in _write_setitem
    self._save(value)
.venv/lib/python3.13/site-packages/execnet/gateway_base.py:1667: in _save
    dispatch(self, obj)
.venv/lib/python3.13/site-packages/execnet/gateway_base.py:1740: in save_dict
    self._write_setitem(key, value)
.venv/lib/python3.13/site-packages/execnet/gateway_base.py:1734: in _write_setitem
    self._save(value)
.venv/lib/python3.13/site-packages/execnet/gateway_base.py:1667: in _save
    dispatch(self, obj)
.venv/lib/python3.13/site-packages/execnet/gateway_base.py:1740: in save_dict
    self._write_setitem(key, value)
.venv/lib/python3.13/site-packages/execnet/gateway_base.py:1734: in _write_setitem
    self._save(value)
.venv/lib/python3.13/site-packages/execnet/gateway_base.py:1667: in _save
    dispatch(self, obj)
.venv/lib/python3.13/site-packages/execnet/gateway_base.py:1740: in save_dict
    self._write_setitem(key, value)
.venv/lib/python3.13/site-packages/execnet/gateway_base.py:1734: in _write_setitem
    self._save(value)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <execnet.gateway_base._Serializer object at 0x7f6a67591eb0>
obj = ResponseCodeError(<AsyncMock id='140094976712656'>)

    def _save(self, obj: object) -> None:
        tp = type(obj)
        try:
            dispatch = self._dispatch[tp]
        except KeyError:
            methodname = "save_" + tp.__name__
 ...

</details>



<!-- START COPILOT CODING AGENT TIPS -->
---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more [Copilot coding agent tips](https://gh.io/copilot-coding-agent-tips) in the docs.

jb3 · 2025-11-11T01:49:43Z

@copilot - pytest 9.0 supports subtests natively now, and these should be used (and the pytest-subtests removed from the pyproject and project), they are documented here: https://docs.pytest.org/en/stable/how-to/subtests.html#subtests

The directive about moving to TOML specifically means the new non-INI method: https://docs.pytest.org/en/stable/changelog.html#new-features

Copilot · 2025-11-11T01:54:14Z

@copilot - pytest 9.0 supports subtests natively now, and these should be used (and the pytest-subtests removed from the pyproject and project), they are documented here: https://docs.pytest.org/en/st...

Removed pytest-subtests plugin and migrated to the new TOML configuration format in commit e6029ef.

Changes:

Removed pytest-subtests==0.14.1 from dev dependencies (pytest 9.0 has native support for unittest's self.subTest())
Changed [tool.pytest.ini_options] to [tool.pytest]
Updated addopts from string to list format: ["-p", "no:nose"]

All 442 tests passing with the new configuration.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

publicsuffix.org
- Triggering command: /home/REDACTED/work/bot/bot/.venv/bin/python -u -c import sys;exec(eval(sys.stdin.readline())) (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Prevents DumpError when test parameters are sent between test executors but cannot be serialised (primarily a problem for out mock objects).

wookie184

not reviewed fully, this seems pretty meh.

wookie184 · 2025-11-11T18:38:37Z

pyproject.toml

+[tool.pytest]
 # We don't use nose style tests so disable them in pytest.
 # This stops pytest from running functions named `setup` in test files.
 # See https://github.com/python-discord/bot/pull/2229#issuecomment-1204436420
-addopts = "-p no:nose"
+addopts = ["-p", "no:nose"]


Nose support was dropped in pytest 8.0.0 so this should no longer be necessary https://docs.pytest.org/en/stable/deprecations.html#support-for-tests-written-for-nose

wookie184 · 2025-11-11T19:14:04Z

tests/bot/exts/moderation/test_silence.py

+        for i, ((duration, message, was_silenced), target) in enumerate(itertools.product(test_cases, targets)):
            with (
                mock.patch.object(self.cog, "_set_silence_overwrites", return_value=was_silenced),
-                self.subTest(was_silenced=was_silenced, target=target, message=message),
+                self.subTest(test_case=i, was_silenced=was_silenced, has_target=target is not None),
                mock.patch.object(self.cog, "send_message") as send_message
            ):


Adding a test case ID seems silly here, as it's coming from an itertools.product and it would be a pain to work out the inputs that it's referring to.

Also the has_targets is not a replacement for the target because there are multiple not none targets.

I'd accept that the message can be removed though, since it's an expected output.

wookie184 · 2025-11-11T19:16:03Z

tests/bot/exts/moderation/test_silence.py

+        for i, channel in enumerate((MockVoiceChannel(), MockTextChannel())):
+            with self.subTest(test_case=i):


I dunno about just wrapping everything in enumerate in general, I feel like e.g. str(type(channel)) would be a more descriptive name for a test case.

wookie184 · 2025-11-11T19:20:04Z

tests/bot/exts/moderation/test_silence.py

+        for i, overwrite_json in enumerate(('{"send_messages": true, "add_reactions": null}', None)):
+            with self.subTest(test_case=i, has_overwrite=overwrite_json is not None):


surely this change just isn't necessary?

jb3 · 2025-11-11T20:17:39Z

Will go through and have a proper look case-by-case, I agree there are probably ways in a lot of them where we can better serialize instead of using enumerate.

Copilot AI assigned Copilot and jb3 Nov 11, 2025

Copilot started work on behalf of jb3 November 11, 2025 01:12 View session

Copilot AI changed the title ~~[WIP] Update pytest to 9.0.0 and migrate to TOML configuration~~ Upgrade pytest to 9.0.0 and fix subTest serialization errors with xdist Nov 11, 2025

Copilot AI requested a review from jb3 November 11, 2025 01:48

Copilot finished work on behalf of jb3 November 11, 2025 01:48

Copilot started work on behalf of jb3 November 11, 2025 01:50 View session

Copilot AI changed the title ~~Upgrade pytest to 9.0.0 and fix subTest serialization errors with xdist~~ Upgrade pytest to 9.0.0, remove pytest-subtests, and migrate to new TOML config Nov 11, 2025

Copilot finished work on behalf of jb3 November 11, 2025 01:55

jb3 added 3 commits November 11, 2025 02:04

Update pytest to 9.0.0 and remove pytest-subtests

bf32749

Move from tool.pytest.ini_options to tool.pytest in pyproject.toml

cf5d49d

Ensure subtest parameters are always serializable for pytest-xdist

e4ac857

Prevents DumpError when test parameters are sent between test executors but cannot be serialised (primarily a problem for out mock objects).

jb3 force-pushed the copilot/update-pytest-and-fix-tests branch from e6029ef to e4ac857 Compare November 11, 2025 02:05

jb3 marked this pull request as ready for review November 11, 2025 02:07

jb3 requested a review from MarkKoz as a code owner November 11, 2025 02:07

python-discord-policy-bot bot requested a review from a team November 11, 2025 02:07

wookie184 requested changes Nov 11, 2025

View reviewed changes

jb3 marked this pull request as draft November 11, 2025 21:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Upgrade pytest to 9.0.0, remove pytest-subtests, and migrate to new TOML config #3439

Upgrade pytest to 9.0.0, remove pytest-subtests, and migrate to new TOML config #3439

Copilot AI commented Nov 11, 2025 •

edited

Loading

Uh oh!

jb3 commented Nov 11, 2025

Uh oh!

Copilot AI commented Nov 11, 2025 •

edited

Loading

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

wookie184 left a comment

Uh oh!

wookie184 Nov 11, 2025

Uh oh!

wookie184 Nov 11, 2025

Uh oh!

wookie184 Nov 11, 2025

Uh oh!

wookie184 Nov 11, 2025

Uh oh!

jb3 commented Nov 11, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		for i, channel in enumerate((MockVoiceChannel(), MockTextChannel())):
		with self.subTest(test_case=i):

		for i, overwrite_json in enumerate(('{"send_messages": true, "add_reactions": null}', None)):
		with self.subTest(test_case=i, has_overwrite=overwrite_json is not None):

Uh oh!

Upgrade pytest to 9.0.0, remove pytest-subtests, and migrate to new TOML config #3439

Are you sure you want to change the base?

Upgrade pytest to 9.0.0, remove pytest-subtests, and migrate to new TOML config #3439

Conversation

Copilot AI commented Nov 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Pattern Applied

Configuration Migration

Additional Fixes

Uh oh!

jb3 commented Nov 11, 2025

Uh oh!

Copilot AI commented Nov 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

wookie184 left a comment

Choose a reason for hiding this comment

Uh oh!

wookie184 Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

wookie184 Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

wookie184 Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

wookie184 Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

jb3 commented Nov 11, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Nov 11, 2025 •

edited

Loading

Copilot AI commented Nov 11, 2025 •

edited

Loading