Skip to content

fix(app): warm up model at startup to avoid first-request connection errors - #392

Open
ousamabenyounes wants to merge 1 commit into
OpenBMB:mainfrom
ousamabenyounes:fix/issue-367
Open

fix(app): warm up model at startup to avoid first-request connection errors#392
ousamabenyounes wants to merge 1 commit into
OpenBMB:mainfrom
ousamabenyounes:fix/issue-367

Conversation

@ousamabenyounes

Copy link
Copy Markdown

Load the VoxCPM model during server startup in run_demo() instead of lazily on the first request. The multi-GB from_pretrained ran inside the first click, blocking that request for minutes and surfacing any load/download failure as an opaque connection error (issue #367). Warming up before launch fails fast and keeps the first generation responsive.

Test verification (RED → GREEN)

With the fix reverted, the new test fails (RED):

platform linux -- Python 3.14.4, pytest-9.0.2, pluggy-1.6.0 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /home/ousama/contribute-work/OpenBMB__VoxCPM
configfile: pyproject.toml
plugins: typeguard-4.4.4
collecting ... collected 1 item

tests/test_app_warmup.py::test_model_is_warmed_up_before_launch call order: ['launch']
FAILED

=================================== FAILURES ===================================
____________________ test_model_is_warmed_up_before_launch _____________________

    def test_model_is_warmed_up_before_launch():
        app = _load_app_module()
    
        calls: list[str] = []
    
        class _FakeInterface:
            def queue(self, *args, **kwargs):
                return self
    
            def launch(self, *args, **kwargs):
                calls.append("launch")
    
        with mock.patch.object(app, "create_demo_interface", return_value=_FakeInterface()), \
             mock.patch.object(app.VoxCPMDemo, "get_or_load_voxcpm",
                               autospec=True,
                               side_effect=lambda self: calls.append("warmup")):
            app.run_demo(device="cpu")
    
        print(f"call order: {calls}")
>       assert "warmup" in calls, "model was never warmed up (loads lazily on first request)"
E       AssertionError: model was never warmed up (loads lazily on first request)
E       assert 'warmup' in ['launch']

tests/test_app_warmup.py:82: AssertionError
=========================== short test summary info ============================
FAILED tests/test_app_warmup.py::test_model_is_warmed_up_before_launch - Asse...
============================== 1 failed in 0.07s ===============================

With the fix applied, the test passes (GREEN):

GREEN attempt 1/2 (exit 0)
============================= test session starts ==============================
platform linux -- Python 3.14.4, pytest-9.0.2, pluggy-1.6.0 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /home/ousama/contribute-work/OpenBMB__VoxCPM
configfile: pyproject.toml
plugins: typeguard-4.4.4
collecting ... collected 1 item

tests/test_app_warmup.py::test_model_is_warmed_up_before_launch call order: ['warmup', 'launch']
PASSED

============================== 1 passed in 0.06s ===============================

Full local suite

Command: python3 -m pytest tests/ -v

============================= test session starts ==============================
platform linux -- Python 3.14.4, pytest-9.0.2, pluggy-1.6.0 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /home/ousama/contribute-work/OpenBMB__VoxCPM
configfile: pyproject.toml
plugins: typeguard-4.4.4
collecting ... collected 46 items / 2 errors

==================================== ERRORS ====================================
____________ ERROR collecting tests/test_lora_checkpoint_loading.py ____________
ImportError while importing test module '/home/ousama/contribute-work/OpenBMB__VoxCPM/tests/test_lora_checkpoint_loading.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.14/importlib/__init__.py:88: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tests/test_lora_checkpoint_loading.py:9: in <module>
    import torch
E   ModuleNotFoundError: No module named 'torch'
__________________ ERROR collecting tests/test_model_utils.py __________________
ImportError while importing test module '/home/ousama/contribute-work/OpenBMB__VoxCPM/tests/test_model_utils.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.14/importlib/__init__.py:88: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tests/test_model_utils.py:20: in <module>
    spec.loader.exec_module(utils)
src/voxcpm/model/utils.py:3: in <module>
    import torch
E   ModuleNotFoundError: No module named 'torch'
=========================== short test summary info ============================
ERROR tests/test_lora_checkpoint_loading.py
ERROR tests/test_model_utils.py
!!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection !!!!!!!!!!!!!!!!!!!!
============================== 2 errors in 0.17s ===============================

Fix #367

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

连接错误

1 participant