Fix subprocess profiling with Python interpreter flags (#863)#1013
Fix subprocess profiling with Python interpreter flags (#863)#1013emeryberger merged 2 commits intomasterfrom
Conversation
When pytest-xdist (or similar tools) spawn workers with "python -u ...",
Scalene's redirect_python intercepts and re-invokes as
"python -m scalene run ... --- -u -c ...". After parsing its own args,
sys.argv becomes ["-u", "-c", "..."], but the existing code only checked
sys.argv[0] == "-c", causing "-u" to be misinterpreted as a filename
("could not find input file .../-u").
Strip Python interpreter flags (-u, -B, -O, etc.) before looking for
-c/-m/script arguments, applying their effects where possible (-u sets
unbuffered I/O, -B disables .pyc writing, -v enables verbose imports).
Fixes #863
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| try: | ||
| sys.stdout.reconfigure(line_buffering=False) # type: ignore[union-attr] | ||
| sys.stderr.reconfigure(line_buffering=False) # type: ignore[union-attr] | ||
| except Exception: |
Check notice
Code scanning / CodeQL
Empty except Note
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 7 days ago
In general, empty except blocks should either (1) be removed, (2) be replaced with handling of specific, expected exception types, or (3) at least log or comment why it is safe to ignore the exception. Here, the code is doing a best‑effort call to reconfigure on sys.stdout and sys.stderr. The safest change without altering behavior is to either (a) add a short explanatory comment in the except body and keep it empty, or (b) log a low‑priority message when reconfiguration fails. Since Scalene already imports warnings and writes error messages to sys.stderr in other places, the best targeted fix is to emit a warning when reconfiguration fails; this retains robustness while still surfacing unexpected problems.
Concretely, in scalene/scalene_profiler.py, around lines 1672–1676, replace the bare except Exception: pass with an except Exception as exc: block that calls warnings.warn with a concise message explaining that unbuffered mode could not be applied, including the exception text. This keeps the program running while making the failure visible, and satisfies the static analysis rule by ensuring the except clause does something meaningful. No new imports are needed since warnings is already imported at line 56.
| @@ -1672,8 +1672,11 @@ | ||
| try: | ||
| sys.stdout.reconfigure(line_buffering=False) # type: ignore[union-attr] | ||
| sys.stderr.reconfigure(line_buffering=False) # type: ignore[union-attr] | ||
| except Exception: | ||
| pass | ||
| except Exception as exc: | ||
| warnings.warn( | ||
| f"Scalene: failed to reconfigure standard streams for unbuffered mode: {exc!r}", | ||
| RuntimeWarning, | ||
| ) | ||
| elif flag == "-B": | ||
| sys.dont_write_bytecode = True | ||
| elif flag == "-v": |
Addresses Copilot code scanning alert about the bare except/pass on the stream reconfigure call. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 participant
Summary
scalene --profile-allfails when pytest-xdist spawns workers withpython -u ...redirect_pythonintercepts subprocess calls, Python interpreter flags like-uend up insys.argvbefore-c/-m, causing Scalene to misinterpret them as filenames-u,-B,-O,-W,-X, etc.) before looking for-c/-m/script, applying their effects where possible (unbuffered I/O, .pyc suppression, verbose imports)Test plan
scalene run --profile-all --- -m pytest -n 3🤖 Generated with Claude Code