Fix multiprocessing logging with non-fork start methods - #107
Open
CyberAuth wants to merge 1 commit into
Open
Conversation
|
All contributors have signed the CLA ✍️ ✅ |
Author
|
I have read the CLA Document and I hereby sign the CLA |
bls-cla-bot Bot
added a commit
to blacklanternsecurity/CLA
that referenced
this pull request
Aug 20, 2026
Author
|
recheck |
CyberAuth
marked this pull request as ready for review
August 20, 2026 18:04
singlerider
approved these changes
Aug 20, 2026
Collaborator
There was a problem hiding this comment.
Solid fix with a clear explanation of the root cause. The serialization-safe approach and test coverage across all three start methods are exactly right. Minor nit: parse_file is now a no-op wrapper that could be inlined, and the bare except: blocks in manspider.py could be more specific. Otherwise, approved.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes a Python 3.14 multiprocessing compatibility issue that could prevent MANSPIDER console logging from functioning and could raise a weakref pickling error.
Root cause
The logging queue was created during module import. Under
spawnandforkserver, child processes re-imported the module and created private queues without active listeners, so file logging continued while console records were lost.MANSPIDER also passed its complete controller to spiderling processes. With multiple concurrent targets, the controller process pool contained an already-started
Process. Its process finalizer contains a weak reference and cannot be serialized.Parser subprocesses similarly received a bound
Spiderlingmethod, which pulled unnecessary live runtime state into the serialized object graph.Fix
fork.Testing
fork,forkserver, andspawnRelated work
This is a serialization-safe alternative to #96 and #106. It preserves the default multiprocessing context and the existing process-based worker architecture.
Security / data note
The reproduction and regression tests use synthetic data only and do not require access to external SMB infrastructure.