Add parallel Xlsx writing via pcntl_fork#4834
Conversation
|
I have for some time been actively engaged in eliminating static settings in favor of instance instance properties, to make testing easier and to avoid problems in shared environments. The effort is far from finished, but I'd still like to avoid static when possible. Here, you could make your new switches an instance variable of the writer rather than a global static setting. Would you be able to make that change? |
e6e8be5 to
3ce4ece
Compare
| $inputFileType = 'Xlsx'; | ||
| $inputFileNames = __DIR__ . '/../templates/32readwrite*[0-9].xlsx'; | ||
|
|
||
| /** @var string[] $argv */ |
There was a problem hiding this comment.
Why is this change needed?
There was a problem hiding this comment.
You're right, this is unrelated to the parallel processing changes — it was a stray static analysis suppression. I'll remove it.
|
parallelEnabled and maxWorkers are instance properties on the Xlsx writer, not static settings. The writer is configured per-instance via setParallelEnabled() and setMaxWorkers(). |
|
@kemo You can use |
Summary
src/PhpSpreadsheet/Parallel/) usingpcntl_fork()with automatic sequential fallbackWriter\Xlsx::save()— the single most CPU-intensive phase of writingArchitecture
ParallelExecutormap(tasks, worker)with auto-backend detection, memory safety checksPcntlBackendpcntl_fork()+ temp-file IPC, batched execution, timeout + zombie reapingSequentialBackendCpuDetectorWriter instance API
Parallel settings live on the
Writer\Xlsxinstance (not global statics):Worker count formula
min(cpuCount - 1, taskCount, 8)— reserves 1 core for parent, caps at 8 to prevent IPC overhead.Safety
$isChildflag prevents children from running parent cleanup (could delete unread temp files)pcntl_waitpid()infinallyblock reaps all children, even on exceptionsfunction_exists()check, numeric signal value (15) avoids undefined constantBenchmarks (Apple M2 Pro, 11 cores)
Speedup is bounded by ZIP compression (~45% of
save()time) which runs sequentially after XML generation (~48%).Test plan
@codeCoverageIgnoreParallelXlsxWriterTest: validates round-trip read-back of parallel-written filesParallelXlsxWriterTest: sequential vs parallel produce identical cell dataParallelXlsxWriterTest: single-sheet parallel-enabled falls back to sequential-1,M,Gsuffix formats