Playground CLI: Use Blueprints v2 #2281
Conversation
adamziel
added
[Type] Enhancement
…ify functions. Lint. Fix process manager unit tests
| hidden: true, | ||
| }) | ||
| // Blueprint CLI options | ||
| .option('mode', { |
There was a problem hiding this comment.
Could we make the name more clear?
Something like blueprint-mode?
There was a problem hiding this comment.
Potentially. This option name is copied frmo the blueprints.php runner but maybe a different name would be more useful here. Or in both places? I'm now thinking --site-type=new|existing|none
There was a problem hiding this comment.
It's good that --site-type=new|existing|none is more explicit.
Instead of mode to declaring an execution personality, site-type declares something about the site itself and lets Playground CLI judge what behavior is appropriate.
There was a problem hiding this comment.
Good call! I'll address this in a follow-up PR as it will require an accompanying change in the Blueprints library
|
@adamziel I explored using Playground CLI snapshots in tests and found a few issues with the CLI that might be relevant.
|
adamziel
force-pushed
the
use-blueprints-v2-runner-in-cli
branch
from
f83ca60 to
a44534d
Compare
|
I've tested it on 110 Blueprints from the Blueprints and it worked. I'd like to merge this PR soon, perhaps even today (Wednesday), and start iterating on improvements. CC @fellyph – we'll need to adjust the Playground CLI README again and communicate this change on make.wp.org/playground, ideally with a bunch of context on Blueprints v2. |
…ill the top-level process when the primary worker crashes
|
The integration glue is ready for merging. However, I found 5 Blueprints where Playground is crashing in different ways. This PR may need to wait for a few more platform improvements around: |
## Description This PR adds supports for reading from non-blocking streams (`stream_set_blocking($fp, true)`) by aligning the streaming logic with unix kernel: * `stream_set_blocking()` is now able to correctly mark the OS stream as non blocking via `fcntl()` * `fd_read()` either blocks or immediately returns based on stream type * `js_open_process()`, `proc_open()` et al. now work directly with kernel pipes. There's no more special event-based handling, polling, or special casing for pumping stdin data to the process. * PIPEFS no longer returns EWOULDBLOCK when the pipe is blocking, lacks data, and the other side is already closed * Removed special cases for polling processes, including `PHPWASM.child_proc_by_fd`. We rely on kernel pipes now. * Removed PHP 7.4 patch to treat blocking streams in a special way ### Motivation for this patch With this PR, we can use the Symfony Process component. It spawns a new process via proc_open(), marks its output stream as non-blocking, and immediately reads the initial output bytes if any are present. Before this change, Playground would block until the first output is produced, and, sometimes until the entire process finishes. This PR is a pre-requisite to #2281 ### Other changes This PR implements `usleep()` (via `-Wl,--wrap=usleep`). It turns out that it wasn't actually blocking the execution and a few unit tests relying on it only passed due to a buggy implementation of blocking streams. ### Follow-up work * Replace `read()` globally with something equivalent to `wasm_read()`. Right now we only do this `RUN /root/replace.sh 's/ret = read/ret = wasm_read/g' /root/php-src/main/streams/plain_wrapper.c` * Consider not overriding `php_pollfd_for` and removing `wasm_poll_socket()` – wrapping select/poll/read syscalls may be enough * We could remove more special casing, including socket polling inside `wasm_poll_socket`, the `js_popen_to_file` function, `awaitData` et al. ## Testing Instructions (or ideally a Blueprint) Tests are included so just confirm the CI is green.
Extracts `sandboxedSpawnHandlerFactory()` as a separate export from `@php-wasm/universal`. Every module can reuse it now instead of recreating new spawn handlers. Pre-requisite for #2281 ## Testing instructions Confirm the CI tests pass
## Motivation for the change, related issues Simplifies the auto mounting logic and adds explicit semantics for creating additional Blueprint steps as a result of an auto mount, e.g. installing and activating a plugin that was automatically detected. Also, adds unit tests for the automounting detection. A part of #2281 ## Testing Instructions (or ideally a Blueprint) Confirm the tests are still green
## Motivation for the change, related issues Moves Blueprints v2 `.phar` file and the TypeScript module into the `@wp-playground/cli` package and brushes up its logic. This PR only moves dead code around. However, it also changes the dependency graph of the project and affects the computed package.json files, as in `@wp-playground/cli` now depends on `fs-ext`. For that reason, there's also a change in the build machinery. Also, `v2.spec.ts` uncovered a piping issue where polling after closing the pipe caused a fatal error. Therefore, this PR adds an FS.isClosed() check. A part of #2281 ## Testing plan CI
Motivation for the change, related issues
Changes the Blueprint runner in Playground CLI from the TypeScript v1 runner to the PHP Blueprints v2 runner.
Implementation details
This PR ships the blueprints.phar runner from the php-toolkit repo and a TypeScript function
runBlueprintV2()that runs the PHP runner with the specified options.CLI flags passthrough
The data flow is:
flowchart LR A(["bun cli.ts"]) A --> B("runCLI()") B --> E("bootRequestHandler()") E --> C("runBlueprintV2()") C --> D("startServer()")runCLI()function that can be used by any frontend – CLI script, Studio, etc.CLI options
wp-playgroundCLIwp-playground <command> [options]Commands
server– start the local serverrun-blueprint– execute a Blueprintbuild-snapshot– build a ZIP snapshotGeneral
--outfile <path>– output file (defaultwordpress.zip)--port <number>– server port (default9400)--php <version>– PHP runtime (overrides Blueprint)--blueprint <path|url>– Blueprint to run--quiet– silence logs--debug– dump PHP errors on boot failure--auto-mount– detect and mount the current working directoryMounting
--mount <host:vfs>– mount after WP install (repeatable)--mount-before-install <host:vfs>– mount before WP install (repeatable)--mount-dir "<host>" "<vfs>"– two-arg mount after WP install (repeatable)--mount-dir-before-install "<host>" "<vfs>"– two-arg mount before WP install (repeatable)Blueprint execution
--mode <create-new-site|apply-to-existing-site|mount-only>– execution mode (defaultcreate-new-site)--db-engine <sqlite|mysql>– database engine (defaultsqlite)--db-host <host>– MySQL host--db-user <user>– MySQL user--db-pass <pass>– MySQL password--db-name <name>– MySQL database--db-path <file>– SQLite file path--truncate-new-site-directory– wipe target directory first before creating a new site. Useful to avoid a "directory must be empty" error--allow <bundled-files,follow-symlinks>– extra permissions to access the local filesystemExperimental
--experimental-multi-worker [N]– enable multi-worker; omitNto use CPU-1--experimental-trace– verbose runtime traceHidden
--wp <version>– WordPress core version override (defaultlatest)--login– auto-login userCombine flags as needed; repeatable ones accept multiple instances.
Simplified logic
The logic handled by the v2 runner was removed from the Playground CLI:
The CLI is now, pretty much, just a thin glue that initializes all the components (PHP, Express.js server, v2 runner) and
Weak spots
--mountand you have to know to mount a local directory as/wordpressbefore the WP installation to start a site in your host filesystem. Maybe that's fine and CLI is just a low-level tool to be wrapped by something like wp-now. But maybe we can find a more convenient usage pattern without sacrificing configurability.Backwards Compatibility Break
@wp-playground/clipackage.features,phpExtensionBundleslandingPage@wp-playground/clicommands will not work with the new version. I wonder if we could continue supporting the "legacy" commands 🤔Follow-up work
blueprints.pharrelease used by Playground to ~100KB (it's absolutely possible, e.g. by using JavaScripts'snew URL()instead of shipping an entire URL parser).data-liberation.pharimporter separately from theblueprints.pharrelease.php-toolkitas a subtree or a submodule within this repo.Testing Instructions (or ideally a Blueprint)
A simple way to try it out is with the following command:
I'll provide a more thorough test plan once we merge #2231 and #2248.