Skip to content

[Playground CLI] explore why plugin files don't exist after mounting #2347

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 43 commits into
base: trunk
Choose a base branch
from

Conversation

bgrgicak
Copy link
Collaborator

@bgrgicak bgrgicak commented Jul 10, 2025

⚠️ This PR is an exploration and shouldn't be merged.

Motivation for the change, related issues

We got a report that WooCommerce "randomly" deactivates in WordPress Studio.
This PR attempts to find the root cause of the deactivation.

Note Studio doesn't use Playground CLI, but the same issue is also observable in the CLI.

Research

Recreate issue

  • Set your local Node version to 22 nvm use 22
  • Run the test script using sh run-cli.sh
  • See that it fails on the Plugin activation step with Plugin file does not exist.

Debugging

Notes

trunk behavior

This branch is based on add/file-mounting-to-nodefs and fails slightly differently from trunk.

When I first started testing this issue with commit 92d0ce7, Playground CLI would boot and activate WooCommerce, but deactivate it after a random number of reloads after a WP_Cron request and after some WC-JSON API calls. I wasn't able to fully understand the issue so, I worked on adding mount tests in and resolved some mount issues in the process.

After rebasing this branch onto add/file-mounting-to-nodefs, Playground CLI started failing to boot due to the Plugin activation step failing with Plugin file does not exist.

WordPress symlinked plugins

Following symlinks used to be broken in WordPress and was fixed by Allow symlinked plugins, there is also a closed WooCommerce issue.

When I mounted a debug.log file to /wordpress/wp-content/debug.log, I observed this warning which confirms that PHP is unable to access the symlinked plugin [16-Jul-2025 10:56:23 UTC] PHP Warning: scandir(/wordpress/wp-content/plugins/woocommerce): Failed to open directory: No such file or directory in /wordpress/wp-admin/includes/plugin.php on line 1115.

It seems like Playground isn't able to register the real path for the plugin using wp_register_plugin_realpath because the mounted plugin directory isn't recognized as a valid plugin.

Other potentially related WooCommerce issues

When I mount a symlink to WooCommerce and start it's onboarding process, the same WC-JSON rest API requests failed every time, but they would succeed after a reload. This might be a completely different issue, but I haven't observed it when testing on a LAMP stack, so I'm mentioning it here.

Screenshot 2025-07-11 at 13 40 03

Other

If you want to mount a WordPress directory into the CLI, there's a bug that will rewrite your WordPress files. [Playground CLI] skip-wordpress-setup overrides the mounted WordPress directory.

@bgrgicak bgrgicak self-assigned this Jul 10, 2025
@bgrgicak
Copy link
Collaborator Author

Here's a recap of everything related to the symlink issue I learned so far.

  • Some FS.lookupPath calls in FSHelpers and other places in Playground didn't set the follow option to true which causes Emscripten will not follow the symlink when it's the final path component. (e.g. /wordpress/wp-content/plugins/woocomerce -> /MY-LOCAL-FS/woocommerce)
    • This is a Playground bug, we can fix it by manually setting the follow option to true, or by overriding Emscripten to make it true by default.
  • Even when following is enabled by default some functions like realpath and lstat need to control the follow option.
  • I found the Allow symlinked plugins resolved WordPress core issue, and the closed Resolve all symlink issues WooCommerce issue.

My next steps are to understand if the mentioned Core and Woo issues are related to what we are experiencing.

@bgrgicak bgrgicak changed the title Force follow to always be true in FS.lookupPath [PHP-wasm] set follow to true by default in FS.lookupPath when followSymlinks is enabled Jul 14, 2025
@bgrgicak
Copy link
Collaborator Author

I'm not sure if the current approach of overriding lookupPath is the best. I would prefer to not override Emscripten functions, also the current implementation does it only when followSymlinks is enabled, so we could still have symlink following issues in MEMFS.

@adamziel
Copy link
Collaborator

adamziel commented Jul 14, 2025

Thanks for exploring this @bgrgicak! It seems like we're looking in the right direction but perhaps we're not in the right place yet. In particular, I'm worried about always following the symlink – that would mean we can't do things to the symlink itself, e.g. unlink it, move it around, etc. Perhaps that's not an issue here, since we're not forcing follow: true, merely using it as a default that the caller can override.

I'd still like to understand the consequences. The tests are passing which is very encouraging. But so they did before this PR. Can we add some tests that would fail without this PR? Also, what are some things that could go wrong if we merge? How could we add them to the test harness?

Some FS.lookupPath calls in FSHelpers and other places in Playground didn't set the follow option to true which causes Emscripten will not follow the symlink when it's the final path component.

Which places specifically? If we can pinpoint the root cause we can understand the problem and implement a surgical fix for those data flows.

we could still have symlink following issues in MEMFS.

I wouldn't worry too much about that, that problem just can't be solved. If a sandboxed symlink points to an unsandboxed resource and the user did not allow it, it should break. It's a feature, not a bug.

I would prefer to not override Emscripten functions

We already do that for fcntl, poll, and others. I don't see anything inherently wrong with overriding Emscripten functions.

@bgrgicak
Copy link
Collaborator Author

Thanks for the feedback!

Yes, I feel like the exploration is moving in the right direction, but it's not there yet. My main concern is that I still don't fully understand why WooCommerce is deactivated sometimes. I would like to point to a single line that causes the issue.

My first step is to wrap the tests in #2338 to ensure this doesn't break any existing behavior.

Also, I'm not sure that setting follow to true by default is the best way forward. I would still prefer for this to just work with existing Emscripten code and only add follow to our helpers if they don't set the option correctly.

Which places specifically? If we can pinpoint the root cause we can understand the problem and implement a surgical fix for those data flows.

#2338 will help uncover these places, but for now it looks like fileExists, copyRecursive, and mv. But to be precise, I first need to create POSIX-compliant tests and compare FS operations that way.

@adamziel
Copy link
Collaborator

adamziel commented Jul 14, 2025

A POSIX-compliant test suite sounds really good!

@bgrgicak
Copy link
Collaborator Author

👋 Hi @bgrgicak, is this something I could take on as you are heading AFK?

Thanks @brandonpayton! I will probably need to handover this. If you have time this week let's start collaborating.
I will share some testing instructions today so you can see the issues.

@bgrgicak bgrgicak changed the title [PHP-wasm] set follow to true by default in FS.lookupPath when followSymlinks is enabled [Playground CLI] explore why plugin files don't exist after mounting Jul 16, 2025
@bgrgicak
Copy link
Collaborator Author

@brandonpayton I updated the PR with my notes and a script that will allow you to recreate it. Please let me know how I can make the handof easier.

@brandonpayton
Copy link
Member

Thanks @brandonpayton! I will probably need to handover this. If you have time this week let's start collaborating.
I will share some testing instructions today so you can see the issues.

@bgrgicak Thank you! I didn't get a chance to look at this today but will prioritize it tomorrow so we have a chance to talk yet this week.

Base automatically changed from add/file-mounting-to-nodefs to trunk July 17, 2025 05:43
@bgrgicak
Copy link
Collaborator Author

bgrgicak commented Jul 17, 2025

@brandonpayton checkout this commit ae7033b. I was able to find which PHP functions don't follow symlinks during plugin activation and cause the issue.

@bgrgicak bgrgicak force-pushed the fix/lookup-path-not-following-symlink-mounts branch from 476e647 to ae7033b Compare July 17, 2025 12:22
@brandonpayton
Copy link
Member

@bgrgicak I took a look at this PR and am planning to meet with you tomorrow to discuss. See you then!

@bgrgicak bgrgicak assigned brandonpayton and unassigned bgrgicak Jul 18, 2025
@brandonpayton
Copy link
Member

I am able to see when the file lookup starts failing with breakpoints in the faccessat system call JS. It's not clear why yet, and I plan to investigate this more tomorrow.

@brandonpayton
Copy link
Member

I am able to see when the file lookup starts failing with breakpoints in the faccessat system call JS. It's not clear why yet, and I plan to investigate this more tomorrow.

The first call to ___syscall_faccessat successfully looks up the FS node for the path /wordpress/wp-content/plugins/plugin.

On the second call to ___syscall_faccessat for the same path, the node lookup fails after following the symlink to a path beginning with /internals/symlinks/. During the lookup, the node for /internals cannot be found.

This is a good clue.

@brandonpayton
Copy link
Member

This isn't the problem, but there is a typo we can correct as well:
We are using the path /internals/symlinks at the moment, but all our other internal paths use a singular /internal/ dir. So we should be using /internal/symlinks.

@brandonpayton
Copy link
Member

It looks like this is an issue with secondary PHP instances and /internal/symlinks not being proxied with ProxyFS.

When /internal/symlinks is added to the ProxyFS path list, I no longer see any issue with the symlinked plugin disappearing during boot.

@brandonpayton
Copy link
Member

A next step is making a good test for this. We can probably test it directly using a primary and secondary PHP instance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Resolve all symlink issues
3 participants