feat: declared background workers + frankenphp_get_worker_handle()

Background workers run a script in a loop outside the HTTP request
cycle, sharing the PHP runtime with the request threads. Rebuilt on
Server from #2499: a background worker attaches to a php_server through
WithWorkerServerScope() like any other worker.

Declared with "background" in a worker block (php_server or global) or
WithWorkerBackground() in Go. name is required, match is rejected,
num >= 1. The lifecycle mirrors HTTP workers: re-run on a cooperative
exit, restart with quadratic backoff on a crash, max_consecutive_failures
fails Init() during startup only. drain() runs on shutdown, reboot and
handler transitions so a parked script wakes up instead of waiting out
the force-kill grace period.

Every worker sees its declared name in $_SERVER['FRANKENPHP_WORKER'],
HTTP workers included: the documented contract is to test its presence,
not its value. Background workers also get
$_SERVER['FRANKENPHP_WORKER_BACKGROUND'], so a script serving both roles
can tell them apart with isset().

The script gets one handle, frankenphp_get_worker_handle(), a stream
that reaches EOF when the worker is drained, meant to carry control
messages later. It is backed by a socket pair, not a pipe: on Windows
PHP's php_select() only waits properly on sockets before 8.5. Streams
do not own the socket (php_sockop_close() would shutdown() it on
Windows), so every call returns a fresh stream and closing one never
affects another; the read timeout is infinite so a blocking read parks
as well as stream_select() does. Both ends are non-inheritable.

A worker counts as ready on its first wait on the handle (select cast
or read), the background analog of frankenphp_handle_request(): Init()
waits for it, ready_workers counts from it, and an exit before it is a
boot failure. The handle's stream ops, copied from the socket ops at
MINIT, report it once per run.

Worker names are scoped like paths: unique within a php_server or
among global workers. The script sees the declared name; metrics and
logs report a scoped worker as "<server name>:<name>", with a numeric
suffix on server names when two blocks resolve to the same one. The
collision-driven renaming in the Caddy module is gone, and
WithWorkerName() resolves within the request's server first.

Supersedes #2543 and #2398.

Author: nicolas-grekas

bgworker_test.go

@@ -17,7 +17,6 @@ import (
 	"github.com/caddyserver/caddy/v2/caddyconfig/httpcaddyfile"
 	"github.com/caddyserver/caddy/v2/modules/caddyhttp"
 	"github.com/dunglas/frankenphp"
-	"github.com/dunglas/frankenphp/internal/fastabs"
 )
 
 var (
@@ -60,15 +59,14 @@ type FrankenPHPApp struct {
 	// EXPERIMENTAL: MaxRequests sets the maximum number of requests a PHP thread handles before restarting (0 = unlimited)
 	MaxRequests int `json:"max_requests,omitempty"`
 
-	opts            []frankenphp.Option
-	metrics         frankenphp.Metrics
-	ctx             context.Context
-	logger          *slog.Logger
-	modules         []*FrankenPHPModule
-	usedWorkerNames map[string]bool
-	httpApp         *caddyhttp.App
-	hasStarted      atomic.Bool
-	started         chan any
+	opts       []frankenphp.Option
+	metrics    frankenphp.Metrics
+	ctx        context.Context
+	logger     *slog.Logger
+	modules    []*FrankenPHPModule
+	httpApp    *caddyhttp.App
+	hasStarted atomic.Bool
+	started    chan any
 }
 
 var errIni = errors.New(`"php_ini" must be in the format: php_ini "<key>" "<value>"`)
@@ -133,7 +131,6 @@ func (f *FrankenPHPApp) Start() error {
 	// register global workers
 	for _, w := range f.Workers {
 		w.FileName = repl.ReplaceKnown(w.FileName, "")
-		w.Name = f.createUniqueWorkerName(w, "")
 		opts, err := w.toWorkerOptions()
 		if err != nil {
 			return err
@@ -224,7 +221,6 @@ func (f *FrankenPHPApp) registerModule(repl *caddy.Replacer, module *FrankenPHPM
 
 	for _, w := range module.Workers {
 		w.FileName = repl.ReplaceKnown(w.FileName, "")
-		w.Name = f.createUniqueWorkerName(w, serverName)
 		workerOptions, err := w.toWorkerOptions()
 		if err != nil {
 			return err
@@ -236,37 +232,6 @@ func (f *FrankenPHPApp) registerModule(repl *caddy.Replacer, module *FrankenPHPM
 	return nil
 }
 
-// avoid name collisions for workers
-// on collision, a name is first qualified with the server name
-// ("<serverName>:<name>") before falling back to a numeric postfix
-func (f *FrankenPHPApp) createUniqueWorkerName(wc workerConfig, serverName string) string {
-	if f.usedWorkerNames == nil {
-		f.usedWorkerNames = make(map[string]bool)
-	}
-
-	if wc.Name == "" {
-		wc.Name, _ = fastabs.FastAbs(wc.FileName)
-	}
-
-	name := wc.Name
-	suffix := 0
-	for {
-		if _, ok := f.usedWorkerNames[name]; !ok {
-			f.usedWorkerNames[name] = true
-			break
-		}
-		if serverName != "" {
-			name = serverName + ":" + wc.Name
-			serverName = ""
-			continue
-		}
-		suffix++
-		name = fmt.Sprintf("%s_%d", wc.Name, suffix)
-	}
-
-	return name
-}
-
 // UnmarshalCaddyfile implements caddyfile.Unmarshaler.
 func (f *FrankenPHPApp) UnmarshalCaddyfile(d *caddyfile.Dispenser) error {
 	for d.Next() {

caddy/app.go

@@ -839,7 +839,7 @@ func TestWorkerMetrics(t *testing.T) {
 	# TYPE frankenphp_worker_request_count counter
 	frankenphp_worker_request_count{worker="` + workerName + `"} 10
 
-	# HELP frankenphp_ready_workers Running workers that have successfully called frankenphp_handle_request at least once
+	# HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, frankenphp_get_worker_handle for background workers
 	# TYPE frankenphp_ready_workers gauge
 	frankenphp_ready_workers{worker="` + workerName + `"} 2
 	`
@@ -996,7 +996,7 @@ func TestNamedWorkerMetrics(t *testing.T) {
 	# TYPE frankenphp_worker_request_count counter
 	frankenphp_worker_request_count{worker="my_app"} 10
 
-	# HELP frankenphp_ready_workers Running workers that have successfully called frankenphp_handle_request at least once
+	# HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, frankenphp_get_worker_handle for background workers
 	# TYPE frankenphp_ready_workers gauge
 	frankenphp_ready_workers{worker="my_app"} 2
 	`
@@ -1092,7 +1092,7 @@ func TestAutoWorkerConfig(t *testing.T) {
 	# TYPE frankenphp_worker_request_count counter
 	frankenphp_worker_request_count{worker="` + workerName + `"} 10
 
-	# HELP frankenphp_ready_workers Running workers that have successfully called frankenphp_handle_request at least once
+	# HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, frankenphp_get_worker_handle for background workers
 	# TYPE frankenphp_ready_workers gauge
 	frankenphp_ready_workers{worker="` + workerName + `"} ` + workers + `
 	`
@@ -1460,7 +1460,7 @@ func TestMultiWorkersMetrics(t *testing.T) {
 	# TYPE frankenphp_worker_request_count counter
 	frankenphp_worker_request_count{worker="service1"} 10
 
-	# HELP frankenphp_ready_workers Running workers that have successfully called frankenphp_handle_request at least once
+	# HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, frankenphp_get_worker_handle for background workers
 	# TYPE frankenphp_ready_workers gauge
 	frankenphp_ready_workers{worker="service1"} 2
 	frankenphp_ready_workers{worker="service2"} 3
@@ -1614,7 +1614,7 @@ func TestWorkerRestart(t *testing.T) {
 
 	// Check metrics
 	expectedMetrics := `
-	# HELP frankenphp_ready_workers Running workers that have successfully called frankenphp_handle_request at least once
+	# HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, frankenphp_get_worker_handle for background workers
 	# TYPE frankenphp_ready_workers gauge
 	frankenphp_ready_workers{worker="service"} 1
 	# HELP frankenphp_total_workers Total number of PHP workers for this worker
@@ -1642,7 +1642,7 @@ func TestWorkerRestart(t *testing.T) {
 
 	// frankenphp_ready_workers should be back to 1 even after worker restarts
 	expectedMetrics = `
-	# HELP frankenphp_ready_workers Running workers that have successfully called frankenphp_handle_request at least once
+	# HELP frankenphp_ready_workers Running workers that have reached their ready point at least once: frankenphp_handle_request for HTTP workers, frankenphp_get_worker_handle for background workers
 	# TYPE frankenphp_ready_workers gauge
 	frankenphp_ready_workers{worker="service"} 1
 	# HELP frankenphp_total_workers Total number of PHP workers for this worker

caddy/caddy_test.go

@@ -1,7 +1,6 @@
 package caddy
 
 import (
-	"path/filepath"
 	"testing"
 	"time"
 
@@ -249,38 +248,73 @@ func TestModuleWorkerWithCustomName(t *testing.T) {
 	require.Equal(t, "../testdata/worker-with-env.php", module.Workers[0].FileName, "Worker should have the correct filename")
 }
 
-func TestCreateUniqueWorkerNames(t *testing.T) {
-	app := &FrankenPHPApp{}
-	filename := "../testdata/worker-with-env.php"
-	absFileName, _ := filepath.Abs(filename)
-	names := make([]string, 6)
-	for i := range 3 {
-		names[i] = app.createUniqueWorkerName(workerConfig{
-			FileName: filename,
-			Name:     "custom-worker-name",
-		}, "")
-		names[i+3] = app.createUniqueWorkerName(workerConfig{
-			FileName: filename,
-		}, "")
-	}
-
-	require.Equal(t, "custom-worker-name", names[0])
-	require.Equal(t, "custom-worker-name_1", names[1])
-	require.Equal(t, "custom-worker-name_2", names[2])
-	require.Equal(t, absFileName, names[3])
-	require.Equal(t, absFileName+"_1", names[4])
-	require.Equal(t, absFileName+"_2", names[5])
+func TestWorkerBackgroundConfig(t *testing.T) {
+	d := caddyfile.NewTestDispenser(`
+	{
+		php_server {
+			worker {
+				name jobs
+				file ../testdata/worker-with-env.php
+				num 2
+				background
+			}
+		}
+	}`)
+	module := &FrankenPHPModule{}
+
+	require.NoError(t, module.UnmarshalCaddyfile(d))
+	require.Len(t, module.Workers, 1)
+	require.True(t, module.Workers[0].Background)
+	require.Equal(t, "jobs", module.Workers[0].Name)
+}
+
+func TestWorkerBackgroundRequiresName(t *testing.T) {
+	d := caddyfile.NewTestDispenser(`
+	{
+		php_server {
+			worker {
+				file ../testdata/worker-with-env.php
+				background
+			}
+		}
+	}`)
+	module := &FrankenPHPModule{}
+
+	err := module.UnmarshalCaddyfile(d)
+	require.ErrorContains(t, err, `background workers must have an explicit "name"`)
 }
 
-func TestCreateUniqueWorkerNamesQualifiedByServer(t *testing.T) {
-	app := &FrankenPHPApp{}
-	wc := workerConfig{FileName: "../testdata/worker-with-env.php", Name: "queue"}
-
-	require.Equal(t, "queue", app.createUniqueWorkerName(wc, "one.example.com"))
-	// on collision, the name is qualified with the server name
-	require.Equal(t, "two.example.com:queue", app.createUniqueWorkerName(wc, "two.example.com"))
-	// when the qualified name is also taken, fall back to the numeric postfix
-	require.Equal(t, "queue_1", app.createUniqueWorkerName(wc, "two.example.com"))
-	// workers without a server keep the numeric postfix behavior
-	require.Equal(t, "queue_2", app.createUniqueWorkerName(wc, ""))
+func TestWorkerBackgroundRequiresNum(t *testing.T) {
+	d := caddyfile.NewTestDispenser(`
+	{
+		php_server {
+			worker {
+				name jobs
+				file ../testdata/worker-with-env.php
+				background
+			}
+		}
+	}`)
+	module := &FrankenPHPModule{}
+
+	err := module.UnmarshalCaddyfile(d)
+	require.ErrorContains(t, err, `background workers must declare "num" >= 1`)
+}
+
+func TestWorkerBackgroundRejectsMatch(t *testing.T) {
+	d := caddyfile.NewTestDispenser(`
+	{
+		php_server {
+			worker {
+				name jobs
+				file ../testdata/worker-with-env.php
+				match /jobs/*
+				background
+			}
+		}
+	}`)
+	module := &FrankenPHPModule{}
+
+	err := module.UnmarshalCaddyfile(d)
+	require.ErrorContains(t, err, `"match" is not supported for background workers`)
 }

caddy/config_test.go

@@ -315,6 +315,10 @@ func (f *FrankenPHPModule) UnmarshalCaddyfile(d *caddyfile.Dispenser) error {
 	// Check if a worker with this filename already exists in this module
 	fileNames := make(map[string]struct{}, len(f.Workers))
 	for _, w := range f.Workers {
+		// background workers are keyed by name, several may share a script
+		if w.Background {
+			continue
+		}
 		if _, ok := fileNames[w.FileName]; ok {
 			return fmt.Errorf(`workers in a single "php" or "php_server" block must not have duplicate filenames: %q`, w.FileName)
 		}

caddy/module.go

@@ -22,7 +22,7 @@ import (
 type workerConfig struct {
 	mercureContext
 
-	// Name for the worker. Default: the absolute path of the worker file, postfixed with a number if the name is already used.
+	// Name for the worker, unique within its php_server (or among global workers). Default: the absolute path of the worker file.
 	Name string `json:"name,omitempty"`
 	// FileName sets the path to the worker script.
 	FileName string `json:"file_name,omitempty"`
@@ -38,6 +38,8 @@ type workerConfig struct {
 	MatchPath []string `json:"match_path,omitempty"`
 	// MaxConsecutiveFailures sets the maximum number of consecutive failures before panicking (defaults to 6, set to -1 to never panick)
 	MaxConsecutiveFailures int `json:"max_consecutive_failures,omitempty"`
+	// Background marks this worker as a background (non-HTTP) worker.
+	Background bool `json:"background,omitempty"`
 
 	options []frankenphp.WorkerOption
 }
@@ -139,15 +141,29 @@ func unmarshalWorker(d *caddyfile.Dispenser) (workerConfig, error) {
 			}
 
 			wc.MaxConsecutiveFailures = v
+		case "background":
+			wc.Background = true
 		default:
-			return wc, wrongSubDirectiveError("worker", "name, file, num, env, watch, match, max_consecutive_failures, max_threads", v)
+			return wc, wrongSubDirectiveError("worker", "name, file, num, env, watch, match, max_consecutive_failures, max_threads, background", v)
 		}
 	}
 
 	if wc.FileName == "" {
 		return wc, d.Err(`the "file" argument must be specified`)
 	}
 
+	if wc.Background {
+		if wc.Name == "" {
+			return wc, d.Err(`background workers must have an explicit "name"`)
+		}
+		if len(wc.MatchPath) != 0 {
+			return wc, d.Err(`"match" is not supported for background workers`)
+		}
+		if wc.Num < 1 {
+			return wc, d.Err(`background workers must declare "num" >= 1`)
+		}
+	}
+
 	if frankenphp.EmbeddedAppPath != "" && filepath.IsLocal(wc.FileName) {
 		wc.FileName = filepath.Join(frankenphp.EmbeddedAppPath, wc.FileName)
 	}
@@ -166,6 +182,10 @@ func (wc *workerConfig) toWorkerOptions() ([]frankenphp.WorkerOption, error) {
 	// options collected while provisioning the module, e.g. the Mercure hub
 	opts = append(opts, wc.options...)
 
+	if wc.Background {
+		opts = append(opts, frankenphp.WithWorkerBackground())
+	}
+
 	// copy the caddy match logic and create a unique matcher function for this worker
 	// inject the matcher into frankenphp
 	if len(wc.MatchPath) > 0 {

caddy/workerconfig.go

@@ -109,8 +109,9 @@ You can also explicitly configure FrankenPHP using the [global option](https://c
 			num <num> # Sets the number of PHP threads to start, defaults to 2x the number of available CPUs.
 			env <key> <value> # Sets an extra environment variable to the given value. Can be specified more than once for multiple environment variables.
 			watch <path> # Sets the path to watch for file changes. Can be specified more than once for multiple paths.
-			name <name> # Sets the name of the worker, used in logs and metrics. Default: absolute path of worker file
+			name <name> # Sets the name of the worker, used in logs and metrics and exposed as $_SERVER['FRANKENPHP_WORKER']. Must be unique among global workers. Default: absolute path of the worker file.
 			max_consecutive_failures <num> # Sets the maximum number of consecutive failures before the worker is considered unhealthy, -1 means the worker will always restart. Default: 6.
+			background # EXPERIMENTAL: marks this worker as a background (non-HTTP) worker; it runs the script in a loop without serving requests, "name" is required, "match" is not allowed. The script is ready once it waits on the stream returned by frankenphp_get_worker_handle() (stream_select() or a blocking read); an exit before that counts as a failure. $_SERVER['FRANKENPHP_WORKER_BACKGROUND'] is set in it. Its threads come on top of num_threads and max_threads, and max_threads is not allowed on it.
 		}
 	}
 }
@@ -187,17 +188,18 @@ php_server [<matcher>] {
 	root <directory> # Sets the root folder to the site. Default: `root` directive.
 	split_path <delim...> # Sets the substrings for splitting the URI into two parts. The first matching substring will be used to split the "path info" from the path. The first piece is suffixed with the matching substring and will be assumed as the actual resource (CGI script) name. The second piece will be set to PATH_INFO for the script to use. Default: `.php`
 	resolve_root_symlink false # Disables resolving the `root` directory to its actual value by evaluating a symbolic link, if one exists (enabled by default).
-	name <name> # Sets the name for this server, used to attribute workers, metrics and logs. Default: the first host matcher of the enclosing route, or the first listener address.
+	name <name> # Sets the name for this server, used to attribute workers, metrics and logs. Default: the first host matcher of the enclosing route, or the first listener address. Suffixed with a number if another php_server resolves to the same name.
 	env <key> <value> # Sets an extra environment variable to the given value. Can be specified more than once for multiple environment variables.
 	file_server off # Disables the built-in file_server directive.
 	request_body_timeout <duration> # Sets an idle timeout on request body reads: a stalled (slow POST) client is cut off while a steady upload of any size succeeds. Default: 60s. Set to 0 to disable.
 	worker { # Creates a worker specific to this server. Can be specified more than once for multiple workers.
 		file <path> # Sets the path to the worker script, can be relative to the php_server root
 		num <num> # Sets the number of PHP threads to start, defaults to 2x the number of available
-		name <name> # Sets the name for the worker, used in logs and metrics. Default: absolute path of worker file. Postfixed with a number if name is already in use.
+		name <name> # Sets the name for the worker, used in logs and metrics and exposed as $_SERVER['FRANKENPHP_WORKER']. Must be unique within this php_server. In logs and metrics, the worker is reported as "<server name>:<name>". Default: absolute path of the worker file.
 		watch <path> # Sets the path to watch for file changes. Can be specified more than once for multiple paths.
 		env <key> <value> # Sets an extra environment variable to the given value. Can be specified more than once for multiple environment variables. Environment variables for this worker are also inherited from the php_server parent, but can be overwritten here.
 		match <path> # match the worker to a path pattern. Overrides try_files and can only be used in the php_server directive.
+		background # EXPERIMENTAL: marks this worker as a background (non-HTTP) worker; it runs the script in a loop without serving requests, "name" is required, "match" is not allowed. The script is ready once it waits on the stream returned by frankenphp_get_worker_handle() (stream_select() or a blocking read); an exit before that counts as a failure. $_SERVER['FRANKENPHP_WORKER_BACKGROUND'] is set in it. Its threads come on top of num_threads and max_threads, and max_threads is not allowed on it.
 	}
 	worker <other_file> <num> # Can also use the short form like in the global frankenphp block.
 }

docs/config.md

@@ -60,6 +60,10 @@ err := frankenphp.Init(
 
 Workers declared without a server scope are global: they match by file path on any server. Since a global worker has no set of requests to match against, combining `WithWorkerMatcher()` with a global worker is a configuration error and `Init()` rejects it.
 
+Worker names are unique within their server (or among global workers), so two servers may each declare a worker named `queue`. The script sees the declared name, while metrics and logs report a server-scoped worker as `<server name>:<worker name>`; server names are made unique with a numeric suffix when needed. `WithWorkerName()` resolves a name within the request's server first, then among global workers.
+
+`WithWorkerBackground()` declares a [background worker](worker.md#background-workers), which runs outside the request cycle.
+
 ## Per-request options
 
 `Server.ServeHTTP()` accepts `RequestOption`s to override the server configuration for a single request, e.g. `WithRequestDocumentRoot()`, `WithRequestSplitPath()`, `WithRequestEnv()` or `WithRequestLogger()`.

docs/library.md

@@ -19,12 +19,12 @@ When [Caddy metrics](https://caddyserver.com/docs/metrics) are enabled, FrankenP
 - `frankenphp_busy_workers{worker="[worker_name]"}`: The number of workers currently processing a request.
 - `frankenphp_worker_request_time{worker="[worker_name]"}`: The time spent processing requests by all workers.
 - `frankenphp_worker_request_count{worker="[worker_name]"}`: The number of requests processed by all workers.
-- `frankenphp_ready_workers{worker="[worker_name]"}`: The number of workers that have called `frankenphp_handle_request` at least once.
+- `frankenphp_ready_workers{worker="[worker_name]"}`: The number of workers that have reached their ready point at least once: `frankenphp_handle_request()` for HTTP workers, `frankenphp_get_worker_handle()` for background workers.
 - `frankenphp_worker_crashes{worker="[worker_name]"}`: The number of times a worker has unexpectedly terminated.
 - `frankenphp_worker_restarts{worker="[worker_name]"}`: The number of times a worker has been deliberately restarted.
 - `frankenphp_worker_queue_depth{worker="[worker_name]"}`: The number of queued requests.
 
-For worker metrics, the `[worker_name]` placeholder is replaced by the worker name in the Caddyfile, otherwise the absolute path of the worker file will be used.
+`[worker_name]` is the worker name from the Caddyfile, or the absolute path of the worker file when it has none. Workers of a `php_server` block are prefixed with the name of that block: `<server name>:<worker name>`.
 
 ## Threads State Endpoint