Right now if a service need to run initialization code in the executor worker, it's somewhat difficult or hackish.
We added hooks.on_executor_initialized, but many executor (ARQ, Ray) don't support it.
Right now, we actually rely on the executor to call services initializer (ie.
|
initialize_hook.register(logger.on_executor_initialized) |
)
It would be better if we extend BootstrapPipeline that take care of this. All executor have to instantiate this call to execute message. We could rely on this to ensure that the worker had initialized the services. It might require the executor worker to load Saturn config and pass it to the bootstrapper and let the services define a open_in_executor and remove the partly supported on_executor_initialized.