New feature
I am proposing a new feature that implements a finer grained check in the AssetManager to distinguish between file-system local and truly remote SCM repositories.
Use case
This feature is useful to resolve a chicken-and-egg conundrum with offline / air-gapped HPC environments and pipelines managed by an on-premise Seqera Platform Enterprise installation:
-
The compute environment needs to be set up with tw compute-envs add slurm ... --env "head:NXF_OFFLINE=true" to ensure that Nextflow doesn't try connecting to the web. Only setting --env "head:NXF_DISABLE_CHECK_LATEST=true" falls short, since it disables the check for the latest Nextflow version, but not attempts to connect to the plugin registry.
-
At the same time, setting NXF_OFFLINE to true prevents pipelines from launching for the first time with an Unknown project local/pipeline.git -- NOTE: automatic download from remote repositories is disabled error.
This can be resolved either by pulling from file-system local paths even in offline mode (this proposal) or alternatively by introducing an NXF_DISABLE_CHECK_PLUGINS for the plugin registry analogous to NXF_DISABLE_CHECK_LATEST for the Nextflow version.
Steps to reproduce
- Create a local bare clone of a Git repository, e.g. with
git clone --bare git@github.com:nextflow-io/socks.git. (This is equivalent to using nf-core pipelines download --platform -r "main" nextflow-io/socks but avoids the additional dependency)
- Try launching a run in
-offline mode with an absolute path and the file:/ prefix: nextflow run -offline file:/$(readlink -f socks.git). (This is the setup that would be chosen in an on-premise Seqera Platform Enterprise installation):
N E X T F L O W ~ version 25.10.4
Unknown project local/socks -- NOTE: automatic download from remote repositories is disabled
- Optionally, repeat without
-offline. nextflow run file:/$(readlink -f socks.git):
N E X T F L O W ~ version 25.10.4
Pulling local/socks ...
downloaded from file:/Users/matthias.zepper/Documents/Coding/Nextflow/download_pipelines/socks
Launching /Users/matthias.zepper/Documents/Coding/Nextflow/download_pipelines/socks
[hopeful_chandrasekhar] DSL2 - revision: 3d3b08ffeb [main]
🧦
- Since the pipeline is now available in
$NXF_ASSETS it will launch even in -offline mode: nextflow run -offline file:/$(readlink -f socks.git):
N E X T F L O W ~ version 25.10.4
Launching /Users/matthias.zepper/Documents/Coding/Nextflow/download_pipelines/socks
[nauseous_darwin] DSL2 - revision: 3d3b08ffeb [main]
🧦
- If the asset is removed
nextflow drop file:/$(readlink -f socks.git) && nextflow run -offline file:/$(readlink -f socks.git) the inital blocked state is restored:
N E X T F L O W ~ version 25.10.4
Unknown project local/socks -- NOTE: automatic download from remote repositories is disabled
Suggested implementation
The requirement to keep cached versions of the pipelines in $NXF_ASSETS defeats the purpose of conveniently managing them centrally for multiple users with Seqera Platform in air-gapped HPC scenarios.
I cannot think of a case when the current behavior of blocking local pulls in -offline mode as well would be required or desirable, therefore I propose to allow them in offline mode.
A draft/preview for this feature is available at this feature branch.
New feature
I am proposing a new feature that implements a finer grained check in the AssetManager to distinguish between file-system local and truly remote SCM repositories.
Use case
This feature is useful to resolve a chicken-and-egg conundrum with offline / air-gapped HPC environments and pipelines managed by an on-premise Seqera Platform Enterprise installation:
The compute environment needs to be set up with
tw compute-envs add slurm ... --env "head:NXF_OFFLINE=true"to ensure that Nextflow doesn't try connecting to the web. Only setting--env "head:NXF_DISABLE_CHECK_LATEST=true"falls short, since it disables the check for the latest Nextflow version, but not attempts to connect to the plugin registry.At the same time, setting
NXF_OFFLINEto true prevents pipelines from launching for the first time with an Unknown project local/pipeline.git -- NOTE: automatic download from remote repositories is disabled error.This can be resolved either by pulling from file-system local paths even in offline mode (this proposal) or alternatively by introducing an
NXF_DISABLE_CHECK_PLUGINSfor the plugin registry analogous toNXF_DISABLE_CHECK_LATESTfor the Nextflow version.Steps to reproduce
git clone --bare git@github.com:nextflow-io/socks.git. (This is equivalent to usingnf-core pipelines download --platform -r "main" nextflow-io/socksbut avoids the additional dependency)-offlinemode with an absolute path and thefile:/prefix:nextflow run -offline file:/$(readlink -f socks.git). (This is the setup that would be chosen in an on-premise Seqera Platform Enterprise installation):-offline.nextflow run file:/$(readlink -f socks.git):$NXF_ASSETSit will launch even in-offlinemode:nextflow run -offline file:/$(readlink -f socks.git):nextflow drop file:/$(readlink -f socks.git) && nextflow run -offline file:/$(readlink -f socks.git)the inital blocked state is restored:Suggested implementation
The requirement to keep cached versions of the pipelines in
$NXF_ASSETSdefeats the purpose of conveniently managing them centrally for multiple users with Seqera Platform in air-gapped HPC scenarios.I cannot think of a case when the current behavior of blocking local pulls in
-offlinemode as well would be required or desirable, therefore I propose to allow them in offline mode.A draft/preview for this feature is available at this feature branch.