Skip to content

nextflow run -offline should not affect file-system local repositories #6988

@MatthiasZepper

Description

@MatthiasZepper

New feature

I am proposing a new feature that implements a finer grained check in the AssetManager to distinguish between file-system local and truly remote SCM repositories.

Use case

This feature is useful to resolve a chicken-and-egg conundrum with offline / air-gapped HPC environments and pipelines managed by an on-premise Seqera Platform Enterprise installation:

  • The compute environment needs to be set up with tw compute-envs add slurm ... --env "head:NXF_OFFLINE=true" to ensure that Nextflow doesn't try connecting to the web. Only setting --env "head:NXF_DISABLE_CHECK_LATEST=true" falls short, since it disables the check for the latest Nextflow version, but not attempts to connect to the plugin registry.

  • At the same time, setting NXF_OFFLINE to true prevents pipelines from launching for the first time with an Unknown project local/pipeline.git -- NOTE: automatic download from remote repositories is disabled error.

This can be resolved either by pulling from file-system local paths even in offline mode (this proposal) or alternatively by introducing an NXF_DISABLE_CHECK_PLUGINS for the plugin registry analogous to NXF_DISABLE_CHECK_LATEST for the Nextflow version.

Steps to reproduce

  1. Create a local bare clone of a Git repository, e.g. with git clone --bare git@github.com:nextflow-io/socks.git. (This is equivalent to using nf-core pipelines download --platform -r "main" nextflow-io/socks but avoids the additional dependency)
  2. Try launching a run in -offline mode with an absolute path and the file:/ prefix: nextflow run -offline file:/$(readlink -f socks.git). (This is the setup that would be chosen in an on-premise Seqera Platform Enterprise installation):

N E X T F L O W ~ version 25.10.4
Unknown project local/socks -- NOTE: automatic download from remote repositories is disabled

  1. Optionally, repeat without -offline. nextflow run file:/$(readlink -f socks.git):

N E X T F L O W ~ version 25.10.4
Pulling local/socks ...
downloaded from file:/Users/matthias.zepper/Documents/Coding/Nextflow/download_pipelines/socks
Launching /Users/matthias.zepper/Documents/Coding/Nextflow/download_pipelines/socks
[hopeful_chandrasekhar] DSL2 - revision: 3d3b08ffeb [main]
🧦

  1. Since the pipeline is now available in $NXF_ASSETS it will launch even in -offline mode: nextflow run -offline file:/$(readlink -f socks.git):

N E X T F L O W ~ version 25.10.4
Launching /Users/matthias.zepper/Documents/Coding/Nextflow/download_pipelines/socks
[nauseous_darwin] DSL2 - revision: 3d3b08ffeb [main]
🧦

  1. If the asset is removed nextflow drop file:/$(readlink -f socks.git) && nextflow run -offline file:/$(readlink -f socks.git) the inital blocked state is restored:

N E X T F L O W ~ version 25.10.4
Unknown project local/socks -- NOTE: automatic download from remote repositories is disabled

Suggested implementation

The requirement to keep cached versions of the pipelines in $NXF_ASSETS defeats the purpose of conveniently managing them centrally for multiple users with Seqera Platform in air-gapped HPC scenarios.

I cannot think of a case when the current behavior of blocking local pulls in -offline mode as well would be required or desirable, therefore I propose to allow them in offline mode.

A draft/preview for this feature is available at this feature branch.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions