Skip to content

Conversation

pinin4fjords
Copy link
Collaborator

@pinin4fjords pinin4fjords commented Aug 14, 2025

Add Essential Nextflow Scripting Patterns side quest

This PR introduces a comprehensive hands-on tutorial that teaches essential Nextflow scripting patterns through practical bioinformatics examples.

Summary

The "Essential Nextflow Scripting Patterns" side quest takes learners on a journey from basic concepts to production-ready patterns, transforming a simple CSV-reading workflow into a sophisticated bioinformatics pipeline step-by-step.

Topics Covered

  • Dataflow vs Scripting: Understanding the boundaries between workflow orchestration and data manipulation
  • Data Manipulation: Working with maps and collections using operators like findAll(), collectEntries(), and groupBy()
  • String Processing: Regex patterns for parsing complex file naming schemes and mastering variable interpolation
  • Reusable Functions: Extracting complex logic into named functions for maintainable workflows
  • Dynamic Logic: Building adaptive processes with closures for dynamic resource allocation
  • Conditional Routing: Intelligent sample routing based on metadata characteristics
  • Safe Operations: Null-safe operators and input validation with clear error messages
  • Event Handlers: Configuration-based logging, notifications, and lifecycle management

Target Audience

Learners who have completed Hello Nextflow or have equivalent experience with basic Nextflow concepts (processes, channels, workflows). Programming experience is helpful but not required—concepts are explained as encountered.

Structure

The tutorial uses a realistic dataset (CSV with biological sample metadata) and evolves a working pipeline through practical challenges, building from fundamentals to advanced production patterns.

Copy link

netlify bot commented Aug 14, 2025

Deploy Preview for nextflow-training ready!

Name Link
🔨 Latest commit 9d56de2
🔍 Latest deploy log https://app.netlify.com/projects/nextflow-training/deploys/68eeab9c2dd5890008bebe2d
😎 Deploy Preview https://deploy-preview-652--nextflow-training.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@pinin4fjords pinin4fjords marked this pull request as draft August 14, 2025 16:59
@bentsherman
Copy link
Member

I think this content is awesome, but I have a higher-level objection to the use of "Groovy" at all.

The Nextflow language specification was created in part to make it easier to talk about Nextflow as a standalone language. So rather than saying "Nextflow is based on Groovy but we promise it's not that bad!" we can instead say "Nextflow is a programming language that runs on the JVM".

It also gets confusing when talking about things like "Nextflow operators" vs "Groovy functions"... well, Nextflow operators are Groovy functions! They have the same syntax and (mostly) the same semantics

And it will get more confusing in 25.10 when it becomes possible to write code like this:

def sample_ids: List<String> = ['sample_001', 'sample_002', 'sample_003']

Which this tutorial might call "pure Groovy code", but isn't really pure Groovy because it isn't valid Groovy syntax. The distinction between Nextflow and Groovy will become less and less useful over time as the languages diverge futher

I think the distinction you are usually getting at with "Nextflow constructs" vs "Groovy constructs" is dataflow logic in the workflow vs "regular" logic (procedural logic?) in places like an operator closure. So I would describe it in those terms instead of Nextflow vs Groovy

Similarly, on the section about collect operator vs Groovy's collect method, well... this method is in the Nextflow standard library! So you don't even need to talk about Groovy, you can just say it's the collect method of the Iterable type vs the Channel type

In fact, I think you should be able to link any piece of syntax or standard library to the Nextflow reference docs -- and if you can't find it, tell me and I will add it. I want to have the Nextflow language fully specified within our docs so that we don't have to mention Groovy at all. I mean it

@pinin4fjords
Copy link
Collaborator Author

pinin4fjords commented Oct 14, 2025

Thanks for the feedback @bentsherman, really useful. I'll re-pitch and get you to re-review if you get time.

Edit: done, with Claude's help.

I've used 'scripting' as the counterpoint to 'dataflow', because we're not necessarily talking to a crowd to whom 'procedural' makes enough sense. See what you think.

pinin4fjords and others added 2 commits October 14, 2025 12:40
Address reviewer feedback to reframe content away from "Nextflow vs Groovy"
and toward presenting Nextflow as a standalone language.

Major changes:
- Renamed from groovy_essentials to essential_scripting_patterns
- Updated terminology: "dataflow vs scripting" instead of "Nextflow vs Groovy"
- Reframed collect example as Channel vs Iterable types in Nextflow standard library
- Added historical context about Groovy while emphasizing Nextflow language specification
- Updated all code comments and examples to reference Nextflow standard library
- Prioritized Nextflow documentation in resources section

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
@bentsherman bentsherman changed the title Add Groovy essentials module Add Scripting essentials module Oct 14, 2025
Copy link
Member

@bentsherman bentsherman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a few suggestions, fixed a few incorrect links.

Try to make sure to follow best practices with Nextflow code examples:

  • Use channel instead of Channel for channel factories
  • Avoid implicit it closure parameter
  • Use tuple() instead of a list when you want to make a tuple

You might have to fight the AI on this since these patterns are historically quite common. Maybe just including the above guidance in your prompt will be enough 🤷

Thanks for putting all of this together. I intend to keep expanding the language reference docs to support these kinds of examples

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants