Feature Request: Add --subdirs flag to obtain directory output for MultiQC compatibility

[hemant-goyal]

Context:
Currently, when running Falco on multiple local files with a single output directory (-o), Falco intelligently prefixes the output filenames to prevent overwriting (e.g., sample_name_R1_fastqc_data.txt or/and sample_name_R2_fastqc_data.txt).

However, downstream parsers like MultiQC strictly mandate the exact filename fastqc_data.txt to successfully ingest the data. Because of the prefixing, MultiQC fails to recognize bulk local Falco outputs unless the user writes a custom wrapper script to isolate each run into its own folder.

Proposed Feature:
Introduce a --subdirs flag. When this argument is passed, Falco would automatically create a sample-specific subdirectory for each input file inside the target -o directory, placing the standard fastqc_data.txt inside it.

Benefit:
This would perfectly mimic FastQC's default directory-creation behavior, making Falco 100% out-of-the-box compatible with MultiQC for bulk local runs without requiring users to write custom bash loops.

(Split from issue #38 as requested).

[andrewdavidsmith]

I think this needs a bit more clarification. You seem to be suggesting that this "Falco intelligently prefixes the output filenames to prevent overwriting", while desirable, might not be the best way to go. I don't see a --subdirs arg in fastqc, so I need a bit more info. I'll look into it myself eventually, but if you can answer it would accelerate things.

[hemant-goyal]

I wanted to clarify exactly why this is the missing puzzle piece for pipeline integration.

Currently, when passing multiple files to a single -o directory, Falco intelligently prefixes the filenames (e.g., sample_name_fastqc_data.txt) to prevent them from overwriting each other. However, MultiQC's internal search_patterns.yaml strictly mandates that the FastQC module only triggers on files named exactly fastqc_data.txt or *_fastqc.zip. During my previous testing with Falco, I explicitly followed this method.

By implementing --subdirs, Falco would organize the batch into sample-specific folders out-of-box, allowing the output files to retain the exact fastqc_data.txt name. This will make Falco natively compatible with MultiQC for bulk local runs without users needing to write custom bash wrappers to isolate the files or custom search configurations.

[andrewdavidsmith]

Would you be able to build from a new branch I create to test that the functionality is what you request? Also, can you specify how this behavior would realign with, or diverge further from, the fastqc behavior?

[Stikus]

@hemant-goyal Why do you prefer subdirs solution above creating .zip archives with base filenames (as in FastQC)? Creating these archives is much favorable for downstream pipelines, targeting same output as FastQC (e.g. MultiQC)

[hemant-goyal]

@andrewdavidsmith Yes, I would be absolutely happy to build from a new branch and run some local validation tests against MultiQC once you have something ready!

As for how --subdirs diverges from FastQC: FastQC does not natively create subdirectories; it dumps everything into the specified --outdir. My proposal for --subdirs was primarily a workaround for how Falco currently handles plaintext .txt outputs to avoid filename collisions without breaking MultiQC's search patterns.
However, @Stikus brings up a good architectural point regarding FastQC parity. Standard FastQC circumvents this whole collision issue by generating [sample_name]_fastqc.zip files, which internally contain the standard fastqc_data.txt. MultiQC would eat those zip files right out of the box.
I think the --subdirs approach is cleaner as individual samples have their own folder, it is just easy to work with.

[andrewdavidsmith]

Thanks, I can probably do this relatively soon.