Skip to content

Conversation

SantaMcCloud
Copy link

FOR CONTRIBUTOR:

  • I have read the Adding workflows guidelines
  • License permits unrestricted use (educational + commercial)
  • Please also take note of the reviewer guidelines below to facilitate a smooth review process.

FOR REVIEWERS:

  • .dockstore.yml: file is present and aligned with creator metadata in workflow. ORCID identifiers are strongly encouraged in creator metadata. The .dockstore.yml file is required to run tests
  • Workflow is sufficiently generic to be used with lab data and does not hardcode sample names, reference data and can be run without reading an accompanying tutorial.
  • In workflow: annotation field contains short description of what the workflow does. Should start with This workflow does/runs/performs … xyz … to generate/analyze/etc …
  • In workflow: workflow inputs and outputs have human readable names (spaces are fine, no underscore, dash only where spelling dictates it), no abbreviation unless it is generally understood. Altering input or output labels requires adjusting these labels in the the workflow-tests.yml file as well
  • In workflow: name field should be human readable (spaces are fine, no underscore, dash only where spelling dictates it), no abbreviation unless generally understood
  • Workflow folder: prefer dash (-) over underscore (_), prefer all lowercase. Folder becomes repository in iwc-workflows organization and is included in TRS id
  • Readme explains what workflow does, what are valid inputs and what outputs users can expect. If a tutorial or other resources exist they can be linked. If a similar workflow exists in IWC readme should explain differences with existing workflow and when one might prefer one workflow over another
  • Changelog contains appropriate entries
  • Large files (> 100 KB) are uploaded to zenodo and location urls are used in test file

@mvdbeek
Copy link
Member

mvdbeek commented Aug 4, 2025

How is this different from #923 ?

@SantaMcCloud
Copy link
Author

How is this different from #923 ?

The difference is that there is no taxonomic classifcation only the binning and the evaluation.

Copy link

github-actions bot commented Aug 5, 2025

Test Results (powered by Planemo)

Test Summary

Test State Count
Total 1
Passed 0
Error 1
Failure 0
Skipped 0
Errored Tests
  • ❌ FAIRyMAGs-binning-evaluation.ga_0

    Workflow invocation details

    • Invocation Messages

    • Steps
      • Step 1: Reads:

        • step_state: scheduled
      • Step 2: Biobox File(s):

        • step_state: scheduled
      • Step 11: toolshed.g2.bx.psu.edu/repos/iuc/semibin/semibin/2.0.2+galaxy0:

        • step_state: scheduled

        • Jobs
          • Job 1:

            • Job state is paused

            Traceback:

            Job Parameters:

            • Job parameter Parameter value
              __input_ext "input"
              __workflow_invocation_uuid__ "301f995c718e11f0a03a7ced8d7042ad"
              annot {"ml_threshold": null}
              bin {"max_edges": "200", "max_node": "1.0", "minfasta_kbs": "200"}
              chromInfo "/tmp/tmp2x7w0yy_/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
              dbkey "?"
              extra_output ["data", "coverage"]
              min_len {"__current_case__": 0, "method": "automatic"}
              mode {"__current_case__": 0, "environment": "", "input_bam": {"values": [{"id": 11, "src": "dce"}]}, "input_fasta": {"values": [{"id": 2, "src": "dce"}]}, "ref": {"__current_case__": 0, "cached_db": "17102022", "select": "cached"}, "select": "single"}
              orf_finder "fast-naive"
              random_seed "0"
              training {"batch_size": "2048", "epoches": "20"}
      • Step 12: toolshed.g2.bx.psu.edu/repos/iuc/concoct_coverage_table/concoct_coverage_table/1.1.0+galaxy2:

        • step_state: scheduled

        • Jobs
          • Job 1:

            • Job state is paused

            Traceback:

            Job Parameters:

            • Job parameter Parameter value
              __input_ext "input"
              __workflow_invocation_uuid__ "301f995c718e11f0a03a7ced8d7042ad"
              bedfile __identifier__
              chromInfo "/tmp/tmp2x7w0yy_/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
              dbkey "?"
              mode {"__current_case__": 0, "bamfile": {"values": [{"id": 11, "src": "dce"}]}, "type": "individual"}
              mode bamfile
      • Step 13: toolshed.g2.bx.psu.edu/repos/iuc/metabat2_jgi_summarize_bam_contig_depths/metabat2_jgi_summarize_bam_contig_depths/2.17+galaxy0:

        • step_state: scheduled

        • Jobs
          • Job 1:

            • Job state is paused

            Traceback:

            Job Parameters:

            • Job parameter Parameter value
              __input_ext "input"
              __workflow_invocation_uuid__ "301f995c718e11f0a03a7ced8d7042ad"
              advanced {"includeEdgeBases": false, "maxEdgeBases": "75", "minMapQual": "0", "noIntraDepthVariance": false, "output_paired_contigs": false, "percentIdentity": "97", "showDepth": false, "weightMapQual": "0.0"}
              bam_indiv_input __identifier__
              chromInfo "/tmp/tmp2x7w0yy_/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
              dbkey "?"
              mode {"__current_case__": 0, "bam_indiv_input": {"values": [{"id": 11, "src": "dce"}]}, "type": "individual", "use_reference_cond": {"__current_case__": 0, "use_reference": "no"}}
              shredding {"minContigDepth": "0.0", "minContigLength": "1", "shredDepth": "5", "shredLength": "16000"}
      • Step 14: toolshed.g2.bx.psu.edu/repos/iuc/collection_element_identifiers/collection_element_identifiers/0.0.2:

        • step_state: scheduled

        • Jobs
          • Job 1:

            • Job state is ok

            Command Line:

            • mv '/tmp/tmp2x7w0yy_/job_working_directory/000/20/configs/tmpou1ny79_' '/tmp/tmp2x7w0yy_/job_working_directory/000/20/outputs/dataset_a3de71b3-f459-42e4-a819-12b7e5c27dcd.dat'

            Exit Code:

            • 0

            Traceback:

            Job Parameters:

            • Job parameter Parameter value
              __input_ext "tabular"
              __workflow_invocation_uuid__ "301f995c718e11f0a03a7ced8d7042ad"
              chromInfo "/tmp/tmp2x7w0yy_/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
              dbkey "?"
              input_collection {"values": [{"id": 31, "src": "hdca"}]}
      • Step 15: Unlabelled step:

        • step_state: new
      • Step 16: Unlabelled step:

        • step_state: new
      • Step 17: toolshed.g2.bx.psu.edu/repos/iuc/concoct/concoct/1.1.0+galaxy2:

        • step_state: scheduled

        • Jobs
          • Job 1:

            • Job state is paused

            Traceback:

            Job Parameters:

            • Job parameter Parameter value
              __input_ext "input"
              __workflow_invocation_uuid__ "301f995c718e11f0a03a7ced8d7042ad"
              advanced {"clusters": "400", "iterations": "500", "kmer_length": "4", "length_threshold": "1000", "no_cov_normalization": false, "read_length": "100", "seed": "1", "total_percentage_pca": "90"}
              chromInfo "/tmp/tmp2x7w0yy_/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
              composition_file __identifier__
              coverage_file __identifier__
              dbkey "?"
              output {"converge_out": false, "log": false, "no_total_coverage": false}
      • Step 18: toolshed.g2.bx.psu.edu/repos/iuc/metabat2/metabat2/2.17+galaxy0:

        • step_state: scheduled

        • Jobs
          • Job 1:

            • Job state is paused

            Traceback:

            Job Parameters:

            • Job parameter Parameter value
              __input_ext "input"
              __workflow_invocation_uuid__ "301f995c718e11f0a03a7ced8d7042ad"
              advanced {"base_coverage_depth_cond": {"__current_case__": 1, "abdFile": {"values": [{"id": 18, "src": "dce"}]}, "base_coverage_depth": "yes", "cvExt": null}, "maxEdges": "200", "maxP": "95", "minCV": "1.0", "minCVSum": "1.0", "minContig": "1500", "minS": "60", "noAdd": false, "pTNF": "0", "seed": "0"}
              advanced abdFile
              chromInfo "/tmp/tmp2x7w0yy_/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
              dbkey "?"
              inFile __identifier__
              out {"extra_outputs": ["lowDepth", "tooShort", "unbinned", "log"], "minClsSize": "200000", "onlyLabel": false, "saveCls": false}
      • Step 19: toolshed.g2.bx.psu.edu/repos/mbernt/maxbin2/maxbin2/2.2.7+galaxy6:

        • step_state: scheduled

        • Jobs
          • Job 1:

            • Job state is paused

            Traceback:

            Job Parameters:

            • Job parameter Parameter value
              __input_ext "fasta"
              __workflow_invocation_uuid__ "301f995c718e11f0a03a7ced8d7042ad"
              adv {"max_iteration": "50", "min_contig_length": "1000", "prob_threshold": "0.5"}
              assembly {"__current_case__": 0, "inputs": {"__current_case__": 1, "abund": {"values": [{"id": 18, "src": "dce"}]}, "type": "abund"}, "type": "individual"}
              chromInfo "/tmp/tmp2x7w0yy_/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
              dbkey "?"
              output {"log": false, "marker": false, "markers": false, "markerset": "107", "plotmarker": false}
      • Step 20: toolshed.g2.bx.psu.edu/repos/bgruening/split_file_to_collection/split_file_to_collection/0.5.2:

        • step_state: scheduled

        • Jobs
          • Job 1:

            • Job state is ok

            Command Line:

            • mkdir ./out && python '/tmp/shed_dir/toolshed.g2.bx.psu.edu/repos/bgruening/split_file_to_collection/2dae863c8f42/split_file_to_collection/split_file_to_collection.py' --out ./out --in '/tmp/tmp2x7w0yy_/files/a/3/d/dataset_a3de71b3-f459-42e4-a819-12b7e5c27dcd.dat' --ftype 'txt' --chunksize 1 --file_names 'split_file' --file_ext 'txt'

            Exit Code:

            • 0

            Traceback:

            Job Parameters:

            • Job parameter Parameter value
              __input_ext "txt"
              __workflow_invocation_uuid__ "301f995c718e11f0a03a7ced8d7042ad"
              chromInfo "/tmp/tmp2x7w0yy_/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
              dbkey "?"
              split_parms {"__current_case__": 5, "input": {"values": [{"id": 31, "src": "hda"}]}, "newfilenames": "split_file", "select_allocate": {"__current_case__": 2, "allocate": "byrow"}, "select_ftype": "txt", "select_mode": {"__current_case__": 0, "chunksize": "1", "mode": "chunk"}}
      • Step 3: Assemblies:

        • step_state: scheduled
      • Step 21: toolshed.g2.bx.psu.edu/repos/iuc/concoct_merge_cut_up_clustering/concoct_merge_cut_up_clustering/1.1.0+galaxy2:

        • step_state: scheduled

        • Jobs
          • Job 1:

            • Job state is paused

            Traceback:

            Job Parameters:

            • Job parameter Parameter value
              __input_ext "input"
              __workflow_invocation_uuid__ "301f995c718e11f0a03a7ced8d7042ad"
              chromInfo "/tmp/tmp2x7w0yy_/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
              cutup_clustering_result __identifier__
              dbkey "?"
      • Step 22: Unlabelled step:

        • step_state: new
      • Step 23: Unlabelled step:

        • step_state: new
      • Step 24: Unlabelled step:

        • step_state: new
      • Step 25: Unlabelled step:

        • step_state: new
      • Step 26: param_value_from_file:

        • step_state: scheduled

        • Jobs
          • Job 1:

            • Job state is ok

            Command Line:

            • cd ../; python _evaluate_expression_.py

            Exit Code:

            • 0

            Traceback:

            Job Parameters:

            • Job parameter Parameter value
              __input_ext "txt"
              __workflow_invocation_uuid__ "301f995c718e11f0a03a7ced8d7042ad"
              chromInfo "/tmp/tmp2x7w0yy_/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
              dbkey "?"
              param_type "text"
              remove_newlines true
      • Step 27: toolshed.g2.bx.psu.edu/repos/iuc/concoct_extract_fasta_bins/concoct_extract_fasta_bins/1.1.0+galaxy2:

        • step_state: scheduled

        • Jobs
          • Job 1:

            • Job state is paused

            Traceback:

            Job Parameters:

            • Job parameter Parameter value
              __input_ext "input"
              __workflow_invocation_uuid__ "301f995c718e11f0a03a7ced8d7042ad"
              chromInfo "/tmp/tmp2x7w0yy_/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
              cluster_file __identifier__
              dbkey "?"
              fasta_file __identifier__
      • Step 28: toolshed.g2.bx.psu.edu/repos/iuc/compose_text_param/compose_text_param/0.1.1:

        • step_state: scheduled

        • Jobs
          • Job 1:

            • Job state is ok

            Command Line:

            • cd ../; python _evaluate_expression_.py

            Exit Code:

            • 0

            Traceback:

            Job Parameters:

            • Job parameter Parameter value
              __input_ext "input"
              __workflow_invocation_uuid__ "301f995c718e11f0a03a7ced8d7042ad"
              chromInfo "/tmp/tmp2x7w0yy_/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
              components [{"__index__": 0, "param_type": {"__current_case__": 0, "component_value": "@SampleID:", "select_param_type": "text"}}, {"__index__": 1, "param_type": {"__current_case__": 0, "component_value": "sample_0", "select_param_type": "text"}}]
              dbkey "?"
      • Step 29: Unlabelled step:

        • step_state: new
      • Step 30: Unlabelled step:

        • step_state: new
      • Step 4: toolshed.g2.bx.psu.edu/repos/iuc/collection_element_identifiers/collection_element_identifiers/0.0.2:

        • step_state: scheduled

        • Jobs
          • Job 1:

            • Job state is ok

            Command Line:

            • mv '/tmp/tmp2x7w0yy_/job_working_directory/000/5/configs/tmphh5imykx' '/tmp/tmp2x7w0yy_/job_working_directory/000/5/outputs/dataset_701169d3-302b-411f-aa08-c33a04e4c462.dat'

            Exit Code:

            • 0

            Traceback:

            Job Parameters:

            • Job parameter Parameter value
              __input_ext "tabular"
              __workflow_invocation_uuid__ "301f995c718e11f0a03a7ced8d7042ad"
              chromInfo "/tmp/tmp2x7w0yy_/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
              dbkey "?"
              input_collection {"values": [{"id": 1, "src": "hdca"}]}
      • Step 31: toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_replace_in_column/9.3+galaxy1:

        • step_state: scheduled

        • Jobs
          • Job 1:

            • Job state is ok

            Command Line:

            • awk -v OFS="\t" -v FS="\t" --re-interval --sandbox '{ $1 = gensub( /@SampleID[^ ]*/, "@SampleID:sample_0", "g", $1 ) ; print $0 ; }' '/tmp/tmp2x7w0yy_/files/0/1/4/dataset_01406860-8b7f-4293-8c4d-c2378f9f118e.dat' > '/tmp/tmp2x7w0yy_/job_working_directory/000/24/outputs/dataset_c5634ecb-20ab-4e5b-a35f-190a11b5110b.dat'

            Exit Code:

            • 0

            Traceback:

            Job Parameters:

            • Job parameter Parameter value
              __input_ext "input"
              __workflow_invocation_uuid__ "301f995c718e11f0a03a7ced8d7042ad"
              chromInfo "/tmp/tmp2x7w0yy_/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
              dbkey "?"
              replacements [{"__index__": 0, "column": "1", "find_pattern": "@SampleID[^ ]*", "replace_pattern": "@SampleID:sample_0"}]
      • Step 32: Unlabelled step:

        • step_state: new
      • Step 33: Unlabelled step:

        • step_state: new
      • Step 34: Unlabelled step:

        • step_state: new
      • Step 35: Unlabelled step:

        • step_state: new
      • Step 36: Unlabelled step:

        • step_state: new
      • Step 37: Unlabelled step:

        • step_state: new
      • Step 38: toolshed.g2.bx.psu.edu/repos/artbio/concatenate_multiple_datasets/cat_multi_datasets/1.4.3:

        • step_state: scheduled

        • Jobs
          • Job 1:

            • Job state is ok

            Command Line:

            • printf "Concatenating files:\n" && printf "marine_pooled_gsa_trimmed..." && cat '/tmp/tmp2x7w0yy_/files/c/5/6/dataset_c5634ecb-20ab-4e5b-a35f-190a11b5110b.dat' >> '/tmp/tmp2x7w0yy_/job_working_directory/000/25/outputs/dataset_a56b1029-a7e5-4e5a-9974-1bb40c58a27c.dat' && printf "Done\n" && sleep 1

            Exit Code:

            • 0

            Standard Output:

            • Concatenating files:
              marine_pooled_gsa_trimmed...Done
              

            Traceback:

            Job Parameters:

            • Job parameter Parameter value
              __input_ext "tabular"
              __workflow_invocation_uuid__ "301f995c718e11f0a03a7ced8d7042ad"
              chromInfo "/tmp/tmp2x7w0yy_/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
              dataset_names false
              dbkey "?"
              global_condition {"__current_case__": 0, "input_type": "singles", "inputs": {"values": [{"id": 35, "src": "hdca"}]}}
              headers "0"
      • Step 39: Unlabelled step:

        • step_state: new
      • Step 40: Unlabelled step:

        • step_state: new
      • Step 5: toolshed.g2.bx.psu.edu/repos/iuc/concoct_cut_up_fasta/concoct_cut_up_fasta/1.1.0+galaxy2:

        • step_state: scheduled

        • Jobs
          • Job 1:

            • Job state is ok

            Command Line:

            • ln -s '/tmp/tmp2x7w0yy_/files/a/7/8/dataset_a78ce864-cb39-45bb-8333-7598bdc391cd.dat' 'input.fa' &&  cut_up_fasta.py 'input.fa' --chunk_size 10000 --overlap_size 0 --merge_last --bedfile '/tmp/tmp2x7w0yy_/job_working_directory/000/6/outputs/dataset_7b245253-9b6a-4745-a637-1fd349ebd1ce.dat' > '/tmp/tmp2x7w0yy_/job_working_directory/000/6/outputs/dataset_fada775e-8b21-4ffe-9804-318b7767765d.dat'

            Exit Code:

            • 0

            Traceback:

            Job Parameters:

            • Job parameter Parameter value
              __input_ext "input"
              __workflow_invocation_uuid__ "301f995c718e11f0a03a7ced8d7042ad"
              bedfile true
              chromInfo "/tmp/tmp2x7w0yy_/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
              chunk_size "10000"
              dbkey "?"
              input_fasta __identifier__
              merge_last true
              overlap_size "0"
      • Step 41: Unlabelled step:

        • step_state: new
      • Step 42: Unlabelled step:

        • step_state: new
      • Step 43: Unlabelled step:

        • step_state: new
      • Step 44: Unlabelled step:

        • step_state: new
      • Step 45: Unlabelled step:

        • step_state: new
      • Step 46: Unlabelled step:

        • step_state: new
      • Step 47: Unlabelled step:

        • step_state: new
      • Step 48: Unlabelled step:

        • step_state: new
      • Step 49: Unlabelled step:

        • step_state: new
      • Step 50: Unlabelled step:

        • step_state: new
      • Step 6: toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.5.3+galaxy1:

        • step_state: scheduled

        • Jobs
          • Job 1:

            • Job state is ok

            Command Line:

            • set -o | grep -q pipefail && set -o pipefail; bowtie2-build --threads ${GALAXY_SLOTS:-4} '/tmp/tmp2x7w0yy_/files/a/7/8/dataset_a78ce864-cb39-45bb-8333-7598bdc391cd.dat' genome && ln -s -f '/tmp/tmp2x7w0yy_/files/a/7/8/dataset_a78ce864-cb39-45bb-8333-7598bdc391cd.dat' genome.fa &&   ln -f -s '/tmp/tmp2x7w0yy_/files/b/5/e/dataset_b5ebe2a3-fa62-49c2-af16-ffb2b78c9d5d.dat' input_f.fastq &&  ln -f -s '/tmp/tmp2x7w0yy_/files/4/6/9/dataset_4692d1a6-e279-49cd-b64a-8836486b3a14.dat' input_r.fastq &&    THREADS=${GALAXY_SLOTS:-4} && if [ "$THREADS" -gt 1 ]; then (( THREADS-- )); fi &&   bowtie2  -p "$THREADS"  -x 'genome'   -1 'input_f.fastq' -2 'input_r.fastq'                2> >(tee '/tmp/tmp2x7w0yy_/job_working_directory/000/7/outputs/dataset_8acef0a5-5d85-4c26-8b82-5a2ce19a48cb.dat' >&2)  | samtools sort -l 0 -T "${TMPDIR:-.}" -O bam | samtools view --no-PG -O bam -@ ${GALAXY_SLOTS:-1} -o '/tmp/tmp2x7w0yy_/job_working_directory/000/7/outputs/dataset_30cfe3f8-0a8e-4de9-a75d-5241a62cbee0.dat'

            Exit Code:

            • 0

            Standard Error:

            • Building a SMALL index
              Renaming genome.3.bt2.tmp to genome.3.bt2
              Renaming genome.4.bt2.tmp to genome.4.bt2
              Renaming genome.1.bt2.tmp to genome.1.bt2
              Renaming genome.2.bt2.tmp to genome.2.bt2
              Renaming genome.rev.1.bt2.tmp to genome.rev.1.bt2
              Renaming genome.rev.2.bt2.tmp to genome.rev.2.bt2
              10000010 reads; of these:
                10000010 (100.00%) were paired; of these:
                  2353230 (23.53%) aligned concordantly 0 times
                  4683487 (46.83%) aligned concordantly exactly 1 time
                  2963293 (29.63%) aligned concordantly >1 times
                  ----
                  2353230 pairs aligned concordantly 0 times; of these:
                    7036 (0.30%) aligned discordantly 1 time
                  ----
                  2346194 pairs aligned 0 times concordantly or discordantly; of these:
                    4692388 mates make up the pairs; of these:
                      4581166 (97.63%) aligned 0 times
                      83684 (1.78%) aligned exactly 1 time
                      27538 (0.59%) aligned >1 times
              77.09% overall alignment rate
              [bam_sort_core] merging from 9 files and 1 in-memory blocks...
              

            Standard Output:

            • Settings:
                Output files: "genome.*.bt2"
                Line rate: 6 (line is 64 bytes)
                Lines per side: 1 (side is 64 bytes)
                Offset rate: 4 (one in 16)
                FTable chars: 10
                Strings: unpacked
                Max bucket size: default
                Max bucket size, sqrt multiplier: default
                Max bucket size, len divisor: 4
                Difference-cover sample period: 1024
                Endianness: little
                Actual local endianness: little
                Sanity checking: disabled
                Assertions: disabled
                Random seed: 0
                Sizeofs: void*:8, int:4, long:8, size_t:8
              Input files DNA, FASTA:
                /tmp/tmp2x7w0yy_/files/a/7/8/dataset_a78ce864-cb39-45bb-8333-7598bdc391cd.dat
              Reading reference sizes
                Time reading reference sizes: 00:00:17
              Calculating joined length
              Writing header
              Reserving space for joined string
              Joining reference sequences
                Time to join reference sequences: 00:00:13
              bmax according to bmaxDivN setting: 408958943
              Using parameters --bmax 306719208 --dcv 1024
                Doing ahead-of-time memory usage test
                Passed!  Constructing with these parameters: --bmax 306719208 --dcv 1024
              Constructing suffix-array element generator
              Building DifferenceCoverSample
                Building sPrime
                Building sPrimeOrder
                V-Sorting samples
                V-Sorting samples time: 00:00:39
                Allocating rank array
                Ranking v-sort output
                Ranking v-sort output time: 00:00:09
                Invoking Larsson-Sadakane on ranks
                Invoking Larsson-Sadakane on ranks time: 00:00:15
                Sanity-checking and returning
              Building samples
              Reserving space for 12 sample suffixes
              Generating random suffixes
              QSorting 12 sample offsets, eliminating duplicates
              QSorting sample offsets, eliminating duplicates time: 00:00:00
              Multikey QSorting 12 samples
                (Using difference cover)
                Multikey QSorting samples time: 00:00:00
              Calculating bucket sizes
              Splitting and merging
                Splitting and merging time: 00:00:00
              Split 1, merged 7; iterating...
              Splitting and merging
                Splitting and merging time: 00:00:00
              Split 2, merged 0; iterating...
              Splitting and merging
                Splitting and merging time: 00:00:00
              Split 1, merged 2; iterating...
              Splitting and merging
                Splitting and merging time: 00:00:00
              Avg bucket size: 2.04479e+08 (target: 306719207)
              Converting suffix-array elements to index image
              Allocating ftab, absorbFtab
              Entering Ebwt loop
              Getting block 1 of 8
                Reserving size (306719208) for bucket 1
                Calculating Z arrays for bucket 1
                Entering block accumulator loop for bucket 1:
                bucket 1: 10%
                bucket 1: 20%
                bucket 1: 30%
                bucket 1: 40%
                bucket 1: 50%
                bucket 1: 60%
                bucket 1: 70%
                bucket 1: 80%
                bucket 1: 90%
                bucket 1: 100%
                Sorting block of length 127045839 for bucket 1
                (Using difference cover)
                Sorting block time: 00:00:58
              Returning block of 127045840 for bucket 1
              Getting block 2 of 8
                Reserving size (306719208) for bucket 2
                Calculating Z arrays for bucket 2
                Entering block accumulator loop for bucket 2:
                bucket 2: 10%
                bucket 2: 20%
                bucket 2: 30%
                bucket 2: 40%
                bucket 2: 50%
                bucket 2: 60%
                bucket 2: 70%
                bucket 2: 80%
                bucket 2: 90%
                bucket 2: 100%
                Sorting block of length 302867108 for bucket 2
                (Using difference cover)
                Sorting block time: 00:02:15
              Returning block of 302867109 for bucket 2
              Getting block 3 of 8
                Reserving size (306719208) for bucket 3
                Calculating Z arrays for bucket 3
                Entering block accumulator loop for bucket 3:
                bucket 3: 10%
                bucket 3: 20%
                bucket 3: 30%
                bucket 3: 40%
                bucket 3: 50%
                bucket 3: 60%
                bucket 3: 70%
                bucket 3: 80%
                bucket 3: 90%
                bucket 3: 100%
                Sorting block of length 54967539 for bucket 3
                (Using difference cover)
                Sorting block time: 00:00:24
              Returning block of 54967540 for bucket 3
              Getting block 4 of 8
                Reserving size (306719208) for bucket 4
                Calculating Z arrays for bucket 4
                Entering block accumulator loop for bucket 4:
                bucket 4: 10%
                bucket 4: 20%
                bucket 4: 30%
                bucket 4: 40%
                bucket 4: 50%
                bucket 4: 60%
                bucket 4: 70%
                bucket 4: 80%
                bucket 4: 90%
                bucket 4: 100%
                Sorting block of length 252680552 for bucket 4
                (Using difference cover)
                Sorting block time: 00:01:40
              Returning block of 252680553 for bucket 4
              Getting block 5 of 8
                Reserving size (306719208) for bucket 5
                Calculating Z arrays for bucket 5
                Entering block accumulator loop for bucket 5:
                bucket 5: 10%
                bucket 5: 20%
                bucket 5: 30%
                bucket 5: 40%
                bucket 5: 50%
                bucket 5: 60%
                bucket 5: 70%
                bucket 5: 80%
                bucket 5: 90%
                bucket 5: 100%
                Sorting block of length 269322435 for bucket 5
                (Using difference cover)
                Sorting block time: 00:01:51
              Returning block of 269322436 for bucket 5
              Getting block 6 of 8
                Reserving size (306719208) for bucket 6
                Calculating Z arrays for bucket 6
                Entering block accumulator loop for bucket 6:
                bucket 6: 10%
                bucket 6: 20%
                bucket 6: 30%
                bucket 6: 40%
                bucket 6: 50%
                bucket 6: 60%
                bucket 6: 70%
                bucket 6: 80%
                bucket 6: 90%
                bucket 6: 100%
                Sorting block of length 248115557 for bucket 6
                (Using difference cover)
                Sorting block time: 00:01:44
              Returning block of 248115558 for bucket 6
              Getting block 7 of 8
                Reserving size (306719208) for bucket 7
                Calculating Z arrays for bucket 7
                Entering block accumulator loop for bucket 7:
                bucket 7: 10%
                bucket 7: 20%
                bucket 7: 30%
                bucket 7: 40%
                bucket 7: 50%
                bucket 7: 60%
                bucket 7: 70%
                bucket 7: 80%
                bucket 7: 90%
                bucket 7: 100%
                Sorting block of length 270511178 for bucket 7
                (Using difference cover)
                Sorting block time: 00:01:58
              Returning block of 270511179 for bucket 7
              Getting block 8 of 8
                Reserving size (306719208) for bucket 8
                Calculating Z arrays for bucket 8
                Entering block accumulator loop for bucket 8:
                bucket 8: 10%
                bucket 8: 20%
                bucket 8: 30%
                bucket 8: 40%
                bucket 8: 50%
                bucket 8: 60%
                bucket 8: 70%
                bucket 8: 80%
                bucket 8: 90%
                bucket 8: 100%
                Sorting block of length 110325558 for bucket 8
                (Using difference cover)
                Sorting block time: 00:00:49
              Returning block of 110325559 for bucket 8
              Exited Ebwt loop
              fchr[A]: 0
              fchr[C]: 426524502
              fchr[G]: 817883105
              fchr[T]: 1209183621
              fchr[$]: 1635835773
              Exiting Ebwt::buildToDisk()
              Returning from initFromVector
              Wrote 574367765 bytes to primary EBWT file: genome.1.bt2.tmp
              Wrote 408958948 bytes to secondary EBWT file: genome.2.bt2.tmp
              Re-opening _in1 and _in2 as input streams
              Returning from Ebwt constructor
              Headers:
                  len: 1635835773
                  bwtLen: 1635835774
                  sz: 408958944
                  bwtSz: 408958944
                  lineRate: 6
                  offRate: 4
                  offMask: 0xfffffff0
                  ftabChars: 10
                  eftabLen: 20
                  eftabSz: 80
                  ftabLen: 1048577
                  ftabSz: 4194308
                  offsLen: 102239736
                  offsSz: 408958944
                  lineSz: 64
                  sideSz: 64
                  sideBwtSz: 48
                  sideBwtLen: 192
                  numSides: 8519978
                  numLines: 8519978
                  ebwtTotLen: 545278592
                  ebwtTotSz: 545278592
                  color: 0
                  reverse: 0
              Total time for call to driver() for forward index: 00:21:39
              Reading reference sizes
                Time reading reference sizes: 00:00:11
              Calculating joined length
              Writing header
              Reserving space for joined string
              Joining reference sequences
                Time to join reference sequences: 00:00:06
                Time to reverse reference sequence: 00:00:02
              bmax according to bmaxDivN setting: 408958943
              Using parameters --bmax 306719208 --dcv 1024
                Doing ahead-of-time memory usage test
                Passed!  Constructing with these parameters: --bmax 306719208 --dcv 1024
              Constructing suffix-array element generator
              Building DifferenceCoverSample
                Building sPrime
                Building sPrimeOrder
                V-Sorting samples
                V-Sorting samples time: 00:00:26
                Allocating rank array
                Ranking v-sort output
                Ranking v-sort output time: 00:00:06
                Invoking Larsson-Sadakane on ranks
                Invoking Larsson-Sadakane on ranks time: 00:00:14
                Sanity-checking and returning
              Building samples
              Reserving space for 12 sample suffixes
              Generating random suffixes
              QSorting 12 sample offsets, eliminating duplicates
              QSorting sample offsets, eliminating duplicates time: 00:00:00
              Multikey QSorting 12 samples
                (Using difference cover)
                Multikey QSorting samples time: 00:00:00
              Calculating bucket sizes
              Splitting and merging
                Splitting and merging time: 00:00:00
              Split 1, merged 6; iterating...
              Splitting and merging
                Splitting and merging time: 00:00:00
              Avg bucket size: 2.33691e+08 (target: 306719207)
              Converting suffix-array elements to index image
              Allocating ftab, absorbFtab
              Entering Ebwt loop
              Getting block 1 of 7
                Reserving size (306719208) for bucket 1
                Calculating Z arrays for bucket 1
                Entering block accumulator loop for bucket 1:
                bucket 1: 10%
                bucket 1: 20%
                bucket 1: 30%
                bucket 1: 40%
                bucket 1: 50%
                bucket 1: 60%
                bucket 1: 70%
                bucket 1: 80%
                bucket 1: 90%
                bucket 1: 100%
                Sorting block of length 181991256 for bucket 1
                (Using difference cover)
                Sorting block time: 00:01:16
              Returning block of 181991257 for bucket 1
              Getting block 2 of 7
                Reserving size (306719208) for bucket 2
                Calculating Z arrays for bucket 2
                Entering block accumulator loop for bucket 2:
                bucket 2: 10%
                bucket 2: 20%
                bucket 2: 30%
                bucket 2: 40%
                bucket 2: 50%
                bucket 2: 60%
                bucket 2: 70%
                bucket 2: 80%
                bucket 2: 90%
                bucket 2: 100%
                Sorting block of length 163456318 for bucket 2
                (Using difference cover)
                Sorting block time: 00:01:04
              Returning block of 163456319 for bucket 2
              Getting block 3 of 7
                Reserving size (306719208) for bucket 3
                Calculating Z arrays for bucket 3
                Entering block accumulator loop for bucket 3:
                bucket 3: 10%
                bucket 3: 20%
                bucket 3: 30%
                bucket 3: 40%
                bucket 3: 50%
                bucket 3: 60%
                bucket 3: 70%
                bucket 3: 80%
                bucket 3: 90%
                bucket 3: 100%
                Sorting block of length 300365053 for bucket 3
                (Using difference cover)
                Sorting block time: 00:01:59
              Returning block of 300365054 for bucket 3
              Getting block 4 of 7
                Reserving size (306719208) for bucket 4
                Calculating Z arrays for bucket 4
                Entering block accumulator loop for bucket 4:
                bucket 4: 10%
                bucket 4: 20%
                bucket 4: 30%
                bucket 4: 40%
                bucket 4: 50%
                bucket 4: 60%
                bucket 4: 70%
                bucket 4: 80%
                bucket 4: 90%
                bucket 4: 100%
                Sorting block of length 252666651 for bucket 4
                (Using difference cover)
                Sorting block time: 00:01:36
              Returning block of 252666652 for bucket 4
              Getting block 5 of 7
                Reserving size (306719208) for bucket 5
                Calculating Z arrays for bucket 5
                Entering block accumulator loop for bucket 5:
                bucket 5: 10%
                bucket 5: 20%
                bucket 5: 30%
                bucket 5: 40%
                bucket 5: 50%
                bucket 5: 60%
                bucket 5: 70%
                bucket 5: 80%
                bucket 5: 90%
                bucket 5: 100%
                Sorting block of length 280318282 for bucket 5
                (Using difference cover)
                Sorting block time: 00:01:43
              Returning block of 280318283 for bucket 5
              Getting block 6 of 7
                Reserving size (306719208) for bucket 6
                Calculating Z arrays for bucket 6
                Entering block accumulator loop for bucket 6:
                bucket 6: 10%
                bucket 6: 20%
                bucket 6: 30%
                bucket 6: 40%
                bucket 6: 50%
                bucket 6: 60%
                bucket 6: 70%
                bucket 6: 80%
                bucket 6: 90%
                bucket 6: 100%
                Sorting block of length 214333332 for bucket 6
                (Using difference cover)
                Sorting block time: 00:01:30
              Returning block of 214333333 for bucket 6
              Getting block 7 of 7
                Reserving size (306719208) for bucket 7
                Calculating Z arrays for bucket 7
                Entering block accumulator loop for bucket 7:
                bucket 7: 10%
                bucket 7: 20%
                bucket 7: 30%
                bucket 7: 40%
                bucket 7: 50%
                bucket 7: 60%
                bucket 7: 70%
                bucket 7: 80%
                bucket 7: 90%
                bucket 7: 100%
                Sorting block of length 242704875 for bucket 7
                (Using difference cover)
                Sorting block time: 00:01:42
              Returning block of 242704876 for bucket 7
              Exited Ebwt loop
              fchr[A]: 0
              fchr[C]: 426524502
              fchr[G]: 817883105
              fchr[T]: 1209183621
              fchr[$]: 1635835773
              Exiting Ebwt::buildToDisk()
              Returning from initFromVector
              Wrote 574367765 bytes to primary EBWT file: genome.rev.1.bt2.tmp
              Wrote 408958948 bytes to secondary EBWT file: genome.rev.2.bt2.tmp
              Re-opening _in1 and _in2 as input streams
              Returning from Ebwt constructor
              Headers:
                  len: 1635835773
                  bwtLen: 1635835774
                  sz: 408958944
                  bwtSz: 408958944
                  lineRate: 6
                  offRate: 4
                  offMask: 0xfffffff0
                  ftabChars: 10
                  eftabLen: 20
                  eftabSz: 80
                  ftabLen: 1048577
                  ftabSz: 4194308
                  offsLen: 102239736
                  offsSz: 408958944
                  lineSz: 64
                  sideSz: 64
                  sideBwtSz: 48
                  sideBwtLen: 192
                  numSides: 8519978
                  numLines: 8519978
                  ebwtTotLen: 545278592
                  ebwtTotSz: 545278592
                  color: 0
                  reverse: 1
              Total time for backward call to driver() for mirror index: 00:18:16
              

            Traceback:

            Job Parameters:

            • Job parameter Parameter value
              __input_ext "input"
              __workflow_invocation_uuid__ "301f995c718e11f0a03a7ced8d7042ad"
              analysis_type {"__current_case__": 0, "analysis_type_selector": "simple", "presets": "no_presets"}
              chromInfo "/tmp/tmp2x7w0yy_/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
              dbkey "?"
              library {"__current_case__": 2, "aligned_file": false, "input_1": {"values": [{"id": 3, "src": "dce"}]}, "paired_options": {"__current_case__": 1, "paired_options_selector": "no"}, "type": "paired_collection", "unaligned_file": false}
              own_file __identifier__
              reference_genome {"__current_case__": 1, "own_file": {"values": [{"id": 2, "src": "dce"}]}, "source": "history"}
              rg {"__current_case__": 3, "rg_selector": "do_not_set"}
              sam_options {"__current_case__": 1, "sam_options_selector": "no"}
              save_mapping_stats true
      • Step 7: toolshed.g2.bx.psu.edu/repos/iuc/cami_amber_add/cami_amber_add/2.0.7+galaxy0:

        • step_state: scheduled

        • Jobs
          • Job 1:

            • Job state is ok

            Command Line:

            • ln -s '/tmp/tmp2x7w0yy_/files/f/a/4/dataset_fa49c4c9-e0dd-499c-adb9-9a0eb6551815.dat' 'marine_pooled_gsb_trimmed' && ln -s '/tmp/tmp2x7w0yy_/files/a/7/8/dataset_a78ce864-cb39-45bb-8333-7598bdc391cd.dat' 'marine_pooled_gsa_trimmed' &&  add_length_column.py -g 'marine_pooled_gsb_trimmed' -f 'marine_pooled_gsa_trimmed' > gold_standard_file.tsv

            Exit Code:

            • 0

            Traceback:

            Job Parameters:

            • Job parameter Parameter value
              __input_ext "input"
              __workflow_invocation_uuid__ "301f995c718e11f0a03a7ced8d7042ad"
              chromInfo "/tmp/tmp2x7w0yy_/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
              dbkey "?"
      • Step 8: toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_awk_tool/9.3+galaxy1:

        • step_state: scheduled

        • Jobs
          • Job 1:

            • Job state is ok

            Command Line:

            • env -i $(which awk) --sandbox -v FS='	' -v OFS='	' --re-interval -f '/tmp/tmp2x7w0yy_/job_working_directory/000/9/configs/tmpimpsyy1u' '/tmp/tmp2x7w0yy_/files/7/0/1/dataset_701169d3-302b-411f-aa08-c33a04e4c462.dat' > '/tmp/tmp2x7w0yy_/job_working_directory/000/9/outputs/dataset_cfad8382-7eb2-4cdc-ab44-19a67970139a.dat'

            Exit Code:

            • 0

            Traceback:

            Job Parameters:

            • Job parameter Parameter value
              __input_ext "input"
              __workflow_invocation_uuid__ "301f995c718e11f0a03a7ced8d7042ad"
              chromInfo "/tmp/tmp2x7w0yy_/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
              code "{gsub( $0 ,\"sample_\" (NR-1)); print}"
              dbkey "?"
      • Step 9: toolshed.g2.bx.psu.edu/repos/devteam/samtools_sort/samtools_sort/2.0.5:

        • step_state: scheduled

        • Jobs
          • Job 1:

            • Job state is error

            Command Line:

            • addthreads=${GALAXY_SLOTS:-1} && (( addthreads-- )) &&   addmemory=${GALAXY_MEMORY_MB_PER_SLOT:-768} && ((addmemory=addmemory*75/100)) &&  samtools sort -@ $addthreads -m $addmemory"M"   -O bam -T "${TMPDIR:-.}" '/tmp/tmp2x7w0yy_/files/3/0/c/dataset_30cfe3f8-0a8e-4de9-a75d-5241a62cbee0.dat' > '/tmp/tmp2x7w0yy_/job_working_directory/000/10/outputs/dataset_ee8a9a78-10f5-47b9-8323-a9169157fb39.dat'

            Exit Code:

            • 1

            Standard Error:

            • [bam_sort_core] merging from 12 files and 1 in-memory blocks...
              [E::bgzf_flush] File write failed (wrong size)
              samtools sort: failed writing to "-": No space left on device
              [E::bgzf_close] File write failed
              

            Traceback:

            Job Parameters:

            • Job parameter Parameter value
              __input_ext "input"
              __workflow_invocation_uuid__ "301f995c718e11f0a03a7ced8d7042ad"
              chromInfo "/tmp/tmp2x7w0yy_/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
              dbkey "?"
              input1 __identifier__
              minhash false
              prim_key_cond {"__current_case__": 0, "prim_key_select": ""}
      • Step 10: __RELABEL_FROM_FILE__:

        • step_state: scheduled

        • Jobs
          • Job 1:

            • Job state is ok

            Traceback:

            Job Parameters:

            • Job parameter Parameter value
              __workflow_invocation_uuid__ "301f995c718e11f0a03a7ced8d7042ad"
              how {"__current_case__": 0, "how_select": "txt", "labels": {"values": [{"id": 11, "src": "hda"}]}, "strict": false}
              input {"values": [{"id": 1, "src": "hdca"}]}
    • Other invocation details
      • history_id

        • 14e0d202dd346f1d
      • history_state

        • error
      • invocation_id

        • 14e0d202dd346f1d
      • invocation_state

        • ready
      • workflow_id

        • 14e0d202dd346f1d

@SantaMcCloud
Copy link
Author

SantaMcCloud commented Aug 28, 2025

@mvdbeek should i either downgrade this tool or should i wait till it is updated because of this error msg?

Unexpected HTTP status code: 400: {"err_msg":"Workflow was not invoked; the following required tools are not installed: toolshed.g2.bx.psu.edu/repos/devteam/samtools_sort/samtools_sort/2.0.5 (version 2.0.5)","err_code":0}

mvdbeek added a commit to mvdbeek/usegalaxy-tools that referenced this pull request Sep 1, 2025
mvdbeek added a commit to galaxyproject/usegalaxy-tools that referenced this pull request Sep 1, 2025
@SantaMcCloud
Copy link
Author

[2025-09-01 00:54:30] ERROR - An error occurred while running DIAMOND. Check log file: output/temporary_files/diamond_result.log

I would make sure your workflow runs on usegalaxy.org as a first step, we use the same reference data via cvmfs. If that doesn't give you a better idea I'm afraid you'll have to modify the binette tool to print the log file to stderr if it fails.

Thank for the information! I did run Binette on .org with the test data generated in .eu and i know the problem now. The memory allocated to Binette has to be increased and after it should run. Also thank you or adding the Tools needed for the Wf to .org.

@SantaMcCloud
Copy link
Author

I have send a bug report since Binette got allocated ~4 GB but is should have way more because in the tvp it says 32 GB is max for Binette https://github.com/galaxyproject/tpv-shared-database/blob/main/tools.yml#L1172

here is the test history just to be sure: https://usegalaxy.org/u/santinof/h/unnamed-history

@mvdbeek
Copy link
Member

mvdbeek commented Sep 1, 2025

we use a pinned tpv version, i've updated the allocation now, you can retry

@SantaMcCloud
Copy link
Author

SantaMcCloud commented Sep 1, 2025

we use a pinned tpv version, i've updated the allocation now, you can retry

did rerun it and now there was no error in this case.

Should i still wait till the test on .orgwas without error after the tool did isntalled or you want to restart the test in here in GitHub?

@mvdbeek
Copy link
Member

mvdbeek commented Sep 1, 2025

Can you run the whole workflow against usegalaxy.org ? I see the peak memory was at 15.4GB, which is very tight.

@SantaMcCloud
Copy link
Author

yes i will run it against org on this case

@SantaMcCloud
Copy link
Author

I did run the WF in .org and there was no error in this case. I will restart the test to see if there is any other problem now. If there is to much data produce or so i can try to cut the test data even more but im not sure how far i can cut it down.

@mvdbeek
Copy link
Member

mvdbeek commented Sep 4, 2025

Can you share the invocation please ?

@SantaMcCloud
Copy link
Author

Can you share the invocation please ?

https://usegalaxy.org/workflows/invocations/e370c44912ff7cbb

here it is

@galaxyproject galaxyproject deleted a comment from github-actions bot Sep 5, 2025
@galaxyproject galaxyproject deleted a comment from github-actions bot Sep 5, 2025
@galaxyproject galaxyproject deleted a comment from github-actions bot Sep 5, 2025
@galaxyproject galaxyproject deleted a comment from github-actions bot Sep 5, 2025
@galaxyproject galaxyproject deleted a comment from github-actions bot Sep 5, 2025
@galaxyproject galaxyproject deleted a comment from github-actions bot Sep 5, 2025
@galaxyproject galaxyproject deleted a comment from github-actions bot Sep 5, 2025
@galaxyproject galaxyproject deleted a comment from github-actions bot Sep 5, 2025
@galaxyproject galaxyproject deleted a comment from github-actions bot Sep 5, 2025
@galaxyproject galaxyproject deleted a comment from github-actions bot Sep 5, 2025
@galaxyproject galaxyproject deleted a comment from github-actions bot Sep 5, 2025
@mvdbeek
Copy link
Member

mvdbeek commented Sep 5, 2025

I think you'll need to get the memory for binette down a little, reducing the input data might do the trick.

Screenshot 2025-09-05 at 11 08 43

You can see an overview under metrics for your invocation:

Screenshot 2025-09-05 at 11 12 08

@SantaMcCloud
Copy link
Author

SantaMcCloud commented Sep 5, 2025

I think you'll need to get the memory for binette down a little, reducing the input data might do the trick.

Screenshot 2025-09-05 at 11 08 43 You can see an overview under metrics for your invocation: Screenshot 2025-09-05 at 11 12 08

i did see it thank you! but how what is the Limit so i can have a look at it to be under it for All tools

@mvdbeek
Copy link
Member

mvdbeek commented Sep 5, 2025

@SantaMcCloud
Copy link
Author

@SantaMcCloud
Copy link
Author

@mvdbeek now it should be fine! If there is anything which i shoudl change just let me know it!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants