Merge pull request #1788 from ComparativeGenomicsToolkit/toil-5.3

prep release v3.0.0

Author: glennhickey

BIN-INSTALL.md

@@ -6,21 +6,17 @@ pre-compile binary, static linked distribution.
 ## Extracting
 If you have not already extract the distribution and cd into the cactus directory:
 ```
-tar -xzf cactus-bin-v2.9.9.tar.gz
-cd cactus-bin-v2.9.9
+tar -xzf cactus-bin-v3.0.0.tar.gz
+cd cactus-bin-v3.0.0
 ```
 
-## GPU Patch:
-
-When using GPU support since v2.9.4, please make sure to pass in the patched config file with `--configFile` (download the patch from the [releases](https://github.com/ComparativeGenomicsToolkit/cactus/releases) page).
-
 ## Setup
 
 To build a python virtualenv and activate, do the following steps. This requires Python version >= 3.9 (so Ubuntu 18.04 users should use `-p python3.9` below):
 ```
-virtualenv -p python3 venv-cactus-v2.9.9
-printf "export PATH=$(pwd)/bin:\$PATH\nexport PYTHONPATH=$(pwd)/lib:\$PYTHONPATH\nexport LD_LIBRARY_PATH=$(pwd)/lib:\$LD_LIBRARY_PATH\n" >> venv-cactus-v2.9.9/bin/activate
-source venv-cactus-v2.9.9/bin/activate
+virtualenv -p python3 venv-cactus-v3.0.0
+printf "export PATH=$(pwd)/bin:\$PATH\nexport PYTHONPATH=$(pwd)/lib:\$PYTHONPATH\nexport LD_LIBRARY_PATH=$(pwd)/lib:\$LD_LIBRARY_PATH\n" >> venv-cactus-v3.0.0/bin/activate
+source venv-cactus-v3.0.0/bin/activate
 python3 -m pip install -U setuptools pip wheel
 python3 -m pip install -U .
 python3 -m pip install -U -r ./toil-requirement.txt

ReleaseNotes.md

@@ -1,4 +1,21 @@
-# Release 2.9.9 2025-09-11
+# Release 3.0.0 2025-09-20
+
+This release includes several significant patches and updates
+
+- FastGA support is now more robust and performant
+- MAF normalization and de-duplication bugs fixed
+- `cactus-hal2maf` interface generalized
+- Toil updated to v9.1.0, which performs much better on Slurm among other improvements
+- Contig naming bug fixed in minigraph (`.sv.gfa.gz`) output
+- `cactus-update-prepare` can now rename genomes
+- `--delEdgeFilter` option added to clip out big edges in pangenoems
+- More realistic ancestral contig sizes, allows kegalign to be run without special config
+- `cactus-pangenome --mgSplit` option to run `minigraph` steps independently for each chromosome
+- `cactus-prepare --script` option added to make a bash script to run pipeline step by step
+- Default outgroup strategy changed to `GreedyLeavesPreference` (and config XML outgroup toggle fixed)
+- Regression causing `cactus-pangenome --vcf` (even without `--vcfwave`) to fail when vcflib not installed fixed. 
+
+# Release 2.9.9 2025-06-11
 
 This release adds FastGA support, along with some bug fixes
 

doc/progressive.md

@@ -277,12 +277,12 @@ The Cactus Docker image contains everything you need to run Cactus (python envir
 
 ```
 wget -q https://raw.githubusercontent.com/ComparativeGenomicsToolkit/cactus/master/examples/evolverMammals.txt -O evolverMammals.txt
-docker run -v $(pwd):/data --rm -it quay.io/comparative-genomics-toolkit/cactus:v2.9.9 cactus /data/jobStore /data/evolverMammals.txt /data/evolverMammals.hal
+docker run -v $(pwd):/data --rm -it quay.io/comparative-genomics-toolkit/cactus:v3.0.0 cactus /data/jobStore /data/evolverMammals.txt /data/evolverMammals.hal
 ```
 
 Or you can proceed interactively by running
 ```
-docker run -v $(pwd):/data --rm -it quay.io/comparative-genomics-toolkit/cactus:v2.9.9 bash
+docker run -v $(pwd):/data --rm -it quay.io/comparative-genomics-toolkit/cactus:v3.0.0 bash
 cactus /data/jobStore /data/evolverMammals.txt /data/evolverMammals.hal
 
 ```
@@ -461,8 +461,6 @@ Cactus supports incrementally updating existing alignments to add, remove, or up
 
 ## GPU Acceleration
 
-**Important** Since [v2.9.4](https://github.com/ComparativeGenomicsToolkit/cactus/releases/tag/v2.9.4), you must use a patched config file (via `--configFile config-v2.9.9-keg-patch.xml`) to prevent cactus for constructing ancestors with contigs that are too big for KegAlign to read.  This file must be passed into all `cactus` and/or `cactus-align` commands.  This is a temporary work-around and should be fixed properly in a release in the near future (download the patch from the [releases](https://github.com/ComparativeGenomicsToolkit/cactus/releases) page).  
-
 [KegAlign](https://github.com/galaxyproject/KegAlign), a GPU-accelerated version of lastz, can be used in the "blast" phase to speed up the runtime considerably, provided the right hardware is available. Unlike lastz, the input sequences do not need to be chunked before running KegAlign, so it also reduces the number of Toil jobs substantially.  The [GPU-enabled Docker releases](https://github.com/ComparativeGenomicsToolkit/cactus/releases) have KegAlign turned on by default and require no extra options from the user.  Otherwise, it is possible to [manually install it](https://github.com/galaxyproject/KegAlign#-installation) and then enable it in `cactus` using the `--gpu` command line option. One effective way of ensuring that only GPU-enabled parts of the workflow are run on GPU nodes is on Terra with `cactus-prepare --gpu --wdl` (see above example).
 
 By default `--gpu` will give all available GPUs to each KegAlign job. This can be tuned by passing in a numeric value, ex `--gpu 8` to assign 8 GPUs to each KegAlign job.  In non-single-machine batch systems, it is mandatory to set an exact value with `--gpu`.  
@@ -478,8 +476,6 @@ Please [cite KegAlign](https://doi.org/10.1101/2024.09.02.610839).
 
 ## FastGA
 
-**WARNING This is a new, experimental and not well tested feature. Try it out if you are interested in benchmarking FastGA, but some further study is required before it should be considered as a replacement for the lastz or gpu modes.**
-
 [FastGA](https://github.com/thegenemyers/FASTGA) is a new, extremely fast whole genome pairwise aligner.  You can use it instead of `lastz` in cactus via the (very experimental) `--fastga` option. `FastGA` only aligns up to about 20% divergence, so any gaps in the `FastGA` will be re-aligned with `lastz`.  You can disable the `lastz` fallback by setting `<blast fastga_fill="0">` in the config XML.
 
 FastGA was designed for use with up to 8 cores. If you use more, there probably won't be much speedup (it's fast enough with 8).  If you use many more, then it will probably crash.  In cactus versions up to and including v2.9.9, it will default to using all the cores on your system, so it's crucial to specify `--lastzCores 8` alongside `--fastga`.  In later releases, it defaults to 8 cores so it is not necessary to specifiy `--lastzCores`.  

setup.py

@@ -24,7 +24,7 @@ def run(self):
 
 setup(
     name = "Cactus",
-    version = "2.9.9",
+    version = "3.0.0",
     author = "Benedict Paten",
     package_dir = {'': 'src'},
     packages = find_packages(where='src'),

src/cactus/shared/common.py

@@ -320,7 +320,7 @@ def getDockerTag(gpu=False):
         return "latest"    
     else:
         # must be manually kept current with each release        
-        return 'v2.9.9' + ('-gpu' if gpu else '')
+        return 'v3.0.0' + ('-gpu' if gpu else '')
 
 def getDockerImage(gpu=False):
     """Get fully specified Docker image name."""