dxCompiler 2.8.2

• Fixes issue where task without runtime section succeeds even when command block results in a failure code
• Forwards command output to the job log rather than buffering it until after command completion
• Updates code to compile with JDK11
• Updates build environment to JDK11, Scala 2.13.7, and SBT 1.5.7

Dependency updates

dxCommon 0.11.0

• Adds SysUtils.runCommand, which exposes options for how to handle stdin/stdout/stderr.
• Adds Paths.BaseEvalPaths.isLocal attribute to differentiate local from remote paths.
• Breaking removes SysUtils.execScript. Use runCommand with the script path as the argument instead.
• Adds validation of path characters to FileUtils.getUriScheme

dxFileAccessProtocols 0.5.3

• Handles ResourceNotFoundException in DxFileSource.exists

wdlTools 0.17.6

• Fixes stderr() function - previously it was returning the file for stdout

dxCompiler 2.8.1

• Excludes apps from bundledDepends
• Localizes files declared in WDL task private variables
• Respects runtime definition in native task stub
• Fixes archiving of executables - tags them as "dxCompilerArchived" rather than renaming them
• Fixes error when parsing a field name with multiple stage-* prefixes (specifically with stage number >= 10)
• Logs full output of command execution

Dependency updates

dxCommon 0.10.1

• Adds PosixPath class for working with POSIX-style paths
• Changes Paths to use PosixPath rather than java.nio.Path
• Prettifies truncated log messages

dxApi 0.13.0

• Adds DxApi.addTags method
• Fixes DxFindDataObjects when used with tags constraint
• Handles record results in DxFindDataObjects
• Adds systemRequirements to DxWorkflowStageDesc

dxFileAccessProtocols 0.5.2

• Uses PosixPath rather than java.nio.Path for manipulating remote paths

dxCompiler 2.8.0

• Indicates whether static instance type selection was used in workflow description annotations and metadata
• Supports cloning workflows between projects, a prerequisite for publishing global workflows
• Fixes dxda manifest downloads for tasks
• Affected by a bug for workflows that include a native platform app via dxni. Bug fixed in v2.8.1.

dxCompiler 2.7.2

• Uses manifest files for large subjob inputs
• Uses dxda to bulk-download manifest files
• Increases number of retries when downloading single manifest files
• Fixes error when parsing a field name with multiple stage-* prefixes
• Allows file-to-string coercion for WDL inputs
• Affected by a bug for workflows that include a native platform app via dxni. Bug fixed in v2.8.1.

Dependency updates

dxApi 0.12.0

• Enables retryLimit to be set for DxApi.uploadFile and DxApi.downloadFile

dxCompiler 2.7.1

• Fixes issue where jobs fail due to out-of-disk error due to excessive logging
• Adds option to specify native app information in runtime section (or hints for WDL 2.0)
• Fixes regression where default instance type was overridden when calling a native app
• Affected by a bug for workflows that include a native platform app via dxni. Bug fixed in v2.8.1.

Dependency updates

dxCommon 0.9.0

• Adds option to Logger.trace* to show beginning and/or end of log when limiting trace length

dxApi 0.11.0

• Fixes DxApi.resolveApp to handle app name with with version (e.g. bwa_mem/1.0.0)
• Adds version field to DxAppDescribe

wdlTools 0.17.4

• Fixes eval.Meta.get to respect override values

dxCompiler 2.7.0

• Handles empty scatters in WDL workflows
• Reserved parameters are now placed in the "Reserved for dxCompiler" parameter group (only affects the display of the app/workflow in the UI)
• Enables other metadata (title, description, version, etc.) to be set via extras.json
• When using manifests, passes any expression values from the helper applet to the called applet workflow so they are added to the output manifest
• When an applet calls executables, adds executables to bundledDepends to support cloning of workflows
• Executor creates hard- rather than soft-links in the input directory, so that linked files are accessible from within containers
• Fixes bug where default input values were not overridden for task inside subworkflow
• Adds support for specifying native app(let) in runtime section
• Fixes some type conversion bugs related to CWL Any type
• Affected by a bug for workflows that include a native platform app via dxni. Bug fixed in v2.8.1.
• Logs entire contents of WDL command at runtime
• Fixes a bug where a default values of subworkflow were not overridden by provided value, as reported in this issue.

Warning: we discovered a regression in this release that may cause tasks to fail with out-of-disk errors due to excessive logging. Please update to 2.7.1 or later.

Dependency updates

dxApi 0.10.1

• Fixes resolveProject to handle container- objects
• Improves error message when API call fails due to connection error

cwlScala 0.7.1

• Handles error when listing folder during path value finalization

wdlTools 0.17.2

• Fixes infinite loop when calling wdlTools.eval.Runtime.contains with "docker" or "container"

dxCompiler 2.6.0

• Implements Directory support for CWL and for WDL development/2.0
• Implements CWL secondaryFiles support
• Implements CWL workflow support
• Breaking Change: the inputs folder is now modified to be read-only for non-streaming files (this has always been the case for streaming files). This means that tasks that create files in the inputs folder will no longer work. For example, a task that creates an index file in place for an input BAM file should be changed from:

  command <<<
  samtools index ~{mybam}
  >>>

  to

  command <<<
  mkdir bams
  ln -s ~{mybam} bams/~{basename(mybam)}
  samtools index bams/~{basename(mybam)}
  >>>

Dependency updates

wdlTools 0.17.1

• Fixes parsing of placeholder options in draft-2 and 1.0 such that default and sep are no longer treated as reserved words

cwlScala 0.7.0

• Breaking Change: Sink.linkMerge is now Optional
• Introduces ParserResult class, which is returned from all Parser.parse* methods
• For packed workflows, parses out $schemas and $namespaces
• Updates to latest cwljava, which fixes several parsing errors
• Fixes CwlType.flatten to correctly handle duplicate types
• Treats scatter sources as identifiers
• Automatically renames the main process if its name collides with another process
• Fixes evalution of compound parameter references
• Set CommandLineTool.successCodes to Set(0) if not specified
• Fixes deserialization of optional fields
• Uses the source file name as the process name when processing a $graph where the top-level element ID is 'main'
• Fixes evaluation for Directory-type values with listings
• Fixes parsing of LoadListingEnum values
• Adds option to Parser.parseFile and Parser.parseString to specify that the CWL file is in "packed" form
• Updates to latest cwljava, which fixes parsing of anonymous schemas in packed documents
• Correctly handles identifiers with namespaces from imported documents
• Fixes error when trying to finalize a File value without location or path
• Updates dxCommon to 0.2.15-SNAPSHOT
• Uses FileSource.listing to determine directory listing during finalization
• Updates to dxCommon 0.7.0
• Breaking change: schema types now have id: Option[Identifier] rather than name: Option[String]
• Parser can now handle $graph style CWL documents
• Adds dependency on dxCommon library
• Improves finalization of file values
• Fixes coercion of StringValue to CwlEnum
• other bugfixes
• Adds EvaluatorContext.createInputs to create an EvaluatorContext from input values
• Performs "finalization" of input values (setting of missing attributes on File and Directory values) when using EvaluatorContext.createInputs or EvaluatorContext.createStaticInputs
• Parser bugfixes
• Incorporate cwljava/39, which fixes workflow parsing issues
• Allow duplicate Requirements/Hints and specify priority rules
• Breaking change: added new CwlMulti type and removed all uses of Vector[CwlType]

dxCompiler 2.5.0

• Docker image dependencies that are DNAnexus platform files are included in applet's bundledDepends
• Adds -instanceTypeSelection compiler option to allow disabling compile-type instance type selection
• Adds -defaultInstanceType option
• Adds support for new London region, aws:eu-west-2-g
• Fixes -projectWideReuse
• Fixes evaluation of outputs when one declaration depends on another

Dependency updates

dxCommon 0.8.0

• Adds getTargetDir methods to LocalizationDisambiguator
• Fixes use of localizationDir together with force in SafeLocalizationDisambiguator
• Adds FileSource.resolveDirectory as a separate method from resolve

dxApi 0.10.0

• Removes price-based selection of instance types in favor of rank-based selection
• Fixes parsing of non-file-type default values that are reference-type links
• Fixes parsing of parameter defaults/suggestions/choices that are of type Hash

dxFileAccessProtocols 0.5.0

• Implements resolveDirectory method

wdlTools 0.17.0

• Breaking Change Eval.applyMap is changed to Eval.applyAll and takes a Vector rather than Map argument. This is done to ensure the expressions are evaluated in order in case there are dependencies between them.
• Fixes parsing of runtime.returnCodes

dxCompiler 2.4.10

• Fix issue with using File declarations in scatter/conditional blocks
• Allow scattering on an empty array
• Update dxda version

dxCompiler 2.4.9

• Files that are generated by calling functions in worklfow expressions are now uploaded to the platform
• dxni now makes optional any input parameter for which the native app(let) has a default value
• Jobs now fail on out-of-disk errors
• Updated dxda and dxfuse to latest versions
• Updated dxCommon and dxApi dependencies