dxCompiler 2.4.8

• Fixes issue where an optional variable inside a conditional could have an inferred type of T??, which is illegal and results in a runtime error
• Specifies operating system when selecting optimal instance type
• Fixes issue with references between output variables
• Uses an output stage if there is an output parameter with a literal value (DNAnexus output parameters do not support default values)
• Fixes issue where compiling with -execTree [pretty|json] did not print the workflow tree
• Adds support for specifying app metadata in the hints section in WDL development version
• All compile-time calls to findDataObjects now search the context project for any file with no project specified
• All runtime dxAPI.describeFilesBulk calls, which invoke the platform's system.findDataObjects, now search in the job/analysis workspace first before searching the source project(s).
  • The call is used to search and describe input files for the analysis and to search and describe output files during the outputs reorganization of the analysis (in its default implementation, not the customized one). If the files cannot be found in the workspace container they are looked for in the project that qualifies file ID (e.g. in project_A if file_B's path is project_A:file_B)
• Better handles insufficient permissions when requesting instance type price list

Dependency updates

dxApi 0.0.6

• Adds option to DxApi.describeFilesBulk to search first in the workspace container
• DxApi.resolveDataObject now searches in the current workspace and/or project if the project is not specified explicitly. The call is used to find one data object at a time, it's not used in bulk resolution.
• Refactors DxFindDataObjects to use separate DxFindDataObjectsConstraints class for specifying constraints
• Uses the currently select project ID as the workspace ID when not running in a job
• Better handles insufficient permissions when requesting instance type price list

dxCompiler 2.4.7

• Fixes issue with using struct types in workflow outputs
• Fixes issue with array with optional item type

dxCompiler 2.4.6

• Fixes regression in WDL code generator where long expressions within placeholders are line-wrapped incorrectly

dxCompiler 2.4.5

• An applet that contains multiple scatter or conditional blocks will now have a name that is the concatenation of all the block names
• Fixes multiple issues with WDL code generator
• Fixes issue with referencing struct fields/call outputs in declarations within nested blocks

Warning: we discovered a regression in this release that affects tasks with long command blocks. Please wait until the next release if this applies to your WDLs.

dxCompiler 2.4.4

• Escapes WDL strings in generated code
• Fixes issues with using expressions in placeholder option values
• Fixes error when evaluating array element access for an optional value
• Fixes localization of files in identically named folders in different projects
• Fixes localization of files with the same name in the same folder

dxCompiler 2.4.3

• Fixes an issue where a file input from an different project than where the workflow is compiled is localized to an invalid path
• File downloads (including Docker images) no longer fail when retried after a previous failure
• Adds the -waitOnUpload compiler option, which causes all file uploads to block until they complete
• Fixes an issue where tasks with outputs that are collections of files (e.g. Array[File]) are compiled with an incorrect default input value
• Fixes an issue where using a field of a struct as a call input causes a runtime error, e.g.

  struct MyStruct {
    String s
  }
  workflow wf {
    input {
      MyStruct my
    }
    call mytask { input: s = my.s }
  }
  ``

dxCompiler 2.4.2

• -imports now works correctly - import statements can reference files that are relative to an import directory rather than the main document
• Coercion from Object to Map is now allowed - this enables the result of read_json to be assigned to a variable of type Map
• Strings with escape sequences are now processed correctly - to preserve escape sequences in a string, they need to be "double-escaped"
  • For example, to pass a regular expression containing a tab character to the second argument of the sub function:

    String s1 = "hello\tBob"
    String s2 = sub(s1, "\\t", " ")

  • Another common example is passing a read group to bwa mem. Whether the string is defined in WDL, in a JSON input file, or on the command line, it needs to be "double-escaped":

    String rg = "@RG\\tID:${sample_name}\\tSM:${sample_name}\\tLB:${sample_name}\\tPL:ILLUMINA"
    
    command <<<
    bwa -R "~{rg}" ...
    >>>

• Fixes an issue where indentation is stripped out of some commands
• The default reorg applet now polls the analysis until it is fully updated before proceeding. For custom reorg applets, see the updated code in the example.
• Fixes an issue with nested scatters that reference private variables from outer scopes, e.g.

  workflow wf {
    input {
      Array[String] samples
      Array[File] files
    } 
    scatter (i in range(length(samples))) {
      scatter (j in range(length(files))) {
        call mytask { 
          input: sample = samples[i], file = files[j]
        }
      }  
    }
  }  

dxCompiler 2.4.1

• Fixes AWS ECR issues: bundles AWS CLI with executor rather than installing at runtime

dxCompiler 2.4.0

• Adds -useManifests option to generate applets and workflows whose inputs and outputs are manifest files
• Fixes issue with using both streaming and non-streaming file inputs in the same task
• Fixes issue with scatter as the first element of a workflow
• Updates to wdlTools 0.12.7, which provides compatibility for some non-complaint syntax allowed by Cromwell
• Fixes -separateOutputs for scatter jobs

dxCompiler 2.3.1

• Fixes issue with call arguments that access fields of private declarations
• Fixes issue with passing null to optional call parameters
• Fixes common applet naming issue with complex nested workflows