Merge branch 'develop' of github.com:unity-sds/unity-sps into 419-reduce-ops-costs

Author: nikki-t

CHANGELOG.md

@@ -9,7 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 - SPS Version 3.1.0 (new)
 - OGC API Version 2.1.0 (unchanged)
-- OGC Python Client Version 2.0.1 (unchanged)
+- OGC Python Client Version 2.1.0 (unchanged)
 
 ## Overview
 
@@ -77,6 +77,7 @@ Additionally, there are a number of small adjustments:
   - [[New Feature]: Enhancements to CWL Modularized DAG](https://github.com/unity-sds/unity-sps/issues/276)
   - [[New Feature]: Modularize the ASIPS workflows](https://github.com/unity-sds/unity-sps/issues/320)
   - [[New Feature]: Increase test coverage for CWL Modular DAG](https://github.com/unity-sds/unity-sps/issues/394)
+  - [[Bug]: Determine if the use of XCOM to hold successful features is causing failures when running DAGs](https://github.com/unity-sds/unity-sps/issues/439)
 - EPIC: Demonstrate Space Use Case
   - [[New Feature]: Demonstrate scalability of Space Use Case](https://github.com/unity-sds/unity-sps/issues/285)
 

airflow/docker/cwl/docker_cwl_entrypoint_modular.sh

@@ -166,14 +166,15 @@ failed_features=$(cat $failed_features_file | jq '.')
 echo Failed features:
 echo $failed_features | jq '.'
 
-# Save catalog.json in a location where it will be picked up by Airflow XCOM mechanism
-mkdir -p /airflow/xcom/
-cp $successful_features_file /airflow/xcom/return.json
+# # Save catalog.json in a location where it will be picked up by Airflow XCOM mechanism
+# mkdir -p /airflow/xcom/
+# cp $successful_features_file /airflow/xcom/return.json
 
 # Optionally, save the requested output file to a location
 # where it will be picked up by the Airflow XCOM mechanism
 # Note: the content of the file MUST be valid JSON or XCOM will fail.
 if [ ! -z "${json_output}" -a "${json_output}" != " " ]; then
+  mkdir -p /airflow/xcom/
   cp ${json_output} /airflow/xcom/return.json
 fi