Skip to content

Conversation

@grallewellyn
Copy link
Contributor

Purpose

This purpose of this PR is to publish a DAG that uses the Docker image running docker in docker to make packaging an algorithm: https://unity-sds.gitbook.io/docs/mdps-overview/tutorials/algorithms-and-algorithm-development/packaging-an-algorithm easier for users
I published the Docker image that we are using in the DAG to our jplmdps account: https://hub.docker.com/r/jplmdps/unity-app-gen

My code for the DAG and Dockerfile is here: unity-sds/unity-app-generator#31

Proposed Changes

  • [ADD] Functioning AppGen DAG

Issues

Relevant issues:
unity-sds/unity-project-management#228
#275
unity-sds/unity-app-generator#21

Testing

Copy link
Collaborator

@LucaCinquini LucaCinquini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@grallewellyn : if you redeploy your SPS with the latest Docker image version (2.6.0 - as specified in the Airflow tfvars file) you will be able to simplify this DAG by importing POD_LABEL, DEFAULT_LOG_LEVEL and EC2_TYPES from the plugin file.

Additionally, there might not be a need to ask the user to select a disk size and EC2 type - should those be fixed to reasonable values that work for all app-generator jobs? Let's talk to @mcduffie about it.

@LucaCinquini LucaCinquini merged commit 033e325 into develop Apr 16, 2025
2 checks passed
@LucaCinquini LucaCinquini deleted the appgen-dag branch April 16, 2025 20:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants