Graph-Choreographer

Graph-Choreographer is an orchestration tool for deploying and executing serverless workflows across the computing continuum, from edge devices (e.g., Jetson, Raspberry Pi), to cloud VMs nodes.
It provides performance-aware and energy-aware scheduling, integrates tightly with Prometheus, Kepler, and Grafana, and supports workflow execution through OpenFaaS and Argo Workflows. The tool acts as the central coordinator for graph operations (BGOs), DAG workflows, and distributed execution pipelines within the Graph-Massivizer ecosystem.

Core Functionalities

• Cluster-aware scheduling across multiple K8s clusters via custom inter/intra schedulers
• Serverless workflow execution with OpenFaaS and Argo Workflows
• Monitoring integration using Prometheus, Kepler, and Grafana
• Energy-aware orchestration based on real-time metrics and DAG structure
• Support for heterogeneous testbeds, including ARM edge devices, cloud VMs, and HPC nodes

Architecture Overview

As shown in the following diagram, Graph-Choreographer has four main components:

• Resource partitioning handles the resource diversity and complex network structures across cloud, fog, and edge resources in the hierarchical computing continuum. In particular:

  • It retrieves the data monitored from the computing continuum, such as processing cores, memory, storage, network bandwidth, number and type of deployed functions, or the providers’ function concurrency limitations;
  • It employs the Graph-Greenifier interface to identify the BGO resource and energy utilization patterns;
  • It applies multilayered graph clustering techniques to group similar computing continuum instances concerning operational delay, sustainability, and energy consumption and sends these clusters to the function scheduling component.

• Function scheduling processes partitioning information, parses BGOs requirements received from Graph-Greenifier, and runs a scheduling optimization model that establishes constraints considering computing continuum utilization, resource limitation, and BGOs dependencies and requirements to minimize execution time, cost, and energy consumption.

• Function provisioning identifies the instance with the highest available resources for deploying new BGOs following the function scheduling decisions. Subsequently, it communicates with the execution engine component to execute the newly deployed BGOs.

• Execution engine considers inputs of the function scheduling and function provisioning components. Consequently, it triggers the providers’ invokers to execute BGOs and their necessary libraries on the computing continuum instances.

Setup Requirments

Graph-Choreographer depends on a complete Kubernetes-based cloud–edge environment with monitoring, workflow management systems, serverless execution, and scheduling integration. The following requirements must be met before deploying the tool.

• Kubernetes ≥ 1.24
• kubectl
• Helm 3.x
• containerd/Docker/CRI-O runtime
• OpenFaaS
• Argo Workflows
• Prometheus, Grafana, cAdvisor, PowerJoular, Kepler deployed on the cluster
• Python 3.8.x with the following dependencies:
  • networkx
  • numpy
  • pyyaml
  • kubernetes
  • requests