interactEM

interactEM is an interactive flow-based programming platform for streaming experimental data to High-Performance Computing (HPC). Users can wire together containerized operators through a web frontend and deploy them on both HPC and edge resources.

Demo video

demo.mp4

Features

• Interactive Web Frontend: Create data pipelines with a React-based frontend.
• Containerized Operators: Uses container technology to ensure consistent execution environments.
• Data Streaming: Operators connect and data flows between them over network with point-to-point communication.

Running locally

Prerequisites

• Docker Desktop and Podman Desktop (recommended for easiest setup)
• NATS Tools: Command-line utilities for interacting with NATS

Configuration

1. Create a .env and setup environment variables

   Create a .env

   cd interactEM
   cp .env.example .env

   Add Github and personal token information in .env file. You will need to get this from your github account.

   GITHUB_USERNAME=your_github_username
   GITHUB_TOKEN=your_personal_token

2. Generate NATS Authentication Files
   Generate auth.conf for the NATS cluster and various .creds and .nk files for NATS authentication.

   cd conf/nats-conf
   ./generate-auth-jwt.sh

Building Docker Images and starting services

1. Build the Required Images

   docker/bake.sh

2. Bring up Services

   docker compose up --force-recreate --remove-orphans --build -d

3. Login

   Go to http://localhost:5173/ in your browser, and you should see a login page. The credentials for the local version are in ./.env.example.

   [email protected]
   FIRST_SUPERUSER_PASSWORD=changethis

Launching an agent

For operators to launch, you need to startup an agent process. See backend/agent/README.md for instructions on how to do this.

License

This project is licensed under the LBNL BSD-3 License - see the LICENSE file for details.