AI Agent Guidelines for Obelisk Components

This document provides guidelines for AI agents working on this repository.

Obelisk is a deterministic workflow engine for durable execution. Learn more at obeli.sk.

Development Environment

Using Nix (Recommended)

This project uses Nix flakes to manage all dependencies including Rust, cargo, and other tools.

Installing Nix with flakes enabled:

# Using the Determinate Systems installer (recommended)
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install

# Or manually enable flakes in an existing Nix installation:
echo "experimental-features = nix-command flakes" | sudo tee -a /etc/nix/nix.conf
sudo systemctl restart nix-daemon.service

Configure Garnix cache for faster builds:

cat << 'EOF' | sudo tee -a /etc/nix/nix.conf
extra-substituters = https://cache.garnix.io
extra-trusted-public-keys = cache.garnix.io:CTFPyKSLcx5RMJKfLo5EEPUObbA78b0YQ2DTCJXqr9g=
EOF
sudo systemctl restart nix-daemon.service

Enter the development shell:

nix develop

This provides all necessary tools: Rust toolchain, wasmtime, wasm-tools, etc.

Without Nix

If Nix is unavailable, install tools matching versions in dev-deps.txt:

# Check required versions
cat dev-deps.txt

# Install Rust toolchain per rust-toolchain.toml
rustup default 1.92
rustup target add wasm32-wasip2

# Install wasmtime (check dev-deps.txt for version)
curl https://wasmtime.dev/install.sh -sSf | bash

# Install obelisk CLI (prefer binstall for faster installation)
cargo binstall obelisk --version <version-from-dev-deps.txt>
# Or if binstall is not available:
cargo install obelisk --version <version-from-dev-deps.txt>

Project Structure

Naming Conventions

Activities (components responsible for side effects): Must start with activity-
- Example: fly/activity-fly-http, openai/activity-openai-responses
Webhooks (HTTP endpoint handlers): Must start with webhook-
- Example: fly/webhook-fly-secrets-updater

Activity Components

Activities are WASIp2 components that perform side effects. Each activity lives in its own directory:

<category>/activity-<name>/
├── .cargo/config.toml    # Build target and test runner config
├── .envrc-example        # Example env vars (if activity requires them)
├── Cargo.toml            # Uses workspace dependencies
├── README.md
├── obelisk-local.toml    # Local Obelisk config (path to wasm)
├── src/
│   └── lib.rs
└── wit/
    ├── impl.wit                              # World definition (any:any)
    ├── <package-name>/                       # Main exported interface
    │   └── <interface>.wit
    └── deps/
        └── <package-name> -> ../<package-name>  # Symlink to main package

If the activity requires environment variables (API keys, tokens, etc.), include a .envrc-example file showing the required variables and document them in the README.

WIT File Organization

Main package directory (wit/<package-name>/): Contains the interface definition. This is the canonical source, readable by humans.
deps symlink (wit/deps/<package-name>): A symlink to the main package. Required for wit-bindgen to resolve dependencies.

impl.wit: Defines the world that exports your interface:

package any:any;

world any {
    export obelisk-components:<your-package>/<interface>@0.1.0;
}

Cargo.toml

Use workspace dependencies:

[package]
name = "activity-<name>"
description = "Activity that does X."

authors.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
rust-version.workspace = true
version.workspace = true

[lib]
crate-type = ["cdylib"]

[dependencies]
serde.workspace = true
serde_json.workspace = true
wit-bindgen.workspace = true
wstd.workspace = true

[build-dependencies]
anyhow.workspace = true
wit-bindgen-rust.workspace = true

WIT Bindings Generation

Prefer using build.rs over the wit_bindgen::generate! macro for generating WIT bindings. This provides better IDE support and more control over code generation.

build.rs:

use anyhow::Result;
use wit_bindgen_rust::Opts;

fn main() -> Result<()> {
    Opts {
        generate_all: true,
        ..Default::default()
    }
    .build()
    .generate_to_out_dir(None)?;

    Ok(())
}

src/lib.rs:

mod generated {
    #![allow(clippy::empty_line_after_outer_attr)]
    include!(concat!(env!("OUT_DIR"), "/any.rs"));
}

use generated::export;
use generated::exports::obelisk_components::your_package::api::*;

struct Component;
export!(Component with_types_in generated);

impl Guest for Component {
    // ...
}

.cargo/config.toml

Configure wasm32-wasip2 as default target and wasmtime as test runner:

[build]
target = "wasm32-wasip2"

[target.wasm32-wasip2]
runner = "wasmtime run -Scli -Shttp --env YOUR_API_KEY --env TEST_YOUR_API_KEY"

Error Handling for Obelisk

Critical: All activity errors MUST use a variant type with an execution-failed case:

variant api-error {
    /// Required for Obelisk - indicates timeout or trap in last retry
    execution-failed,
    /// Your other error cases
    configuration-error(string),
    request-failed(string),
    api-error(error-details),
}

The execution-failed variant is used by Obelisk to handle:

Execution timeouts
Component traps/panics
Retry exhaustion

Do not use result<T, string> for activity functions - use structured error variants.

Testing

Test Structure

Tests run in wasmtime via the .cargo/config.toml runner:

#[cfg(test)]
mod tests {
    use super::*;

    const ENV_API_KEY: &str = "API_KEY";

    fn set_up() {
        // Copy TEST_* env var to the actual env var
        let test_token = std::env::var(format!("TEST_{ENV_API_KEY}"))
            .expect("TEST_API_KEY must be set");
        unsafe { std::env::set_var(ENV_API_KEY, test_token) };
    }

    #[test]
    fn unit_test_no_external_deps() {
        // Runs without setup
    }

    #[test]
    #[ignore]  // Integration tests are ignored by default
    fn integration_test_requires_api_key() {
        set_up();
        // Test with real API
    }
}

Running Tests

# Unit tests only (from activity directory)
cargo test

# Integration tests (requires API keys)
export TEST_YOUR_API_KEY="..."
cargo test -- --ignored --nocapture

# All tests
cargo test -- --include-ignored

HTTP Requests with wstd

Use wstd for async HTTP in WASIp2 components:

use wstd::http::{Body, Client, Request};
use wstd::runtime::block_on;

async fn make_request() -> Result<String, Error> {
    let req = Request::post("https://api.example.com/endpoint")
        .header("Authorization", format!("Bearer {}", api_key))
        .header("Content-Type", "application/json")
        .body(Body::from(json_body))?;
    
    let resp = Client::new().send(req).await?;
    let body = resp.into_body().contents().await?;
    Ok(String::from_utf8_lossy(&body).to_string())
}

// In your Guest impl:
fn my_function() -> Result<String, ApiError> {
    block_on(make_request())
}

Code Style

Run cargo fmt before committing
Run cargo clippy --target wasm32-wasip2 and fix all warnings
Follow existing patterns in the repository

Obelisk Configuration

obelisk-local.toml

Each activity should have an obelisk-local.toml for local development:

api.listening_addr = "127.0.0.1:5005"
webui.listening_addr = "127.0.0.1:8080"

[[activity_wasm]]
name = "activity_your_name"
location.path = "${OBELISK_TOML_DIR}/../../target/wasm32-wasip2/release/activity_your_name.wasm"
exec.lock_expiry.seconds = 5
env_vars = ["YOUR_API_KEY"]
forward_stdout = "stderr"
forward_stderr = "stderr"

Run locally with:

cargo build --release
obelisk server run --config ./obelisk-local.toml

Submit an execution (in another terminal):

obelisk execution submit --follow \
  <package>:<interface>/<function-name> \
  -- '<json-param-1>' '<json-param-2>' ...

Parameters are JSON values with snake_case field names (matching serde serialization). Use --follow to see the execution output.

obelisk-oci.toml

The obelisk-oci.toml file is auto-generated by a GitHub Action that is triggered manually. Do not create or edit this file manually.

Adding a New Activity

Create directory: <category>/activity-<name>/
Add to workspace Cargo.toml members
Create .cargo/config.toml with target and runner
Create Cargo.toml using workspace dependencies (include build-dependencies)
Create build.rs for WIT binding generation
Create WIT interface in wit/<package-name>/<interface>.wit
Create wit/impl.wit exporting your interface
Create symlink: wit/deps/<package-name> -> ../<package-name>
Implement in src/lib.rs (use generated module pattern)
Add tests
Add README.md (document required env vars if any)
Add obelisk-local.toml for local development
Add .envrc-example if the activity requires environment variables

End-to-End Testing

E2E tests verify the components work correctly against real (or mock) APIs.

Running E2E Tests

# Run all e2e tests with mock servers
./scripts/test-e2e.sh all

# Run specific component tests
./scripts/test-e2e.sh openai    # Uses mock server
./scripts/test-e2e.sh sendgrid  # Uses mock server  
./scripts/test-e2e.sh postmark  # Uses mock server
./scripts/test-e2e.sh http      # Uses mock server
./scripts/test-e2e.sh github    # Uses real API (requires TEST_GITHUB_TOKEN)
./scripts/test-e2e.sh fly       # Uses real API (requires TEST_FLY_API_TOKEN)

Mock Servers

Mock servers are provided in scripts/mocks/ for testing without real API credentials:

mock-openai-server.py - Mocks OpenAI Responses API
mock-sendgrid-server.py - Mocks SendGrid Mail Send API
mock-postmark-server.py - Mocks Postmark Email API
mock-http-server.py - Generic HTTP echo server

Environment Variables for E2E Tests

Mock-based tests (no real credentials needed):

Tests automatically set up mock environment variables

Real API tests:

TEST_GITHUB_TOKEN - GitHub Personal Access Token
TEST_GITHUB_LOGIN - GitHub username to test against (default: obeli-sk)
TEST_GITHUB_REPO - GitHub repo URL to test against
TEST_FLY_API_TOKEN - Fly.io API token
TEST_FLY_ORG - Fly.io organization slug (default: personal)

Configurable API Base URLs

Components support configurable API base URLs for testing:

OPENAI_API_BASE_URL - Override OpenAI API endpoint
SENDGRID_API_URL - Override SendGrid API endpoint
POSTMARK_API_URL - Override Postmark API endpoint

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AI Agent Guidelines for Obelisk Components

Development Environment

Using Nix (Recommended)

Without Nix

Project Structure

Naming Conventions

Activity Components

WIT File Organization

Cargo.toml

WIT Bindings Generation

.cargo/config.toml

Error Handling for Obelisk

Testing

Test Structure

Running Tests

HTTP Requests with wstd

Code Style

Obelisk Configuration

obelisk-local.toml

obelisk-oci.toml

Adding a New Activity

End-to-End Testing

Running E2E Tests

Mock Servers

Environment Variables for E2E Tests

Configurable API Base URLs

FilesExpand file tree

AGENTS.md

Latest commit

History

AGENTS.md

File metadata and controls

AI Agent Guidelines for Obelisk Components

Development Environment

Using Nix (Recommended)

Without Nix

Project Structure

Naming Conventions

Activity Components

WIT File Organization

Cargo.toml

WIT Bindings Generation

.cargo/config.toml

Error Handling for Obelisk

Testing

Test Structure

Running Tests

HTTP Requests with wstd

Code Style

Obelisk Configuration

obelisk-local.toml

obelisk-oci.toml

Adding a New Activity

End-to-End Testing

Running E2E Tests

Mock Servers

Environment Variables for E2E Tests

Configurable API Base URLs