diff --git a/docs/.index b/docs/.index index e355bfe..03f3e36 100644 --- a/docs/.index +++ b/docs/.index @@ -12,4 +12,5 @@ nav: - Agent Workflow Server: agws - Agent Manifest: manifest - How-To Guides: how-to-guides + - Glossary: glossary.md - How to Contribute: contributing.md diff --git a/docs/glossary.md b/docs/glossary.md new file mode 100644 index 0000000..404f95b --- /dev/null +++ b/docs/glossary.md @@ -0,0 +1,349 @@ +# SLIM Messaging Glossary + +This glossary defines technical terms referenced across the SLIM messaging documentation: core concepts, naming, sessions, routing, configuration, security, integrations (SLIMRPC, A2A, MCP), and deployment tooling. Terms are alphabetized. + +--- + +## A + +
+ +- **Ack (Acknowledgment)** + + --- + + A confirmation—explicit or implicit—that a sent message was delivered or processed. In reliable modes, missing acks trigger retries until `max_retries` is reached or a timeout fires. + +- **Anycast** + + --- + + Routing mode where a message addressed to a 3-component name (`org/namespace/service`) is delivered to exactly one currently reachable instance. Each message may choose a different instance. Used in the discovery phase of point-to-point and group sessions. + +
+ +--- + +## C + +
+ +- **Channel (Group Channel)** + + --- + + Logical group namespace: the first three SLIM name components (`org/namespace/service`) without the client hash. All joined participants receive each message. Also referred to as a Group. + +- **Channel Moderator** + + --- + + Manager of a group channel that invites/removes participants and coordinates MLS state. + +- **Client (Application Client)** + + --- + + Runtime endpoint connected to a SLIM node. Identified by full four-component name: `org/namespace/service/clientHash`. + +- **Client Instance ID (Client Hash)** + + --- + + Fourth name component, derived from identity material (e.g., hash of public key). Ensures unique, routable unicast identity per instance. + +- **Connection (Routing Connection)** + + --- + + Configured link (endpoint + parameters) enabling a SLIM node to route traffic to another SLIM node. + +- **Controller (SLIM Controller)** + + --- + + Central orchestration service offering northbound (administrative) and southbound (node) gRPC interfaces for configuration, topology, and group/channel operations. + +
+ +--- + +## D + +
+ +- **Data Plane (Messaging Layer)** + + --- + + Operational pipeline for message routing, delivery, encryption (MLS), and session state, implemented in SLIM nodes. It can be used synonymously with "SLIM Node". + +- **Discovery (Service Discovery)** + + --- + + Initial interaction where packets can be sent to any available instance. This is used at the beginning of a point-to-point session and when a new invite is sent in a group session. + +
+ +--- + +## E + +
+ +- **Endpoint** + + --- + + Host:port (and scheme) on which a server listens or a client connects. Note that in the configuration the client needs to specify the protocol (e.g. http://localhost:46357), but not the server (e.g. localhost:46357) + +
+ +--- + +## G + +
+ +- **Group** + + --- + + Set of participants joined to a channel. Shares message distribution and optionally an MLS security context. See also: Channel. + +
+ +--- + +## I + +
+ +- **Identity** + + --- + + Cryptographic or token-based representation of a workload (shared secret, JWT, SPIFFE SVID) determining trust and naming uniqueness. + +- **Invitation (Group Invitation)** + + --- + + Control message enabling a client to join an existing group. + +
+ +--- + +## L + +
+ +- **Local Name** + + --- + + The fully qualified SLIM name representing the current application endpoint. See Client (Application Client) + +
+ +--- + +## M + +
+ +- **Max Retries (Session Config)** + + --- + + Upper limit on retransmission attempts for a single message lacking timely acknowledgment. + +- **MLS (Message Layer Security)** + + --- + + End-to-end group key agreement and encryption protocol (RFC 9420) providing confidentiality and integrity beyond hop-level TLS termination. + +- **Moderator** + + --- + + Alias of Channel Moderator. + +- **Group Session** + + --- + + Session type enabling many-to-many distribution; every message is delivered to all participants in the channel. + +
+ +--- + +## N + +
+ +- **Name (SLIM Name)** + + --- + + Structured routing identifier: `organization/namespace/service/clientHash` (full) or the first three components for channel addressing. The first component, `organization`, usually represents the top-level administrative or tenant boundary; the second component, `namespace`, can be used to encode environment, region, or an organizational partition; the third component, `service`, specifies the service exposed by the client. When multiple instances of the same service are deployed (such as several pods in a Kubernetes cluster), these first three components remain the same for all instances. The last component, `clientHash`, is generated by SLIM from the client's identity material (e.g., a hash of the public key) and uniquely identifies each specific client instance. + +- **Node (SLIM Node)** + + --- + + Runtime process implementing the data plane: message routing, session handling, MLS operations, optional control endpoints. + +
+ +--- + +## P + +
+ +- **Participant (Group Participant)** + + --- + + Client that has accepted an invitation and joined a group, receiving all channel traffic. + +
+ +--- + +## R + +
+ +- **Retry** + + --- + + Re-attempt of sending a message after a timeout window, capped by `max_retries`. + +- **Route (Routing Entry)** + + --- + + Controller-managed mapping directing messages for a particular name (or prefix) through a specific connection. + +
+ +--- + +## S + +
+ +- **Session** + + --- + + Abstraction for interaction context (Point-to-Point, Group). Manages encryption (if MLS enabled), retries, and sequencing. + +- **Session Configuration** + + --- + + Per-session parameters selected at creation (e.g., mode, `mls_enabled`, metadata). In Python bindings: `PySessionConfiguration.PointToPoint(...)`, etc. + +- **Session Layer** + + --- + + Middleware that abstracts encryption, invitations, routing resolution, and reliability, offering simple send/receive primitives. + +- **Shared Secret Identity** + + --- + + Simplest identity bootstrap (common secret) for demos/tests before stronger mechanisms (e.g., JWT) are deployed. + +- **Slimctl** + + --- + + Command-line tool managing routes, connections, subscriptions, and nodes via Controller or direct node endpoints. + +- **SLIM (Secure Low-Latency Interactive Messaging)** + + --- + + Framework delivering secure, scalable, low-latency messaging with unified naming, session semantics, encryption, and extensibility. + +- **SLIMA2A** + + --- + + Integration of A2A agent protocol over SLIM using SLIMRPC-generated stubs. + +- **SLIM Controller** + + --- + + See Controller. + +- **SLIMRPC** + + --- + + Protobuf-based RPC framework operating over SLIM transport (unary and streaming patterns). Analogous to gRPC but using SLIM naming and channels. + +- **SLIMRPC Compiler (protoc-slimrpc-plugin)** + + --- + + Protoc plugin generating Python stubs and servicers for SLIMRPC services defined in `.proto` files. + +- **Subscription** + + --- + + Binding between a routed name/prefix and a connection so messages destined to that name traverse the appropriate path. + +- **Publish** + + --- + + Operation that sends a message on a session (context: point-to-point/group). May await acknowledgment depending on reliability mode. In Python bindings: `session.publish(...)` or `session.publish_to(ctx, ...)`. + +- **Message Context** + + --- + + Metadata accompanying a received message (e.g., routing info). Used to reply (`publish_to`) preserving addressing. + +
+ +--- + +## T + +
+ +- **Timeout (Request / Session Timeout)** + + --- + + Upper bound for waiting on acknowledgments or RPC responses before retry/failure escalation. + +
+ +--- + +## U + +
+ +- **Point-to-Point Session** + + --- + + Session mode for communication with a specific client instance. Ensures message affinity to a single endpoint. + +
diff --git a/docs/messaging/.index b/docs/messaging/.index index 5678e96..1923892 100644 --- a/docs/messaging/.index +++ b/docs/messaging/.index @@ -3,14 +3,17 @@ nav: - Getting Started with SLIM: - Getting Started: slim-howto.md - Configuration Reference: slim-data-plane-config.md - - SLIM Messaging Layer: slim-data-plane.md + - SLIM Messaging Layer: + - Overview: slim-data-plane.md + - SLIM Sessions: slim-session.md + - SLIM Authentication: slim-authentication.md - SLIM Controller: slim-controller.md - - SLIM Group Management: slim-group.md - - SLIM Group Communication Tutorial: slim-group-tutorial.md + - SLIM Groups: + - Group Creation and Management: slim-group.md + - Group Communication Tutorial: slim-group-tutorial.md - SLIM Integrations: - SLIMRPC: - Overview: slim-rpc.md - Protoc Plugin: slim-slimrpc-compiler.md - SLIM A2A: slim-a2a.md - MCP over SLIM: slim-mcp.md - diff --git a/docs/messaging/slim-a2a.md b/docs/messaging/slim-a2a.md index 3b6beb6..db9feb0 100644 --- a/docs/messaging/slim-a2a.md +++ b/docs/messaging/slim-a2a.md @@ -8,15 +8,15 @@ SLIMRPC is a framework that enables Protocol Buffers (protobuf) Remote Procedure To compile a protobuf file and generate the clients and service stub you can use the [SLIMRPC compiler](./slim-slimrpc-compiler.md). This works in a similar way to the protoc compiler. -For SLIM A2A we compiled the [a2a.proto](https://github.com/a2aproject/A2A/blob/main/specification/grpc/a2a.proto) file using the SLIM RPC compiler. The generated code is in [a2a_pb2_slimrpc.py](https://github.com/agntcy/slim/blob/main/data-plane/python/integrations/slima2a/slima2a/types/a2a_pb2_slimrpc.py). +For SLIM A2A we compiled the [a2a.proto](https://github.com/a2aproject/A2A/blob/main/specification/grpc/a2a.proto) file using the SLIM RPC compiler. The generated code is in [a2a_pb2_slimrpc.py](https://github.com/agntcy/slim/tree/slim-v0.6.0/data-plane/python/integrations/slima2a/examples/travel_planner_agent). ## How to use SLIMA2A -Using SLIMA2A is very similar to using the standard A2A implementation. As a reference example here we use the [travel planner agent](https://github.com/a2aproject/a2a-samples/tree/main/samples/python/agents/travel_planner_agent) available on the A2A samples repository. The version adapted to use SLIM A2A can be found in [travel_planner_agent](https://github.com/agntcy/slim/tree/main/data-plane/python/integrations/slima2a/examples/travel_planner_agent) folder. In the following section, we highlight and explain the key difference between the standard and SLIM A2A implementations. +Using SLIMA2A is very similar to using the standard A2A implementation. As a reference example here we use the [travel planner agent](https://github.com/a2aproject/a2a-samples/tree/main/samples/python/agents/travel_planner_agent) available on the A2A samples repository. The version adapted to use SLIM A2A can be found in [travel_planner_agent](https://github.com/agntcy/slim/tree/slim-v0.6.0/data-plane/python/integrations/slima2a/examples/travel_planner_agent) folder. In the following section, we highlight and explain the key difference between the standard and SLIM A2A implementations. ### Travel Planner: Server -In this section we highlight the main differences between the SLIM A2A [server](https://github.com/agntcy/slim/blob/main/data-plane/python/integrations/slima2a/examples/travel_planner_agent/server.py) implementation with respect to the original implementation in the A2A repository. +In this section we highlight the main differences between the SLIM A2A [server](https://github.com/agntcy/slim/blob/slim-v0.6.0/data-plane/python/integrations/slima2a/examples/travel_planner_agent/server.py) implementation with respect to the original implementation in the A2A repository. 1. Import the SLIMRPC package. ```python @@ -76,7 +76,7 @@ Your A2A server is now ready to run on SLIM. ### Travel Planner: Client -These are the main differences between the [client](https://github.com/agntcy/slim/blob/main/data-plane/python/integrations/slima2a/examples/travel_planner_agent/client.py) using SLIM A2A and the standard one. +These are the main differences between the [client](https://github.com/agntcy/slim/blob/slim-v0.6.0/data-plane/python/integrations/slima2a/examples/travel_planner_agent/client.py) using SLIM A2A and the standard one. 1. Create a channel. This requires a configuration that is similar to the server ```python diff --git a/docs/messaging/slim-authentication.md b/docs/messaging/slim-authentication.md new file mode 100644 index 0000000..f4dd90a --- /dev/null +++ b/docs/messaging/slim-authentication.md @@ -0,0 +1,398 @@ +# Identity Management + +## Overview + +Each SLIM client needs to have a valid identity. In the [SLIM Group +Communication Tutorial](slim-group-tutorial.md), we used a simple shared secret +to quickly set up identities for the clients. In a real-world scenario, you +would typically use a more secure method, such as tokens or certificates, to +authenticate clients and establish their identities. + +SLIM supports JWT (JSON Web Tokens) for identity management. Tokens can come +from an external identity provider or can be generated by the SLIM nodes +directly if you provide the necessary private key for signing the tokens and +public key for verification. Check the [Identity +Test](https://github.com/agntcy/slim/blob/slim-v0.6.0/data-plane/python/bindings/tests/test_identity.py) +for an example of how to use JWT tokens with SLIM if you have your own keys. + +If you are running your SLIM clients in a Kubernetes environment, using +[SPIRE](https://spiffe.io/) is a very common approach to give an identity +to each client. SPIRE provides a way to issue +[SPIFFE IDs](https://spiffe.io/docs/latest/spiffe-about/spiffe-concepts/#spiffe-id) to +workloads, in the form of JWT tokens, which SLIM can then use to authenticate +clients. This allows for secure and scalable identity management in distributed +systems. + +## Example: Using SPIRE with SLIM in Kubernetes (SPIRE / JWT) + +SLIM integrates well with SPIRE, as it allows you to use the JWT tokens +generated by SPIRE as client identities, and at the same time it can verify +these tokens using the key bundle provided by SPIRE. + +This section shows how to use SPIRE with SLIM to manage client identities. The following topics are covered: + +* Creating a local KIND cluster (with an in-cluster image registry). +* Installing SPIRE (server and agents). +* Building and pushing SLIM images to the local registry. +* Deploying the SLIM node (control and rendezvous components). +* Deploying two distinct SLIM client workloads, each with its own ServiceAccount (and thus its own SPIFFE ID). +* Running the point-to-point example using JWT-based authentication derived from SPIRE. + +If you already have a Kubernetes cluster or an existing SPIRE deployment, you +can adapt only the relevant subsections. + +This tutorial is based on the [SLIM examples](https://github.com/agntcy/slim/blob/slim-v0.6.0/data-plane/python/bindings/examples). + +### Prerequisites + +The following prerequisites are required: + +- [Docker](https://docs.docker.com/get-docker/) +- [kubectl](https://kubernetes.io/docs/tasks/tools/) +- [Helm](https://helm.sh/docs/intro/install/) +- [KIND](https://kind.sigs.k8s.io/docs/user/quick-start/) +- [git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) + +Clone the SLIM repository if you haven't already: + +```bash +git clone https://github.com/agntcy/slim.git && cd slim/data-plane/python/bindings/examples +``` + +### Creating a KIND Cluster with a Local Image Registry + +The helper script below provisions a KIND cluster and configures a local +registry (localhost:5001) that the cluster’s container runtime can pull from: + +```bash +curl -L https://kind.sigs.k8s.io/examples/kind-with-registry.sh | sh +``` + +### Installing SPIRE + +To install SPIRE, you need to install the server, CRDs, and agents: + +```bash +helm upgrade --install \ + -n spire-server \ + spire-crds spire-crds \ + --repo https://spiffe.github.io/helm-charts-hardened/ \ + --create-namespace + +helm upgrade --install \ + -n spire-server \ + spire spire --repo \ + https://spiffe.github.io/helm-charts-hardened/ +``` + +Wait for the SPIRE components to become ready: + +```bash +kubectl get pods -n spire-server +``` + +All pods should reach Running/READY status before proceeding. + +### SPIFFE ID Strategy + +The default SPIRE server Helm chart installs a Cluster SPIFFE ID controller +object (`spire-server-spire-default`) that issues workload identities following +the pattern: + +`spiffe://domain.test/ns//sa/` + +We rely on this object by default. If you need more granular issuance (specific label +selectors, different trust domain, etc.), consult the [ClusterSPIFFEID +documentation](https://github.com/spiffe/spire-controller-manager/blob/main/docs/clusterspiffeid-crd.md). + +### Building SLIM Images (node and examples) + +You can use pre-built images if available; here we build and push fresh ones to +the local registry: + +```bash +REPO_ROOT=$(git rev-parse --show-toplevel) +pushd "${REPO_ROOT}" +IMAGE_REPO=localhost:5001 docker bake slim && docker push localhost:5001/slim:latest +IMAGE_REPO=localhost:5001 docker bake bindings-examples && docker push localhost:5001/bindings-examples:latest +popd +``` + +### Deploying the SLIM Node + +```bash +REPO_ROOT=$(git rev-parse --show-toplevel) +pushd "${REPO_ROOT}/charts" +helm install \ + --create-namespace \ + -n slim \ + slim ./slim \ + --set slim.image.repository=localhost:5001/slim \ + --set slim.image.tag=latest +``` + +Confirm the pod is running: + +```bash +kubectl get pods -n slim +``` + +### Deploying Client Configuration (ConfigMap) + +We first provide a config for `spiffe-helper`, which retrieves SVIDs or JWTs from +the SPIRE agent and writes them to disk. The key fields are: + +- `agent_address`: Path to the SPIRE agent API socket. +- `cert_dir`: Where artifacts (cert, key, bundles, or JWTs) are written. +- `jwt_svids`: Audience and output filename for requested JWT SVIDs. +- `daemon_mode = true`: Run continuously to renew materials. + +```bash +kubectl apply -f - <` @@ -273,11 +315,11 @@ List routes on a SLIM instance: Add a route to the SLIM instance: -`slimctl route add via --node-id=` +`slimctl route add via --node-id=` Delete a route from the SLIM instance: -`slimctl route del via --node-id=` +`slimctl route del via --node-id=` Print version information: @@ -285,7 +327,32 @@ Print version information: Run `slimctl --help` for more details on flags and usage. -### Example: Create, Delete Route +### Example 1: Create, Delete Route using node-id + +Add route for node `slim/a` to forward messages for `org/default/alice/0` to node `slim/b`. +```bash +slimctl node list + +Node ID: slim/b status: CONNECTED + Connection details: + - Endpoint: 127.0.0.1:46457 + MtlsRequired: false + ExternalEndpoint: test-slim.default.svc.cluster.local:46457 +Node ID: slim/a status: CONNECTED + Connection details: + - Endpoint: 127.0.0.1:46357 + MtlsRequired: false + ExternalEndpoint: test-slim.default.svc.cluster.local:46357 + +slimctl route add org/default/alice/0 via slim/b --node-id slim/a + + +# Delete an existing route +slimctl route del org/default/alice/0 via slim/b --node-id slim/a +``` + +### Example 2: Create, Delete Route using `connection_config.json` + ```bash # Add a new route @@ -301,7 +368,7 @@ slimctl route add org/default/alice/0 via connection_config.json slimctl route del org/default/alice/0 via http://localhost:46367 ``` -For full reference of connection_config.json, see the [client-config-schema.json](https://github.com/agntcy/slim/blob/main/data-plane/core/config/src/grpc/schema/client-config.schema.json). +For full reference of connection_config.json, see the [client-config-schema.json](https://github.com/agntcy/slim/blob/slim-v0.6.0/data-plane/core/config/src/grpc/schema/client-config.schema.json). ### Managing SLIM Nodes Directly @@ -333,7 +400,7 @@ To enable this, configure the node to host a server allowing the client to conne clients: [] ``` -List connection on a SLIM instance: +List connections on a SLIM instance: `slimctl node-connect connection list --server=` List routes on a SLIM instance: diff --git a/docs/messaging/slim-data-plane.md b/docs/messaging/slim-data-plane.md index ea7ff9f..25eaa38 100644 --- a/docs/messaging/slim-data-plane.md +++ b/docs/messaging/slim-data-plane.md @@ -23,8 +23,8 @@ org/namespace/service/client Kubernetes cluster), the organization, namespace, and service components remain the same for all instances. - **Client**: The final component is generated by SLIM and is a hash of the - client's identity (e.g., a hash of the public key). This uniquely identifies - each specific client instance. + client's identity (e.g., a hash of its public key). This uniquely identifies + each client instance. This naming structure supports both Anycast and Unicast message delivery: @@ -33,9 +33,9 @@ This naming structure supports both Anycast and Unicast message delivery: - **Unicast**: By including the fourth component, the message is delivered directly to the specified client instance. -This approach enables efficient client discovery in fact the message will be +This approach enables efficient client discovery. In fact, the message will be delivered by the SLIM network to a client that is able to process it, even if the -real name of the client is unknown. The anycast forwarding is mostly used in +real name of the client is unknown. Anycast forwarding is primarily used in this discovery phase. In addition to client endpoints, SLIM allows messages to be sent to Channels. A @@ -68,51 +68,16 @@ The session layer offers several functionalities: formatting, routing, and delivery confirmation, while providing simple send and receive primitives to applications. -The session layer provides two main APIs: a 1:1 session, where two clients -communicate directly, and a group session, where multiple clients exchange -messages on a shared channel. - -### One-to-one Session - -A one-to-one (1:1) session is used when an application needs to communicate with -a single endpoint. It supports several modes: - -- **Reliable or Unreliable**: Applications can choose whether the session layer - implements retransmission mechanisms for reliable communication or disables - them for lower overhead. -- **Fire-and-Forget or Request/Reply**: In fire-and-forget mode, the - application sends a message to the other endpoint without waiting for a reply. - For reliable sessions, retransmissions are handled transparently by the - session layer. In request/reply mode, a timeout is setup at application level, - which is cancelled when the other endpoint replies. -- **Anycast or Sticky**: In anycast mode, each message can be forwarded to a - different client exposing the same service name, which is useful for stateless - applications. In sticky mode, the first message is sent using anycast to - discover an available client; subsequent messages are sent to the same - endpoint. - -By default, all sessions are secured using MLS. In a 1:1 session, the group -consists of only two clients, which must maintain some state for the MLS -protocol. When MLS is enabled, the session is automatically forced to use the -sticky mode, so that all the messages will be sent to the same client. - -### Group Session - -In an N:N (group) session, multiple clients can exchange messages on the same -channel. There are two types of clients in this session: a standard participant -that can only be invited to the channel and participate in messaging, and a -moderator. The moderator is a special client that has the following functionalities: - -- **Invite/Remove clients**: The moderator is the only client that can create a - channel and can modify the list of clients participating in the group - communication. -- **MLS state management**: A channel has also an associated MLS group to - guarantee security. The moderator performs the functionalities of the MLS - delivery service, that routes MLS messages among the group participants in - order to keep the state always updated. - -As for the naming, for more information on the session layer, see the [SLIM -Specifications](https://spec.slim.agntcy.org/draft-agntcy-slim.html) - -A comprehensive tutorial on how to set up a secure group communication system using -SLIM can be found in the [Group Communication Tutorial](./slim-group-tutorial.md). + +The session layer offers two primary APIs for establishing new sessions: + +- **Point-to-Point**: Facilitates point-to-point communication with a specific service + instance. This session performs a discovery phase to bind the session + to a single instance; all subsequent messages in the session are sent to that + same endpoint. + +- **Group**: Supports many-to-many communication over a named channel. +Every message sent to the channel is delivered to all current participants. + +For more information about each session type, see the +[SLIM session](./slim-session.md) documentation. diff --git a/docs/messaging/slim-group-tutorial.md b/docs/messaging/slim-group-tutorial.md index f27691f..af900c5 100644 --- a/docs/messaging/slim-group-tutorial.md +++ b/docs/messaging/slim-group-tutorial.md @@ -1,590 +1,845 @@ # SLIM Group Communication Tutorial -This tutorial will show how to set up a secure group communication system using -SLIM. The group will be created by defining a pub-sub session with a moderator, -which will invite the other members. Messages will be sent to the shared -channel, where every member can read and write. Messages are end-to-end -encrypted using the [MLS -protocol](https://datatracker.ietf.org/doc/html/rfc9420). +This tutorial shows how to set up secure group communication using +SLIM. The group is created by defining a group session and inviting +participants. Messages are sent to a shared channel where every member can read +and write. All messages are end-to-end encrypted using the +[MLS protocol](https://datatracker.ietf.org/doc/html/rfc9420). This tutorial is +based on the +[group.py](https://github.com/agntcy/slim/blob/slim-v0.6.0/data-plane/python/bindings/examples/src/slim_bindings_examples/group.py) +example in the SLIM repo. ## Key Features - **Name-based Addressing**: In SLIM, all endpoints (channels and clients) have a name, and messages use a name-based addressing scheme for content routing. -- **Session Management**: Allows for the creation and management of sessions - with moderators. +- **Session Management**: Allows for the creation and management of sessions using + both the SLIM Python Bindings and the SLIM Controller. - **Broadcast Messaging**: Facilitates broadcast messaging to multiple subscribers. - **End-to-End Encryption**: Ensures secure communication using the [MLS protocol](https://datatracker.ietf.org/doc/html/rfc9420). -### Setting up the SLIM Instance +## Configure Client Identity and Implement the SLIM App -As all members of the group will be communicating via a SLIM network, we can set -up a SLIM instance representing the SLIM network. We will use the pre-built -docker image for this purpose. - -First execute this command to create the SLIM configuration file. Details about -the configuration can be found in the [SLIM -Repo](https://github.com/agntcy/slim/tree/main/data-plane/config). - -```bash -cat << EOF > ./config.yaml -tracing: - log_level: info - display_thread_names: true - display_thread_ids: true - -runtime: - n_cores: 0 - thread_name: "slim-data-plane" - drain_timeout: 10s - -services: - slim/0: - pubsub: - servers: - - endpoint: "0.0.0.0:46357" - tls: - insecure: true - - clients: [] -EOF -``` - -This configuration will start a SLIM instance with a server listening on port -46357, without TLS encryption for simplicity. Messages will be still encrypted -using the MLS protocol, but the connections between SLIM nodes will not use TLS. -In a production environment, it is recommended to always use TLS and configure -proper authentication and authorization mechanisms. - -You can run the SLIM instance using Docker: - -```bash -docker run -it \ - -v ./config.yaml:/config.yaml -p 46357:46357 \ - ghcr.io/agntcy/slim:latest /slim --config /config.yaml -``` - -If everything goes fine, you should see an output like this one: - -``` -2025-07-31T09:07:45.859161Z INFO main ThreadId(01) application_lifecycle: slim: Runtime started -2025-07-31T09:07:45.859213Z INFO main ThreadId(01) application_lifecycle: slim: Starting service: slim/0 -2025-07-31T09:07:45.859624Z INFO main ThreadId(01) application_lifecycle: slim_service: starting service -2025-07-31T09:07:45.859683Z INFO main ThreadId(01) application_lifecycle: slim_service: starting server 0.0.0.0:46357 -2025-07-31T09:07:45.859793Z INFO main ThreadId(01) application_lifecycle: slim_service: server configured: setting it up config=ServerConfig { endpoint: 0.0.0.0:46357, tls_setting: TlsServerConfig { config: Config { ca_file: None, ca_pem: None, include_system_ca_certs_pool: false, cert_file: None, cert_pem: None, key_file: None, key_pem: None, tls_version: "tls1.3", reload_interval: None }, insecure: true, client_ca_file: None, client_ca_pem: None, reload_client_ca_file: false }, http2_only: true, max_frame_size: None, max_concurrent_streams: None, max_header_list_size: None, read_buffer_size: None, write_buffer_size: None, keepalive: KeepaliveServerParameters { max_connection_idle: 3600s, max_connection_age: 7200s, max_connection_age_grace: 300s, time: 120s, timeout: 20s }, auth: None } -2025-07-31T09:07:45.861393Z INFO slim-data-plane ThreadId(11) slim_service: running service -``` - -### Configure Client Identity and Implementing the SLIM App +Every participant in a group requires a unique identity for authentication and for use by the MLS protocol. This section explains how to set up identity and create a SLIM application instance. -Each member of the group will run a local slim app instance that will be used to -communicate with the SLIM network. Also each member will have a unique identity -that will be used to authenticate the member in the SLIM network. +### Identity -#### Identity +Each participant must have a unique identity. This is required to set up end-to-end encryption using the MLS protocol. The identity can be a JWT or a shared secret. For simplicity, this example uses a shared secret. For JWT-based identity, see the [tutorial](https://github.com/agntcy/slim/tree/slim-v0.6.0/data-plane/python/bindings/examples#running-in-kubernetes-spire--jwt) in the SLIM repository. -Each member of the group must have a unique identity. This is a requirement to -setup the end-to-end encryption using the MLS protocol. The identity can be -represented by a JWT, or a shared secret. For simplicity, we will use a shared -secret. You can find examples using JWT in the [SLIM -Repo](https://github.com/agntcy/slim/tree/slim-v0.4.0/data-plane/python-bindings/examples/src/slim_bindings_examples/common.py#L71-L112). - -The python objects managing the identity are called `PyIdentityProvider` and -`PyIdentityVerifier`. The `PyIdentityProvider` is responsible for providing the -identity, while the `PyIdentityVerifier` is responsible for verifying the -identity. +The Python objects managing the identity are called `PyIdentityProvider` and `PyIdentityVerifier`. The `PyIdentityProvider` provides the identity, while the `PyIdentityVerifier` verifies it: ```python -def shared_secret_identity( - identity: str, secret: str -) -> tuple[PyIdentityProvider, PyIdentityVerifier]: +def shared_secret_identity(identity: str, secret: str): """ - Create a provider and verifier using a shared secret. + Create a provider & verifier pair for shared-secret (symmetric) authentication. + + Args: + identity: Logical identity string (often same as PyName string form). + secret: Shared secret used to sign / verify tokens (not for production). - :param identity: A unique string, identifier of the app. - :param secret: A shared secret used for authentication. - :return: A tuple of (provider, verifier). + Returns: + (provider, verifier): Tuple of PyIdentityProvider & PyIdentityVerifier. """ - provider: PyIdentityProvider = PyIdentityProvider.SharedSecret( + provider = slim_bindings.PyIdentityProvider.SharedSecret( # type: ignore identity=identity, shared_secret=secret ) - verifier: PyIdentityVerifier = PyIdentityVerifier.SharedSecret( + verifier = slim_bindings.PyIdentityVerifier.SharedSecret( # type: ignore identity=identity, shared_secret=secret ) - return provider, verifier ``` -#### SLIM App +This is a helper function defined in +[common.py](https://github.com/agntcy/slim/blob/slim-v0.6.0/data-plane/python/bindings/examples/src/slim_bindings_examples/common.py#L85) +that can be used to create a `PyIdentityProvider` and `PyIdentityVerifier` from two input strings. + +### SLIM App -The provider and the verifier will be used to create a local SLIM app, that can -be used to exchange messages with other apps via the SLIM network. +The provider and verifier are used to create a local SLIM application that can exchange messages with other participants via the SLIM network. To create the SLIM app, use the helper function defined in [common.py](https://github.com/agntcy/slim/blob/slim-v0.6.0/data-plane/python/bindings/examples/src/slim_bindings_examples/common.py#L289): ```python -async def create_slim_app(secret: str, local_name: PyName) -> PyService: +async def create_local_app( + local: str, + slim: dict, + remote: str | None = None, + enable_opentelemetry: bool = False, + shared_secret: str = "secret", + jwt: str | None = None, + spire_trust_bundle: str | None = None, + audience: list[str] | None = None, +): """ - Create a SLIM app instance with the given shared secret. - This app will be used to communicate with other SLIM nodes in the network. - - :param secret: A shared secret used for authentication. - :param local_name: A unique name for the SLIM app instance. - It will be used to identify the app in the SLIM network. - :return: A SLIM app instance. + Build and connect a Slim application instance given user CLI parameters. + + Resolution precedence for auth: + 1. If jwt + bundle + audience provided -> JWT/JWKS flow. + 2. Else -> shared secret (must be provided, raises if missing). + + Args: + local: Local identity string (org/ns/app). + slim: Dict of connection parameters (endpoint, tls flags, etc.). + remote: Optional remote identity (unused here, reserved for future). + enable_opentelemetry: Enable OTEL tracing export. + shared_secret: Symmetric secret for shared-secret mode. + jwt: Path to static JWT token (for StaticJwt provider). + spire_trust_bundle: Path to a spire trust bundle file (containing the JWKs for each trust domain). + audience: Audience list for JWT verification. + + Returns: + Slim: Connected high-level Slim instance. """ - - # Create the provider and verifier using the shared secret. - provider, verifier = shared_secret_identity( - identity=f"{local_name}", - secret=secret, + # Initialize tracing (synchronous init; not awaited as binding returns immediately). + slim_bindings.init_tracing( + { + "log_level": "info", + "opentelemetry": { + "enabled": enable_opentelemetry, + "grpc": { + "endpoint": "http://localhost:4317", + }, + }, + } ) - # Create the SLIM app. This is a in-process SLIM client that can be used to - # exchange messages with other SLIM nodes in the network. - slim_app = await Slim.new(local_name, provider, verifier) + # Derive identity provider & verifier using JWT/JWKS if all pieces supplied. + if jwt and spire_trust_bundle and audience: + print("Using JWT + JWKS authentication.") + provider, verifier = jwt_identity( + jwt, + spire_trust_bundle, + aud=audience, + ) + else: + print( + "Warning: Falling back to shared-secret authentication. Don't use this in production!" + ) + # Fall back to shared secret (dev-friendly default). + provider, verifier = shared_secret_identity( + identity=local, + secret=shared_secret, + ) - # Connect the SLIM app to the SLIM network. - _ = await slim_app.connect( - {"endpoint": "http://127.0.0.1:46357", "tls": {"insecure": True}} - ) + # Convert local identifier to a strongly typed PyName. + local_name = split_id(local) + + # Instantiate Slim (async constructor prepares underlying PyService). + local_app = await slim_bindings.Slim.new(local_name, provider, verifier) - # Return the SLIM app instance. - return slim_app + # Provide feedback to user (instance numeric id). + format_message_print(f"{local_app.id_str}", "Created app") + + # Establish outbound connection using provided parameters. + _ = await local_app.connect(slim) + + # Confirm endpoint connectivity. + format_message_print(f"{local_app.id_str}", f"Connected to {slim['endpoint']}") + + return local_app ``` -### Implementing the Moderator +This function takes several parameters as input: + +- `local` (required, str): The SLIM name of the local application in the form + `org/ns/service`. +- `slim` (required, dict): Configuration to connect to the remote SLIM node. For example: -The moderator will be responsible for creating the group and inviting other -members. The moderator will create a session and send an invitation message to -the other members. The invitation will contain the session ID and the channel -name for the group communication. + ```python + { + "endpoint": "http://127.0.0.1:46357", + "tls": {"insecure": True}, + } + ``` -The moderator can be implemented as a Python service using the SLIM SDK. +- `enable_opentelemetry` (bool, default: `False`): Enable OpenTelemetry + tracing. If `True`, traces are sent to `http://localhost:4317` by default. +- `shared_secret` (str | None, default: `None`): Shared secret for identity and + authentication. Required if JWT, bundle and audience are not provided. +- `jwt` (str | None, default: `None`): JWT token for identity. Used with + `spire_trust_bundle` and `audience` for JWT-based authentication. +- `spire_trust_bundle` (str | None, default: `None`): JWT trust bundle + (list of JWKs, one for each trust domain). It is expected in JSON format such as: -#### Creating the Session and Inviting Members + ```json + { + "trust-domain-1.org": "base-64-encoded-jwks", + "trust-domain-2.org": "base-64-encoded-jwks", + ... + } + ``` -The moderator creates a session and invites other members to join the group. The +- `audience` (list[str] | None, default: `None`): List of allowed audiences for + JWT authentication. + +If `jwt`, `spire_trust_bundle`, and `audience` are not provided, `shared_secret` must be set (only recommended for local testing or examples, not production). In this example, we use the shared secret option, but the same function supports all authentication flows. + +## Group Communication Using the Python Bindings + +Now that you know how to set up a SLIM application, we can see how to create a group where multiple participants can exchange messages. We start by showing how to create a group session using the Python bindings. + +In this setting, one participant acts as moderator: it creates the group session and invites participants by sending invitation control messages. A detailed description of group sessions and the invitation process is available [here](https://github.com/agntcy/slim/blob/slim-v0.6.0/data-plane/python/bindings/SESSION.md). + +### Creating the Group Session and Inviting Members + +The creator of the group session invites other members to join the group. The session will be identified by a unique session ID, and the group communication -will take place over a specific channel name. The moderator will be responsible +will take place over a specific channel name. The session creator is responsible for managing the session lifecycle, including creating, updating, and terminating the session as needed. As each participant is provided with an identity, setting up MLS for end-to-end -encryption is straightforward. The moderator will create a session with the +encryption is straightforward: the session is created with the `mls_enabled` flag set to `True`, which will enable the MLS protocol for the session. This ensures that all messages exchanged within the session are end-to-end encrypted, providing confidentiality and integrity for the group communication. ```python -async def create_session_and_invite_members( - slim_app: PyService, invitees: list[PyName] + # Create & connect the local Slim instance (auth derived from args). + local_app = await create_local_app( + local, + slim, + enable_opentelemetry=enable_opentelemetry, + shared_secret=shared_secret, + jwt=jwt, + spire_trust_bundle=spire_trust_bundle, + audience=audience, + ) + + # Parse the remote channel/topic if provided; else None triggers passive mode. + chat_channel = split_id(remote) if remote else None + + # Track background tasks (receiver loop + optional keyboard loop). + tasks: list[asyncio.Task] = [] + + # Session sharing between tasks + session_ready = asyncio.Event() + shared_session_container = [None] # Use list to make it mutable across functions + + # Session object only exists immediately if we are moderator. + created_session = None + if chat_channel and invites: + # We are the moderator; create the group session now. + format_message_print( + f"Creating new group session (moderator)... {split_id(local)}" + ) + created_session = await local_app.create_session( + slim_bindings.PySessionConfiguration.Group( # type: ignore # Build group session configuration + channel_name=chat_channel, # Logical group channel (PyName) all participants join; acts as group/topic identifier. + max_retries=5, # Max per-message resend attempts upon missing ack before reporting a delivery failure. + timeout=datetime.timedelta( + seconds=5 + ), # Ack / delivery wait window; after this duration a retry is triggered (until max_retries). + mls_enabled=enable_mls, # Enable Messaging Layer Security for end-to-end encrypted & authenticated group communication. + ) + ) + + # Small delay so underlying routing / session creation stabilizes. + await asyncio.sleep(1) + + # Invite each provided participant. Route is set before inviting to ensure + # outbound control messages can reach them. For more info see + # https://github.com/agntcy/slim/blob/slim-v0.6.0/data-plane/python/bindings/SESSION.md#invite-a-new-participant + for invite in invites: + invite_name = split_id(invite) + await local_app.set_route(invite_name) + await created_session.invite(invite_name) + print(f"{local} -> add {invite_name} to the group") +``` + +This code comes from the +[group.py](https://github.com/agntcy/slim/blob/slim-v0.6.0/data-plane/python/bindings/examples/src/slim_bindings_examples/group.py) +example. The local application is created using the helper function shown earlier. +The channel name (the logical group topic) is produced via the +[split_id](https://github.com/agntcy/slim/blob/slim-v0.6.0/data-plane/python/bindings/examples/src/slim_bindings_examples/common.py#L63) +helper by parsing the `remote` parameter. A new group session is then created +using `local_app.create_session(...)` with a +`slim_bindings.PySessionConfiguration.Group` configuration. The key parameters are the following: + +- `channel_name`: Logical channel/topic used to exchange messages among participants. +- `max_retries`: Maximum number of retransmission attempts (upon missing ack) before + notifying the application of delivery failure. +- `timeout`: Duration to wait for an acknowledgment; if the ack is not received in time a retry is triggered. If + omitted / None, the session is unreliable (no retry/ack flow). +- `mls_enabled`: Set to `True` to enable MLS for end-to-end encryption. + +After the session creation, the moderator invites participants via `created_session.invite`. +Before sending each invitation it must call `local_app.set_route(invite_name)` so +SLIM knows how to deliver the control messages. + +### Implement Participants and Receive Messages + +The group participants are implemented in a similar way, but they +do not create the session. They create the SLIM service instance and wait +for invites. Once they receive the invite, they can read and write on the shared channel. + +```python +async def receive_loop( + local_app, created_session, session_ready, shared_session_container ): """ - Create a session with the given session ID and channel ID. + Receive messages for the bound session. - :param slim_app: The SLIM app instance. - :return: The created session. + Behavior: + * If not moderator: wait for a new group session (listen_for_session()). + * If moderator: reuse the created_session reference. + * Loop forever until cancellation or an error occurs. """ + if created_session is None: + print_formatted_text("Waiting for session...", style=custom_style) + session = await local_app.listen_for_session() + else: + session = created_session + + # Make session available to other tasks + shared_session_container[0] = session + session_ready.set() + + while True: + try: + # Await next inbound message from the group session. + # The returned parameters are a message context and the raw payload bytes. + # Check session.py for details on PyMessageContext contents. + ctx, payload = await session.get_message() + print_formatted_text( + f"{ctx.source_name} > {payload.decode()}", + style=custom_style, + ) + except asyncio.CancelledError: + # Graceful shutdown path (ctrl-c or program exit). + break + except Exception as e: + # Non-cancellation error; surface and exit the loop. + print_formatted_text(f"-> Error receiving message: {e}") + break +``` - # Define the shared channel for group communication. - # This channel will be used by all members of the group to exchange messages. - shared_channel = PyName("agntcy", "namespace", "group_channel") - - # Create a new session. The group session is a bidirectional streaming session. - # Here is where we enable the MLS protocol for end-to-end encryption. - session_info = await slim_app.create_session( - PySessionConfiguration.Streaming( - PySessionDirection.BIDIRECTIONAL, - topic=shared_channel, # The channel ID for group communication. - moderator=True, # This session is created by the moderator. - max_retries=5, # Maximum number of retries for reliability. - timeout=datetime.timedelta(seconds=5), # Timeout for message delivery. - mls_enabled=True, # Enable MLS for end-to-end encryption. +Each non-moderator participant listens for an incoming session using +`local_app.listen_for_session()`. This returns a +[PySession](https://github.com/agntcy/slim/blob/slim-v0.6.0/data-plane/python/bindings/slim_bindings/session.py) +object containing metadata such as session ID, type, source name, and destination name. +The moderator already holds this information and therefore reuses the existing +`created_session` (see `session = created_session`). + +Participants (including the moderator) then call `ctx, payload = await session.get_message()` to receive +messages. `payload` contains the raw message bytes and `ctx` is a +[PyMessageContext](https://github.com/agntcy/slim/blob/slim-v0.6.0/data-plane/python/bindings/slim_bindings/_slim_bindings.pyi#L22) +with source, destination, message type, and metadata. + +### Publish Messages to the Session + +All participants can publish messages on the shared channel: + +```python +async def keyboard_loop(session_ready, shared_session_container, local_app): + """ + Interactive loop allowing participants to publish messages. + + Typing 'exit' or 'quit' (case-insensitive) terminates the loop. + Each line is published to the group topic as UTF-8 bytes. + """ + try: + # 1. Initialize an async session + prompt_session = PromptSession(style=custom_style) + + # Wait for the session to be established + await session_ready.wait() + + print_formatted_text( + f"Welcome to the group {shared_session_container[0].dst}!\nSend a message to the group, or type 'exit' or 'quit' to quit.", + style=custom_style, ) - ) - print(f"Session created: {session_info}") + while True: + # Run blocking input() in a worker thread so we do not block the event loop. + user_input = await prompt_session.prompt_async( + f"{shared_session_container[0].src} > " + ) + + if user_input.lower() in ("exit", "quit"): + # Also terminate the receive loop. + await local_app.delete_session(shared_session_container[0]) + break + + try: + # Send message to the channel_name specified when creating the session. + # As the session is group, all participants will receive it. + # calling publish_with_destination on a group session will raise an error. + await shared_session_container[0].publish(user_input.encode()) + except KeyboardInterrupt: + # Handle Ctrl+C gracefully + break + except Exception as e: + print_formatted_text(f"-> Error sending message: {e}") + except asyncio.CancelledError: + # Handle task cancellation gracefully + pass +``` - # Invite other members to the session. - for invitee in invitees: - print(f"Inviting {invitee}") - await slim_app.set_route(invitee) # Allow messages to be sent to the invitee. - await slim_app.invite( - session_info, invitee - ) # Send an invitation to the invitee. +Messages are sent using `shared_session_container[0].publish(user_input.encode())`. +Only the payload is provided and there is no explicit destination, because the +group channel was fixed at session creation and delivery fan-outs to all +participants. - # Return the created session. - return session_info +### Run the Group Communication Example -async def run_moderator(secret: str): - local_name = PyName("agntcy", "namespace", "moderator") +Now we will show how to run a new group session and +how to enable group communication on top of SLIM. The full code can be found in +[group.py](https://github.com/agntcy/slim/blob/slim-v0.6.0/data-plane/python/bindings/examples/src/slim_bindings_examples/group.py) +in the SLIM repo. To run the example, follow the steps listed here: - # Create the moderator SLIM app instance. - moderator_slim_app = await create_slim_app(secret, local_name) +#### Run SLIM - # Define the invitees for the group session. - invitees = [ - PyName("agntcy", "namespace", "participant1"), - ] +As all members of the group are communicating via a SLIM network, we can set +up a SLIM instance representing the SLIM network. We use the pre-built +docker image for this purpose. - async with moderator_slim_app: - # Create a session and invite the members. - session_info = await create_session_and_invite_members(moderator_slim_app, invitees) +First execute this command to create the SLIM configuration file. Details about +the [configuration](https://github.com/agntcy/slim/tree/slim-v0.6.0/data-plane/config) +can be found in the SLIM repo. - # Wait for a bit to ensure all participants are ready. - await asyncio.sleep(1) +```bash +cat << EOF > ./config.yaml +tracing: + log_level: info + display_thread_names: true + display_thread_ids: true - # Send a message to the group channel. - await send_message(moderator_slim_app, session_info, "Hello group!") +runtime: + n_cores: 0 + thread_name: "slim-data-plane" + drain_timeout: 10s - # Wait for a message from a participant - _, msg = await moderator_slim_app.receive(session=session_info.id) +services: + slim/0: + dataplane: + servers: + - endpoint: "0.0.0.0:46357" + tls: + insecure: true - # Print message - print(f"Received message from participant: {msg.decode()}") + clients: [] + controller: + servers: [] +EOF ``` -### Implementing the Group Participants +This configuration starts a SLIM instance with a server listening on port +46357, without TLS encryption for simplicity. Messages are still encrypted +using the MLS protocol, but the connections between SLIM nodes do not use TLS. +In a production environment, it is recommended to always use TLS and configure +proper authentication and authorization mechanisms. -The group participants will be implemented similarly to the moderator, but they -will not create the session. They will create the SLIM service instance and wait -for invites sent by the moderator. Once they receive the invite, they can read -and write on the shared channel. +You can run the SLIM instance using Docker: -```python -async def run_participant(secret: str): - local_name = PyName("agntcy", "namespace", "participant1") +```bash +docker run -it \ + -v ./config.yaml:/config.yaml -p 46357:46357 \ + ghcr.io/agntcy/slim:0.6.0 /slim --config /config.yaml +``` - participant_slim_app = await create_slim_app(secret, local_name) +If everything goes fine, you should see an output like this one: - print(f"Listening for sessions - locator: {local_name}") +```bash +2025-10-06T08:22:54.529981Z INFO main ThreadId(01) application_lifecycle: slim: Runtime started +2025-10-06T08:22:54.530116Z INFO main ThreadId(01) application_lifecycle: slim: Starting service: slim/0 +2025-10-06T08:22:54.530157Z INFO main ThreadId(01) application_lifecycle: slim_service::service: starting service +2025-10-06T08:22:54.530193Z INFO main ThreadId(01) application_lifecycle: slim_service::service: starting server 0.0.0.0:46357 +... +``` - async with participant_slim_app: - # Listen for new sessions opened by moderators - recv_session, _ = await participant_slim_app.receive() +#### Start the Participants - # Session is received, now we can read and write on the shared channel. - print(f"Received session: {recv_session.id}") +In this example we use two participants: `agntcy/ns/client-1` and `agntcy/ns/client-2`. +Authentication uses a shared secret. In the SLIM repository, go to the folder +`slim/data-plane/python/bindings/examples` and run these commands in two different terminals: - # Receive messages from the session - recv_session, msg_rcv = await participant_slim_app.receive(session=recv_session.id) +```bash +uv run --package slim-bindings-examples group \ + --local agntcy/ns/client-1 \ + --slim '{"endpoint": "http://localhost:46357", "tls": {"insecure": true}}' \ + --shared-secret "secret" +``` - # Print the message - print(f"Received: {msg_rcv.decode()}") +```bash +uv run --package slim-bindings-examples group \ + --local agntcy/ns/client-2 \ + --slim '{"endpoint": "http://localhost:46357", "tls": {"insecure": true}}' \ + --shared-secret "secret" - # Reply with a message - await participant_slim_app.publish( - recv_session, - f"{msg_rcv.decode()} from participant".encode(), - recv_session.destination_name, - ) +``` - # Wait to ensure message is sent - await asyncio.sleep(1) +This start two participants authenticated with a shared secret. +The output of these commands should look like this: + +```bash +Warning: Falling back to shared-secret authentication. Don't use this in production! +Agntcy/ns/client-1/456243414154990054 Created app +Agntcy/ns/client-1/456243414154990054 Connected to http://localhost:46357 +Waiting for session... ``` -### Putting All Together +#### Create the Group -Here is the complete code to run the moderator and the participants in a single -script. You can run this script to see how the group communication works using -SLIM. +Run the moderator application to create the session and invite the two +participants. In another terminal run: -The same example can be found in the [SLIM examples -folder](https://github.com/agntcy/slim/tree/slim-v0.4.0/data-plane/python-bindings/examples/src/slim_bindings_examples). -In particular this tutorial is based on the -[pubsub.py](https://github.com/agntcy/slim/tree/slim-v0.4.0/data-plane/python-bindings/examples/src/slim_bindings_examples/pubsub.py) -example +```bash +uv run --package slim-bindings-examples group \ + --local agntcy/ns/moderator \ + --slim '{"endpoint": "http://localhost:46357", "tls": {"insecure": true}}' \ + --shared-secret "secret" \ + --remote agntcy/ns/chat \ + --invites agntcy/ns/client-1 \ + --invites agntcy/ns/client-2 \ + --enable-mls +``` -#### moderator.py +The result should look like: -```python -import asyncio -import datetime - -from slim_bindings import ( - Slim, - PyName, - PyService, - PyIdentityProvider, - PyIdentityVerifier, - PySessionConfiguration, - PySessionDirection, - init_tracing, -) - - -def shared_secret_identity( - identity: str, secret: str -) -> tuple[PyIdentityProvider, PyIdentityVerifier]: - """ - Create a provider and verifier using a shared secret. +```bash +Agntcy/ns/moderator/16858445264489265394 Created app +Agntcy/ns/moderator/16858445264489265394 Connected to http://localhost:46357 +Creating new group session (moderator)... 169ca82eb17d6bc2/eef9769a4c6990d1/fc9bbc406957794b/ffffffffffffffff (agntcy/ns/moderator/ffffffffffffffff) +agntcy/ns/moderator -> add 169ca82eb17d6bc2/eef9769a4c6990d1/58ec40d7c837e0b9/ffffffffffffffff (agntcy/ns/client-1/ffffffffffffffff) to the group +agntcy/ns/moderator -> add 169ca82eb17d6bc2/eef9769a4c6990d1/b521a3788f1267a8/ffffffffffffffff (agntcy/ns/client-2/ffffffffffffffff) to the group +Welcome to the group 169ca82eb17d6bc2/eef9769a4c6990d1/4abb367236cabc2a/ffffffffffffffff (agntcy/ns/chat/ffffffffffffffff)! +Send a message to the group, or type 'exit' or 'quit' to quit. +169ca82eb17d6bc2/eef9769a4c6990d1/fc9bbc406957794b/e9f53aa5ef3fb8f2 (agntcy/ns/moderator/e9f53aa5ef3fb8f2) > +``` - :param identity: A unique string, identifier of the app. - :param secret: A shared secret used for authentication. - :return: A tuple of (provider, verifier). - """ - provider: PyIdentityProvider = PyIdentityProvider.SharedSecret( - identity=identity, shared_secret=secret - ) - verifier: PyIdentityVerifier = PyIdentityVerifier.SharedSecret( - identity=identity, shared_secret=secret - ) +Now `client-1` and `client-2` are invited to the group, so on both of their terminals you should +be able to see a welcome message such as: - return provider, verifier +```bash +Welcome to the group 169ca82eb17d6bc2/eef9769a4c6990d1/4abb367236cabc2a/ffffffffffffffff (agntcy/ns/chat/ffffffffffffffff)! +Send a message to the group, or type 'exit' or 'quit' to quit. +169ca82eb17d6bc2/eef9769a4c6990d1/58ec40d7c837e0b9/6a34b65ebc955471 (agntcy/ns/client-1/6a34b65ebc955471) > +``` +At this point, you can write messages from any terminal and they will be received by all other group participants. -async def create_slim_app(secret: str, local_name: PyName) -> PyService: - """ - Create a SLIM app instance with the given shared secret. - This app will be used to communicate with other SLIM nodes in the network. +## Group Communication Using the SLIM Controller - :param secret: A shared secret used for authentication. - :param local_name: A unique name for the SLIM app instance. - It will be used to identify the app in the SLIM network. - :return: A SLIM app instance. - """ +Previously, we saw how to run group communication using the Python bindings with an in-application moderator. +This participant creates the group session and invites all other participants. +In this section, we describe how to create and orchestrate a group using the SLIM Controller, and we show how all +these functions can be delegated to the controller. We reuse the same group example code in this section as well. - # Create the provider and verifier using the shared secret. - provider, verifier = shared_secret_identity( - identity=f"{local_name}", - secret=secret, - ) +Identity handling is unchanged between the two approaches; refer back to [SLIM Identity](#configure-client-identity-and-implement-the-slim-app). Below are the steps to run the controller-managed version. - # Create the SLIM app. This is a in-process SLIM client that can be used to - # exchange messages with other SLIM nodes in the network. - slim_app = await Slim.new(local_name, provider, verifier) +### Application Differences - # Connect the SLIM app to the SLIM network. - _ = await slim_app.connect( - {"endpoint": "http://127.0.0.1:46357", "tls": {"insecure": True}} - ) +With the controller, you do not need to set up a moderator in your application. All participants can be run as we did for `client-1` and `client-2` in the previous examples. In code, this means you can avoid creating a new group session (using `local_app.create_session`) and the invitation loop. You only need to implement the `receive_loop` where the application waits for new sessions. This greatly simplifies your code. - # Return the SLIM app instance. - return slim_app +### Run the Group Communication example +Now we will show how to set up a group using the SLIM Controller. The reference code for the +application is still [group.py](https://github.com/agntcy/slim/blob/slim-v0.6.0/data-plane/python/bindings/examples/src/slim_bindings_examples/group.py). To run this example, follow the steps listed here. -async def create_session_and_invite_members( - slim_app: PyService, invitees: list[PyName] -): - """ - Create a session with the given session ID and channel ID. +#### Run the SLIM Controller - :param slim_app: The SLIM app instance. - :return: The created session. - """ +First, start the SLIM Controller. Full details are in the [Controller](./slim-controller.md) documentation; here we reproduce the minimal setup. Create a configuration file: - # Define the shared channel for group communication. - # This channel will be used by all members of the group to exchange messages. - shared_channel = PyName("agntcy", "namespace", "group_channel") - - # Create a new session. The group session is a bidirectional streaming session. - # Here is where we enable the MLS protocol for end-to-end encryption. - session_info = await slim_app.create_session( - PySessionConfiguration.Streaming( - PySessionDirection.BIDIRECTIONAL, - topic=shared_channel, # The channel ID for group communication. - moderator=True, # This session is created by the moderator. - max_retries=5, # Maximum number of retries for reliability. - timeout=datetime.timedelta(seconds=5), # Timeout for message delivery. - mls_enabled=True, # Enable MLS for end-to-end encryption. - ) - ) +```bash +cat << EOF > ./config-controller.yaml +northbound: + httpHost: 0.0.0.0 + httpPort: 50051 + logging: + level: DEBUG + +southbound: + httpHost: 0.0.0.0 + httpPort: 50052 + +# number of node reconciler threads +reconciler: + threads: 3 + +logging: + level: INFO +EOF +``` - print(f"Session created: {session_info}") +This config defines two APIs exposed by the controller: - # Invite other members to the session. - for invitee in invitees: - print(f"Inviting {invitee}") - await slim_app.set_route(invitee) # Allow messages to be sent to the invitee. - await slim_app.invite( - session_info, invitee - ) # Send an invitation to the invitee. +- Northbound API: used by an operator (e.g. via slimctl) to configure channels and participants, as well as the SLIM network. +- Southbound API: used by SLIM nodes to synchronize with the controller. - # Return the created session. - return session_info +Start the controller with Docker: +```bash +docker run -it \ + -v ./config-controller.yaml:/config.yaml -p 50051:50051 -p 50052:50052 \ + ghcr.io/agntcy/slim/control-plane:0.6.0 --config /config.yaml +``` -async def send_message(slim_app: PyService, session_info, message: str): - """ - Send a message to the group channel. +If everything goes fine, you should see an output like this: - :param slim_app: The SLIM app instance. - :param session_info: The session information. - :param message: The message to send. - """ +```bash +2025-10-06T08:06:06Z INF Starting route reconcilers +2025-10-06T08:06:06Z INF Starting Route Reconciler thread_name=reconciler-1 +2025-10-06T08:06:06Z INF Starting Route Reconciler thread_name=reconciler-0 +2025-10-06T08:06:06Z INF Starting Route Reconciler thread_name=reconciler-2 +2025-10-06T08:06:06Z INF Southbound API Service is Listening on [::]:50052 +2025-10-06T08:06:06Z INF Northbound API Service is listening on [::]:50051 +``` - # Send the message to the shared channel. - await slim_app.publish( - session_info, - message.encode(), - PyName("agntcy", "namespace", "group_channel"), - ) +#### Run the SLIM Node +With the controller running, start a SLIM node configured to talk to it over the Southbound API. This node config includes two additional settings compared to the file from the previous section: -async def run_moderator(secret: str): - local_name = PyName("agntcy", "namespace", "moderator") +- A controller client used to connect to the Southbound API running on port 50052. +- A shared secret token provider that will be used by the SLIM node to send messages over the SLIM network. As with the normal application, you can use a shared secret or a proper JWT. - # Create the moderator SLIM app instance. - moderator_slim_app = await create_slim_app(secret, local_name) +Create the `config-slim.yaml` for the node using the command below. We use the `host.docker.internal` endpoint to reach the controller from inside the Docker container via the host. - # Define the invitees for the group session. - invitees = [ - PyName("agntcy", "namespace", "participant1"), - ] +```bash +cat << EOF > ./config-slim.yaml +tracing: + log_level: info + display_thread_names: true + display_thread_ids: true - async with moderator_slim_app: - # Create a session and invite the members. - session_info = await create_session_and_invite_members( - moderator_slim_app, invitees - ) +runtime: + n_cores: 0 + thread_name: "slim-data-plane" + drain_timeout: 10s - # Wait for a bit to ensure all participants are ready. - await asyncio.sleep(1) +services: + slim/0: + dataplane: + servers: + - endpoint: "0.0.0.0:46357" + tls: + insecure: true - # Send a message to the group channel. - await send_message(moderator_slim_app, session_info, "Hello group!") + clients: [] + controller: + servers: [] + clients: + - endpoint: "http://host.docker.internal:50052" + tls: + insecure: true + token_provider: + shared_secret: "secret" +EOF +``` - # Wait for a message from a participant - _, msg = await moderator_slim_app.receive(session=session_info.id) +This starts a SLIM node that connects to the controller: - # Print message - print(f"Received message from participant: {msg.decode()}") +```bash +docker run -it \ + -v ./config-slim.yaml:/config.yaml -p 46357:46357 \ + ghcr.io/agntcy/slim:0.6.0 /slim --config /config.yaml +``` +If everything goes fine, you should see an output like this one: -def main(): - try: - asyncio.run(run_moderator("shared_secret")) - except KeyboardInterrupt: - print("Client interrupted by user.") +```bash +2025-10-06T08:22:54.529981Z INFO main ThreadId(01) application_lifecycle: slim: Runtime started +2025-10-06T08:22:54.530116Z INFO main ThreadId(01) application_lifecycle: slim: Starting service: slim/0 +2025-10-06T08:22:54.530157Z INFO main ThreadId(01) application_lifecycle: slim_service::service: starting service +2025-10-06T08:22:54.530193Z INFO main ThreadId(01) application_lifecycle: slim_service::service: starting server 0.0.0.0:46357 +... +``` + +On the Controller side, you can see that the new node registers with the controller. The +output should be similar to this: +```bash +2025-10-06T11:47:14+02:00 INF Registering node with ID: slim/0 svc=southbound +2025-10-06T11:47:14+02:00 INF Connection details: [endpoint: 127.0.0.1:46357] svc=southbound +2025-10-06T11:47:14+02:00 INF Create generic routes for node node_id=slim/0 service=RouteService +2025-10-06T11:47:14+02:00 INF Sending routes to registered node slim/0 node_id=slim/0 +2025-10-06T11:47:14+02:00 INF Sending configuration command to registered node connections_count=0 message_id=8e9d311a-0012-4fb2-93dc-cda2cb0dd2ef node_id=slim/0 subscriptions_count=0 subscriptions_to_delete_count=0 +2025-10-06T11:47:14+02:00 INF Sending routes completed successfully ack_messages=[] node_id=slim/0 original_message_id=8e9d311a-0012-4fb2-93dc-cda2cb0dd2ef ``` -#### participant.py +#### Run the Participants -```python -import asyncio -import datetime - -from slim_bindings import ( - Slim, - PyName, - PyService, - PyIdentityProvider, - PyIdentityVerifier, - PySessionConfiguration, - PySessionDirection, - PySessionInfo, - init_tracing, -) - - -def shared_secret_identity( - identity: str, secret: str -) -> tuple[PyIdentityProvider, PyIdentityVerifier]: - """ - Create a provider and verifier using a shared secret. +Because the controller manages the group lifecycle, no participant needs to be designated as moderator in code. Every application instance just waits for a session invite. In three separate terminals, from the folder +`slim/data-plane/python/bindings/examples` run: - :param identity: A unique string, identifier of the app. - :param secret: A shared secret used for authentication. - :return: A tuple of (provider, verifier). - """ - provider: PyIdentityProvider = PyIdentityProvider.SharedSecret( - identity=identity, shared_secret=secret - ) - verifier: PyIdentityVerifier = PyIdentityVerifier.SharedSecret( - identity=identity, shared_secret=secret - ) +```bash +uv run --package slim-bindings-examples group \ + --local agntcy/ns/client-1 \ + --slim '{"endpoint": "http://localhost:46357", "tls": {"insecure": true}}' \ + --shared-secret "secret" +``` - return provider, verifier +```bash +uv run --package slim-bindings-examples group \ + --local agntcy/ns/client-2 \ + --slim '{"endpoint": "http://localhost:46357", "tls": {"insecure": true}}' \ + --shared-secret "secret" +``` +```bash +uv run --package slim-bindings-examples group \ + --local agntcy/ns/client-3 \ + --slim '{"endpoint": "http://localhost:46357", "tls": {"insecure": true}}' \ + --shared-secret "secret" +``` -async def create_slim_app(secret: str, local_name: PyName) -> PyService: - """ - Create a SLIM app instance with the given shared secret. - This app will be used to communicate with other SLIM nodes in the network. +Each terminal should show output similar to: - :param secret: A shared secret used for authentication. - :param local_name: A unique name for the SLIM app instance. - It will be used to identify the app in the SLIM network. - :return: A SLIM app instance. - """ +```bash +Warning: Falling back to shared-secret authentication. Don't use this in production! +Agntcy/ns/client-1/9494657801285491688 Created app +Agntcy/ns/client-1/9494657801285491688 Connected to http://localhost:46357 +Waiting for session... +``` - # Create the provider and verifier using the shared secret. - provider, verifier = shared_secret_identity( - identity=f"{local_name}", - secret=secret, - ) +At this point all applications are waiting for a new session. - # Create the SLIM app. This is a in-process SLIM client that can be used to - # exchange messages with other SLIM nodes in the network. - slim_app = await Slim.new(local_name, provider, verifier) +#### Manage the Group with slimctl - # Connect the SLIM app to the SLIM network. - _ = await slim_app.connect( - {"endpoint": "http://127.0.0.1:46357", "tls": {"insecure": True}} - ) +Use `slimctl` (see [slim-controller](./slim-controller.md)) to send administrative commands to the controller. - # Return the SLIM app instance. - return slim_app +First, you need to run `slimctl`. You can download it from the slim repo using this script: +```bash +#!/bin/bash +set -e + +# This script automatically detects your OS and architecture, +# then downloads the appropriate slimctl binary. + +# Detect OS and architecture +OS=$(uname -s | tr '[:upper:]' '[:lower:]') +ARCH=$(uname -m) + +# Map architecture to the format used in release asset names +case "$ARCH" in + x86_64) + ARCH="amd64" + ;; + aarch64 | arm64) + ARCH="arm64" + ;; + *) + echo "Unsupported architecture: $ARCH" >&2 + exit 1 + ;; +esac + +# Check for supported OS +if [ "$OS" != "linux" ] && [ "$OS" != "darwin" ]; then + echo "Unsupported OS: $OS" >&2 + exit 1 +fi + +# Construct the download URL +VERSION="v0.6.0" +BINARY_NAME="slimctl-${OS}-${ARCH}" +DOWNLOAD_URL="https://github.com/agntcy/slim/releases/download/slimctl-${VERSION}/${BINARY_NAME}" + +# Download the binary +echo "Downloading slimctl for ${OS}-${ARCH}..." +curl -L "${DOWNLOAD_URL}" -o slimctl + +# Make it executable +chmod +x slimctl +``` -async def run_participant(secret: str): - local_name = PyName("agntcy", "namespace", "participant1") +To verify that `slimctl` was downloaded successfully, run the following command: - participant_slim_app = await create_slim_app(secret, local_name) +```bash +./slimctl version +``` - print(f"Listening for sessions - locator: {local_name}") +##### Create the Group - async with participant_slim_app: - # Listen for new sessions opened by moderators - recv_session, _ = await participant_slim_app.receive() +Select any running participant to be the initial member of the group. This participant acts as the logical +moderator of the channel, similar to the Python bindings example. However, you don't +need to handle this explicitly in the code. Run the following command to create the channel: - # Session is received, now we can read and write on the shared channel. - print(f"Received session: {recv_session.id}") +```bash +./slimctl channel create moderators=agntcy/ns/client-1/9494657801285491688 +``` - # Receive messages from the session - recv_session, msg_rcv = await participant_slim_app.receive( - session=recv_session.id - ) +The full name of the application can be taken from the output in the console. The value +`9494657801285491688` is the actual id of the `client-1` application returned by +SLIM. In your case, this value will be different. - # Print the message - print(f"Received: {msg_rcv.decode()}") +Expected response from `slimctl`: - # Reply with a message - await participant_slim_app.publish( - recv_session, - f"{msg_rcv.decode()} from participant".encode(), - recv_session.destination_name, - ) +```bash +Received response: agntcy/ns/xyIGhc2igNGmkeBDlZ +``` - # Wait to ensure message is sent - await asyncio.sleep(1) +The value `agntcy/ns/xyIGhc2igNGmkeBDlZ` is the channel (or group) identifier (name) that must be used in subsequent commands. +On the application side, `client-1` was added to the session, so you should see +something like this: -def main(): - try: - asyncio.run(run_participant("shared_secret")) - except KeyboardInterrupt: - print("Client interrupted by user.") +```bash +Welcome to the group 169ca82eb17d6bc2/eef9769a4c6990d1/e8ab33f6d6111780/ffffffffffffffff (agntcy/ns/xyIGhc2igNGmkeBDlZ/ffffffffffffffff)! +Send a message to the group, or type 'exit' or 'quit' to quit. +169ca82eb17d6bc2/eef9769a4c6990d1/58ec40d7c837e0b9/83c3ccf725835be8 (agntcy/ns/client-1/83c3ccf725835be8) > ``` -## Example: 1:1 Communication +##### Add Participants + +Now that the new group is created, add the additional participants `client-2` and `client-3` using the following `slimctl` commands: + +```bash +./slimctl participant add -c agntcy/ns/xyIGhc2igNGmkeBDlZ agntcy/ns/client-2 +./slimctl participant add -c agntcy/ns/xyIGhc2igNGmkeBDlZ agntcy/ns/client-3 +``` + +The expected `slimctl` output is: + +```bash +Adding participant to channel ID agntcy/ns/xyIGhc2igNGmkeBDlZ: agntcy/ns/client-2 +Participant added successfully to channel ID agntcy/ns/xyIGhc2igNGmkeBDlZ: agntcy/ns/client-2 +``` + +Now all the participants are part of the same group, and so each client log should show that the join was successful: + +```bash +Welcome to the group 169ca82eb17d6bc2/eef9769a4c6990d1/e8ab33f6d6111780/ffffffffffffffff (agntcy/ns/xyIGhc2igNGmkeBDlZ/ffffffffffffffff)! +Send a message to the group, or type 'exit' or 'quit' to quit. +169ca82eb17d6bc2/eef9769a4c6990d1/b521a3788f1267a8/e4011f7be5222a24 (agntcy/ns/client-2/e4011f7be5222a24) > +``` + +At this point, every member can send messages, and they will be received by all the other participants. + +##### Remove a Participant -The slim repository also includes examples of 1:1 communication sessions. Using -the SLIM SDK for 1:1 sessions is very similar to the approach demonstrated in -the group communication example. For reference, see the -[fire_and_forget.py](https://github.com/agntcy/slim/tree/slim-v0.4.0/data-plane/python-bindings/examples/src/slim_bindings_examples/fire_and_forget.py) -and -[request_reply.py](https://github.com/agntcy/slim/tree/slim-v0.4.0/data-plane/python-bindings/examples/src/slim_bindings_examples/request_reply.py) -files. +To remove one of the participants from the channel, run the following command: -1:1 communication is particularly useful when you want to use SLIM as a -transport layer for protocols that are inherently point-to-point, such as MCP or -A2A. In these cases, you typically need to communicate with a single server, but -you can still benefit from the simplicity and security that the SLIM messaging -layer provides. +```bash +./slimctl participant delete -c agntcy/ns/xyIGhc2igNGmkeBDlZ agntcy/ns/client-3 +``` + +The `slimctl` expected output is this: + +```bash +Deleting participant from channel ID agntcy/ns/xyIGhc2igNGmkeBDlZ: agntcy/ns/client-3 +Participant deleted successfully from channel ID agntcy/ns/xyIGhc2igNGmkeBDlZ: agntcy/ns/client-3 +``` + +The application on `client-3` exits because the session related to the group was closed, which breaks the +receive loop in the Python code. Notice that this command does not work +for `client-1`, which was added as the first participant. In fact, removing `client-1` is +equivalent to deleting the channel itself. + +##### Delete channel + +To delete the channel, run the following command: + +```bash +./slimctl channel delete agntcy/ns/xyIGhc2igNGmkeBDlZ +``` + +The `slimctl` output is this: + +```bash +Channel deleted successfully with ID: agntcy/ns/xyIGhc2igNGmkeBDlZ +``` -For a detailed guide on using MCP on top of SLIM, please refer to the [SLIM and -MCP Integration](slim-mcp.md) page. +All applications connected to the group stop because the receive loops end. diff --git a/docs/messaging/slim-group.md b/docs/messaging/slim-group.md index 0e50092..9b7d716 100644 --- a/docs/messaging/slim-group.md +++ b/docs/messaging/slim-group.md @@ -1,4 +1,4 @@ -# SLIM Group Management +# SLIM Group Creation and Management One of the key features of [SLIM](slim-core.md) is its support for secure group communication. In SLIM, a group consists of multiple clients that communicate through a shared @@ -7,440 +7,154 @@ Messaging Layer](slim-data-plane.md). When MLS is enabled, group communication benefits from end-to-end encryption. This guide provides all the information you need to create and manage groups within a -SLIM network. +SLIM network. A full tutorial with examples is available in +[Group Communication Tutorial](./slim-group-tutorial.md). -## Creating Groups with a Moderator +## Creating Groups with the Python Bindings -As described in the [SLIM Messaging Layer](slim-data-plane.md), each group is -managed by a moderator. A moderator is a special client with the ability to -create channels, add or remove participants, and perform the functions that are -typically delegated to the Delivery Service in the MLS protocol. +This section shows how to use the SLIM Python bindings to create a group. +This requires a [group session](./slim-session.md#group-session). A group +session is a channel shared among multiple participants and used to +send messages to everyone. When a new participant wants to join the channel, +they must be invited by the channel creator. -You can implement a moderator using the SLIM Python bindings to set up a group session and -configure all the required state to enable secure communication between -participants. The moderator can be part of a Python application and can either +The channel creator can be part of a Python application and can either actively participate in the communication process (possibly implementing some -of the application logic) or serve solely as a channel moderator. For a complete -example of how to use the moderator, see the [SLIM Group -Communication Tutorial](slim-group-tutorial.md). This section provides the basic -steps to follow, along with Python code snippets, for setting up a group. - -### Step 1: Create the Moderator - -Create the moderator by instantiating a -streaming bidirectional session, which initializes the corresponding state in -the SLIM session layer. In this example, communication between participants -will be encrypted end-to-end, as MLS is enabled. - - ```python - # Define the shared channel for group communication. - # This channel will be used by all members of the group to exchange messages. - shared_channel = PyName("agntcy", "namespace", "group_channel") - - # Create a new session. The group session is a bidirectional streaming session. - # Here is where we enable the MLS protocol for end-to-end encryption. - session_info = await slim_app.create_session( - PySessionConfiguration.Streaming( - PySessionDirection.BIDIRECTIONAL, - topic=shared_channel, # The channel ID for group communication. - moderator=True, # This session is created by the moderator. - max_retries=5, # Maximum number of retries for reliability. - timeout=datetime.timedelta(seconds=5), # Timeout for message delivery. - mls_enabled=True, # Enable MLS for end-to-end encryption. - ) - ) - ``` - -### Step 2: Invite Participants to the Channel - -The moderator now needs to invite -other participants to the channel. Note that not all participants need to be -added at the beginning; you can also add them later, even after communication -on the channel has already started. - - ```python - # Invite other members to the session. - for invitee in invitees: - print(f"Inviting {invitee}") - await slim_app.set_route(invitee) # Allow messages to be sent to the invitee. - await slim_app.invite( - session_info, invitee - ) # Send an invitation to the invitee. - ``` - -### Step 3: Listen for Invitations - -To receive an invitation to the channel, each -participant must listen for incoming messages. The moderator will send the -invitation directly to the participant's name, not via the channel, since the -participant does not yet know the channel name. - - ```python - async with participant_slim_app: - # Listen for new sessions opened by moderators - recv_session, _ = await participant_slim_app.receive() - - # Session is received, now we can read and write on the shared channel. - print(f"Received session: {recv_session.id}") - - # Receive messages from the session - recv_session, msg_rcv = await participant_slim_app.receive(session=recv_session.id) - - # Print the message - print(f"Received: {msg_rcv.decode()}") - ``` - -At this point, the group is set up and clients can start exchanging messages. -However, this configuration is not automatically reflected in the [SLIM -Controller](slim-controller.md) and must be reported manually. In particular, if -the SLIM network consists of multiple nodes, registration with the control plane -is mandatory to properly set up routes between nodes. We plan to automate this -process to make it easier for developers in the future. The group setup by the -moderator will work out of the box only if a single SLIM node is present in the -network. - -The next section describes how to register the newly created group -with the SLIM Controller and how to properly configure routes between nodes. - -## Creating Groups with the SLIM Controller +of the application logic) or serve solely as a channel moderator. -The controller API exposes operations to manage SLIM groups. You can use this API in client -applications to create and manage SLIM groups, add clients to -groups, and set routes between SLIM nodes. +This section provides the basic +steps to follow, along with Python code snippets, for setting up a group session. +The full code is available in the [group.py](https://github.com/agntcy/slim/blob/slim-v0.6.0/data-plane/python/bindings/examples/src/slim_bindings_examples/group.py) example in the SLIM repository. -You can generate gRPC API SDKs from the [schema -registry](https://buf.build/agntcy/slim/sdks/main:protobuf). +### Create the Channel -The following sections show example Python code fragments: - -### Creating a SLIM Channel +The channel can be created with a group session, +which initializes the corresponding state in the SLIM session layer. +In a group session, communication between participants can be encrypted +end-to-end, enabling MLS. ```python -import grpc -from controlplane.v1 import controlplane_pb2_grpc as controlplane_api -from controlplane.v1 import controlplane_pb2 - -# Create gRPC connection -channel = grpc.insecure_channel("localhost:50051") -client = controlplane_api.ControlPlaneServiceStub(channel) - -# Create channel request -create_channel_request = controlplane_pb2.CreateChannelRequest( - moderators=["agncty/namespace/moderator"] # Name of the moderator +created_session = await local_app.create_session( + slim_bindings.PySessionConfiguration.Group( # type: ignore # Build group session configuration + channel_name=chat_channel, # Logical group channel (PyName) all participants join; acts as group/topic identifier. + max_retries=5, # Max per-message resend attempts upon missing ack before reporting a delivery failure. + timeout=datetime.timedelta( + seconds=5 + ), # Ack / delivery wait window; after this duration a retry is triggered (until max_retries). + mls_enabled=enable_mls, # Enable Messaging Layer Security for end-to-end encrypted & authenticated group communication. + ) ) - -try: - response = client.CreateChannel(create_channel_request) - channel_id = response.channel_id - - if not response: - print("\nNo channels found") - return - - print(f"Channel created with ID: {channel_id}") - -except grpc.RpcError as e: - print(f"Request failed: {e}") -finally: - channel.close() ``` -### Adding Participants to a SLIM Group +### Invite Participants to the Channel + +Once the group session is created, new participants can be invited +to join. Not all participants need to be added at the beginning; you can add them later, even after communication has started. ```python -import grpc -from controlplane.v1 import controlplane_pb2_grpc as controlplane_api -from controlplane.v1 import controlplane_pb2 - -# Create gRPC connection -channel = grpc.insecure_channel("localhost:50051") -client = controlplane_api.ControlPlaneServiceStub(channel) - -# Add participant request -add_participant_request = controlplane_pb2.AddParticipantRequest( - participant_id="agncty/namespace/participant_1", - channel_id="agncty/namespace/group_channel" -) +# Invite each provided participant. Route is set before inviting to ensure +# outbound control messages can reach them. For more info see +# https://github.com/agntcy/slim/blob/slim-v0.6.0/data-plane/python/bindings/SESSION.md#invite-a-new-participant +for invite in invites: + invite_name = split_id(invite) + await local_app.set_route(invite_name) + await created_session.invite(invite_name) + print(f"{local} -> add {invite_name} to the group") +``` -try: - ack = client.AddParticipant(add_participant_request) +### Listen for New Sessions and Messages - print(f"ACK received for {ack.original_message_id}: success={ack.success}") +Participants that need to join the group start without a session and wait to be +invited. To wait for an invitation, the application calls `listen_for_session`. +When an invite message is received, a new session is created at the SLIM session layer, +and `listen_for_session` returns the metadata for the newly created session. -except grpc.RpcError as e: - print(f"Request failed: {e}") -finally: - channel.close() +```python +print_formatted_text("Waiting for session...", style=custom_style) +session = await local_app.listen_for_session() ``` -### Setting Routes Between SLIM Nodes +When a new session is available, the participant can start listening for messages: ```python -import grpc -from controlplane.v1 import controlplane_pb2_grpc as controlplane_api -from controlplane.v1 import controlplane_pb2 -from grpc_api import grpc_api_pb2 as grpcapi - -# Create gRPC connection -channel = grpc.insecure_channel("localhost:50051") -client = controlplane_api.ControlPlaneServiceStub(channel) - -try: - # Create connection to the target node - connection = grpcapi.Connection( - connection_id="http://127.0.0.1:46357", - config_data='{"endpoint": "http://127.0.0.1:46357"}' - ) - - create_connection_request = controlplane_pb2.CreateConnectionRequest( - connection=connection, - node_id="slim/0" - ) - - create_connection_response = client.CreateConnection(create_connection_request) - - if not create_connection_response.success: - raise Exception("Failed to create connection") - - connection_id = create_connection_response.connection_id - print(f"Connection created successfully with ID: {connection_id}") - - # Add subscription for a group to a SLIM node - subscription = grpcapi.Subscription( - component_0="agncty", - component_1="namespace", - component_2="group_channel", - connection_id=connection_id - ) - - create_subscription_request = controlplane_pb2.CreateSubscriptionRequest( - node_id="slim/0", - subscription=subscription - ) - - create_subscription_response = client.CreateSubscription(create_subscription_request) - - if not create_subscription_response.success: - raise Exception("Failed to create subscription") - - print(f"Subscription created successfully with ID: {create_subscription_response.subscription_id}") - -except grpc.RpcError as e: - print(f"gRPC error: {e}") -except Exception as e: - print(f"Error: {e}") -finally: - channel.close() +while True: + try: + # Await next inbound message from the group session. + # The returned parameters are a message context and the raw payload bytes. + # Check session.py for details on PyMessageContext contents. + ctx, payload = await session.get_message() + print_formatted_text( + f"{ctx.source_name} > {payload.decode()}", + style=custom_style, + ) + except asyncio.CancelledError: + # Graceful shutdown path (ctrl-c or program exit). + break + except Exception as e: + # Non-cancellation error; surface and exit the loop. + print_formatted_text(f"-> Error receiving message: {e}") + break ``` -## Identity Management - -To set up an MLS group, each client needs to have a valid identity. In the [SLIM -Group Communication Tutorial](slim-group-tutorial.md), we used a simple shared -secret to quickly set up identities for the clients. In a real-world scenario, -you would typically use a more secure method, such as tokens or certificates, to -authenticate clients and establish their identities. - -SLIM supports JWT (JSON Web Tokens) for identity management. Tokens can come -from an external identity provider or can be generated by the SLIM nodes -directly if you provide the necessary private key for signing the tokens and -public key for verification. Check the [Identity -Test](https://github.com/agntcy/slim/blob/main/data-plane/python-bindings/tests/test_identity.py) -for an example of how to use JWT tokens with SLIM if you have your own keys. - -If you are running your SLIM clients in a Kubernetes environment, a very common -approach to give an identity to each client is to use SPIRE -(https://spiffe.io/). SPIRE provides a way to issue [SPIFFE -IDs](https://spiffe.io/docs/latest/spiffe-about/spiffe-concepts/#spiffe-id) to -workloads, in the form of JWT tokens, which SLIM can then use to -authenticate clients. This allows for secure and scalable identity management in -distributed systems. +### Send Messages on a Channel -### Using SPIRE with SLIM +Each participant can also send messages at any time to the new session, and each message will be delivered to all participants connected to the same channel. -SLIM integrates well with SPIRE, as it allows you to use the JWT tokens -generated by SPIRE as client identities, and at the same time it can verify -these tokens using the key bundle provided by SPIRE. This section shows how to -use SPIRE with SLIM to manage client identities in a group communication -scenario. - -#### Installing SPIRE and Creating a ClusterSPIFFEID CRD - -To use SPIRE with SLIM, you need to have a SPIRE server and agent running in -your environment. You can follow the [SPIRE installation -guide](https://artifacthub.io/packages/helm/spiffe/spire#install-instructions) -to install SPIRE using Helm in a Kubernetes cluster, and to assign a SPIFFE ID -to each of your workloads. - -#### The SPIFFE Helper +```python +# Send message to the channel_name specified when creating the session. +# As the session is group, all participants will receive it. +# calling publish_with_destination on a group session will raise an error. +await shared_session_container[0].publish(user_input.encode()) +``` -The [SPIFFE Helper](https://github.com/spiffe/spiffe-helper) is a simple utility -for fetching X.509 SVID certificates and JWT tokens from the SPIFFE Workload -API. You can use it to obtain the JWT tokens for SLIM clients, which can then be -used to authenticate clients and establish their identities in the SLIM network. +## Creating Groups with the SLIM Controller -Since tokens are normally short-lived, the SPIFFE Helper can be used to -automatically refresh the tokens when they expire, ensuring that clients always -have a valid identity. +Another way to create a group in a SLIM network is to use the +[SLIM Controller](./slim-controller.md). For a complete description +on how to run it and the commands to use for the group creation and +management, please refer to the [Group Communication Tutorial](./slim-group-tutorial.md). +In this section, we list the `slimctl` commands to replicate +what we showed in the previous section. -You can run SPIFFE Helper as a sidecar container in the SLIM client pods, -ensuring that the client can always access a valid JWT token. The SLIM clients -can then use the token to authenticate themselves with the SLIM network. +### Create the Channel -#### Running SPIFFE Helper as a Sidecar Container +First of all, you need to run the applications that you want to add to the group. +At that point, you can create the group by specifying the first participant in the +group. This will assign the role of moderator (like in the Python bindings examples), +but all the invites/removals will be done using the Controller and no action needs to be +performed in the application. -To run the SPIFFE Helper as a sidecar container, you can add it to your SLIM -client's pod definition. First you need to create a ConfigMap with the SPIFFE -Helper configuration, which will be mounted as a volume in the pod. Here is an -example of how to create the ConfigMap: +To create the group, run: ```bash -cat <=0.3.6"] +dependencies = ["slim-bindings>=0.6.0"] ``` A tutorial on how to use the bindings in an application can be found in the [messaging layer documentation](./slim-data-plane.md). Otherwise examples are available in the -[SLIM Repository](https://github.com/agntcy/slim/tree/slim-v0.4.0/data-plane/python-bindings/examples/src/slim_bindings_examples). +[SLIM Repository](https://github.com/agntcy/slim/tree/slim-v0.6.0/data-plane/python/bindings/examples/src/slim_bindings_examples). ### Slimctl `slimctl` is a command-line tool for managing SLIM Nodes and Controllers. It can be downloaded from the [releases -page](https://github.com/agntcy/slim/releases/tag/slimctl-v0.2.1) in the SLIM repo. +page](https://github.com/agntcy/slim/releases/tag/slimctl-v0.6.0) in the SLIM repo. #### Installation @@ -160,14 +162,14 @@ Choose the appropriate installation method for your operating system: === "macOS (Apple Silicon)" ```bash - curl -LO https://github.com/agntcy/slim/releases/download/slimctl-v0.2.1/slimctl-darwin-arm64 + curl -LO https://github.com/agntcy/slim/releases/download/slimctl-v0.6.0/slimctl-darwin-arm64 sudo mv slimctl-darwin-arm64 /usr/local/bin/slimctl sudo chmod +x /usr/local/bin/slimctl ``` !!! note "macOS Security" You may need to allow the binary to run if it's blocked by Gatekeeper: - + ```bash sudo xattr -rd com.apple.quarantine /usr/local/bin/slimctl ``` @@ -177,7 +179,7 @@ Choose the appropriate installation method for your operating system: === "Linux (AMD64)" ```bash - curl -LO https://github.com/agntcy/slim/releases/download/slimctl-v0.2.1/slimctl-linux-amd64 + curl -LO https://github.com/agntcy/slim/releases/download/slimctl-v0.6.0/slimctl-linux-amd64 sudo mv slimctl-linux-amd64 /usr/local/bin/slimctl sudo chmod +x /usr/local/bin/slimctl ``` diff --git a/docs/messaging/slim-mcp.md b/docs/messaging/slim-mcp.md index 4fbe5eb..1ea3a7d 100644 --- a/docs/messaging/slim-mcp.md +++ b/docs/messaging/slim-mcp.md @@ -30,10 +30,10 @@ architecture. In this section of the tutorial, we implement and deploy two sample applications: -- A [LlamaIndex - agent](https://github.com/agntcy/slim/tree/main/data-plane/integrations/mcp/slim-mcp/examples/llamaindex-time-agent) that communicates with an MCP server over SLIM to perform time queries and timezone conversions. +- A [LlamaIndex agent](https://github.com/agntcy/slim/tree/slim-v0.6.0/data-plane/python/integrations/slim-mcp/slim_mcp/examples/llamaindex_time_agent) +that communicates with an MCP server over SLIM to perform time queries and timezone conversions. - An [MCP time - server](https://github.com/agntcy/slim/tree/main/data-plane/integrations/mcp/slim-mcp/examples/mcp-server-time) that implements SLIM as its transport protocol and processes requests from the LlamaIndex agent. + server](https://github.com/agntcy/slim/tree/slim-v0.6.0/data-plane/python/integrations/slim-mcp/slim_mcp/examples/mcp_server_time) that implements SLIM as its transport protocol and processes requests from the LlamaIndex agent. ### Prerequisites @@ -63,7 +63,7 @@ runtime: services: slim/0: - pubsub: + dataplane: servers: - endpoint: "0.0.0.0:46357" tls: @@ -71,10 +71,10 @@ services: clients: [] controller: - server: - endpoint: "0.0.0.0:46358" - tls: - insecure: true + servers: + - endpoint: "0.0.0.0:46358" + tls: + insecure: true EOF ``` @@ -1041,19 +1041,19 @@ local proxy instance: ```bash cat << EOF > ./config-proxy.yaml # SLIM-MCP Proxy Configuration - + # Tracing settings for log visibility tracing: log_level: info display_thread_names: true display_thread_ids: true - + # Runtime configuration runtime: n_cores: 0 thread_name: "slim-data-plane" drain_timeout: 10s - + # Service configuration for connecting to the SLIM node services: slim/0: diff --git a/docs/messaging/slim-rpc.md b/docs/messaging/slim-rpc.md index 7549d2e..546dd92 100644 --- a/docs/messaging/slim-rpc.md +++ b/docs/messaging/slim-rpc.md @@ -24,7 +24,7 @@ documentation](./slim-slimrpc-compiler.md). In SLIMRPC, each service and its individual RPC handlers are assigned a SLIM name, facilitating efficient message routing and processing. Consider the [example -protobuf](https://github.com/agntcy/slim/blob/main/data-plane/python/integrations/slimrpc/slimrpc/examples/simple/example.proto) +protobuf](https://github.com/agntcy/slim/blob/slim-v0.6.0/data-plane/python/integrations/slimrpc/slimrpc/examples/simple/example.proto) definition, which defines four distinct services: ```proto @@ -95,7 +95,7 @@ standard gRPC. This section provides a detailed walkthrough of a basic SLIMRPC client-server interaction, leveraging the simple example provided in -[example](https://github.com/agntcy/slim/tree/main/data-plane/python/integrations/slimrpc/slimrpc/examples/simple) +[example](https://github.com/agntcy/slim/tree/slim-v0.6.0/data-plane/python/integrations/slimrpc/slimrpc/examples/simple) folder. ### Generated Code @@ -104,9 +104,9 @@ The foundation of this example is the `example.proto` file, which is a standard Protocol Buffers definition file. This file is compiled using the [SLIMRPC compiler](./slim-slimrpc-compiler.md) to generate the necessary Python stub code. The generated code is available in two files: -[example_pb2.py](https://github.com/agntcy/slim/blob/main/data-plane/python/integrations/slimrpc/slimrpc/examples/simple/types/example_pb2.py) +[example_pb2.py](https://github.com/agntcy/slim/blob/slim-v0.6.0/data-plane/python/integrations/slimrpc/slimrpc/examples/simple/types/example_pb2.py) and -[example_pb2_slimrpc.py](https://github.com/agntcy/slim/blob/main/data-plane/python/integrations/slimrpc/slimrpc/examples/simple/types/example_pb2_slimrpc.py). +[example_pb2_slimrpc.py](https://github.com/agntcy/slim/blob/slim-v0.6.0/data-plane/python/integrations/slimrpc/slimrpc/examples/simple/types/example_pb2_slimrpc.py). Specifically, `example_pb2_slimrpc.py` contains the SLIMRPC-specific stubs for both client and server implementations. Below are the key classes and functions generated by the compiler: @@ -216,7 +216,7 @@ def add_TestServicer_to_server(servicer, server: slimrpc.Server): ### Server implementation The server-side logic is defined in -[server.py](https://github.com/agntcy/slim/blob/main/data-plane/python/integrations/slimrpc/slimrpc/examples/simple/server.py). +[server.py](https://github.com/agntcy/slim/blob/slim-v0.6.0/data-plane/python/integrations/slimrpc/slimrpc/examples/simple/server.py). Similar to standard gRPC implementations, the core service functionality is provided by the TestService class, which inherits from TestServicer (as introduced in the previous section). This class contains the concrete @@ -287,7 +287,7 @@ disables TLS for simplicity in this example. The `shared_secret` parameter is used for initializing the Message Layer Security (MLS) protocol. Note that using a hardcoded shared_secret like "my_shared_secret" is not recommended, please refer to [the documentation for proper MLS -configuration](./slim-group.md#identity-management). +configuration](./slim-group.md). Finally, the add_TestServicer_to_server function is called to register the implemented TestService with the SLIMRPC server, making its RPC methods @@ -304,7 +304,7 @@ available. ### Client implementation The client-side implementation, found in -[client.py](https://github.com/agntcy/slim/blob/main/data-plane/python/integrations/slimrpc/slimrpc/examples/simple/client.py), +[client.py](https://github.com/agntcy/slim/blob/slim-v0.6.0/data-plane/python/integrations/slimrpc/slimrpc/examples/simple/client.py), largely mirrors the structure of a standard gRPC client. The primary distinction and SLIM-specific aspect lies in the creation of the SLIMRPC channel: @@ -355,7 +355,7 @@ details, as these aspects are handled automatically by SLIMRPC and SLIM. All RPC services underneath utilize a sticky point-to-point session. The SLIM session creation is implemented in inside SLIMRPC in -[channel.py](https://github.com/agntcy/slim/blob/main/data-plane/python/integrations/slimrpc/slimrpc/channel.py): +[channel.py](https://github.com/agntcy/slim/blob/slim-v0.6.0/data-plane/python/integrations/slimrpc/slimrpc/channel.py): ```python # Create a session diff --git a/docs/messaging/slim-session.md b/docs/messaging/slim-session.md new file mode 100644 index 0000000..49d03be --- /dev/null +++ b/docs/messaging/slim-session.md @@ -0,0 +1,343 @@ +# SLIM Sessions + +This document explains the SLIM session layer and the two supported session +types. It helps you understand the two session interfaces, reliability, and security trade‑offs. + +The SLIM repository ships with practical, runnable [examples](https://github.com/agntcy/slim/tree/slim-v0.6.0/data-plane/python/bindings/examples) that demonstrate how to create sessions and exchange messages between applications using the Python bindings. + +## Point-to-Point Session + +The point-to-point session enables point-to-point communication with a specific +instance. This session +performs a discovery phase to bind to one instance and all subsequent traffic in +the session targets that same endpoint. With reliability enabled, each message in +the session must be acked. + +If MLS is enabled, the point-to-point session establishes a two‑member MLS group after +discovery. This mirrors the Group flow but with only two participants +(see [Group Session](#group-session)). + +The diagram below illustrates a point-to-point session from App-A to agntcy/ns/App-B. +App-A first discovers an available instance (App-B/1), then performs the MLS +setup, and finally sends multiple messages to that same instance, each followed +by an Ack. If MLS is not enabled, the MLS setup is skipped. + +```mermaid +sequenceDiagram + autonumber + + participant App-A + participant SLIM Node + participant App-B/1 + participant App-B/2 + + Note over App-A,App-B/2: Discovery + App-A->>App-A: Init MLS state + App-A->>SLIM Node: Discover agntcy/ns/App-B + SLIM Node->>App-B/1: Discover agntcy/ns/App-B + App-B/1->>SLIM Node: Discover Reply (agntcy/ns/App-B/1) + SLIM Node->>App-A: Discover Reply (agntcy/ns/App-B/1) + + Note over App-A,App-B/2: Invite + App-A->>SLIM Node: Invite agntcy/ns/App-B/1 + SLIM Node->>App-B/1: Invite agntcy/ns/App-B/1 + App-B/1->>App-B/1: Create new point-to-point Session + App-B/1->>SLIM Node: Invite Reply (MLS key package) + SLIM Node->>App-A: Invite Reply (MLS key package) + App-A->>App-A: Update MLS state + + Note over App-A,App-B/2: MLS setup + App-A->>SLIM Node: MLS Welcome agntcy/ns/App-B/1 + SLIM Node->>App-B/1: MLS Welcome agntcy/ns/App-B/1 + App-B/1->>App-B/1: Init MLS state + App-B/1->>SLIM Node: Ack(MLS Welcome) + SLIM Node->>App-A: Ack(MLS Welcome) + + Note over App-A,App-B/2: Message exchange + App-A->>SLIM Node: Message to agntcy/ns/App-B/1 + SLIM Node->>App-B/1: Message to agntcy/ns/App-B/1 + App-B/1->>SLIM Node: Ack + SLIM Node->>App-A: Ack + + App-A->>SLIM Node: Message to agntcy/ns/App-B/1 + SLIM Node->>App-B/1: Message to agntcy/ns/App-B/1 + App-B/1->>SLIM Node: Ack + SLIM Node->>App-A: Ack +``` + +### Create a Point-to-Point Session + +Using the SLIM Python bindings, you can create a point-to-point session as follows: + +```python +# Assume local_app is an initialized application instance +session = await local_app.create_session( + slim_bindings.PySessionConfiguration.PointToPoint( + peer_name=remote_name, + max_retries=5, + timeout=datetime.timedelta(seconds=5), + mls_enabled=True, # Enable MLS for end-to-end security + ) +) +``` + +Parameters: + +* `peer_name` (required, PyName): Identifier of the remote participant + instance. +* `max_retries` (optional, int): Retry attempts per message if Ack missing. +* `timeout` (optional, timedelta): Wait per attempt for an Ack before retry. + If `timeout` is not set, the session is best‑effort. +* `mls_enabled` (optional, bool): Enable end‑to‑end encryption (MLS). + +### Sending and Replying in a Point-to-Point Session + +As the point-to-point session is bound to a single remote instance after discovery, +outbound messages use the implicit destination. Use `publish` for normal sends +and `publish_to` to reply using a previously received message context. + +This example shows how to send and reply in a point-to-point session: + +```python +# Send a message using publish it will reach the endpoint +# specified and the session creation +await session.publish(b"hello") + +# Await reply from remote (pattern depends on your control loop) +msg_ctx, payload = await session.get_message() +print(payload.decode()) + +# Send a correlated response back (echo style) +# The message will be sent according to the info in msg_ctx +await session.publish_to(msg_ctx, payload) +``` + +### Point-to-Point Example + +This [example](https://github.com/agntcy/slim/blob/slim-v0.6.0/data-plane/python/bindings/examples/src/slim_bindings_examples/point_to_point.py) walks through the creation of a point-to-point session. When running the point-to-point example multiple times, the session binds to different running instances, while the message stream always sticks to the same endpoint. + +The example demonstrates how to publish messages, enable reliability, and enable MLS for end‑to‑end security. The associated [README](https://github.com/agntcy/slim/blob/slim-v0.6.0/data-plane/python/bindings/examples/src/slim_bindings_examples/README_point_to_point.md) shows more information and how to run the example using the Taskfile provided in the repository. + +## Group Session + +The Group session allows many-to-many communication on a named channel. Each +message is delivered to all participants connected to the same session. + +The creator of the channel can act as a moderator, meaning that it can add or +remove participants from the session. Moderation can be built into your application +or delegated to a separate control service or the SLIM control plane. + +Below are examples using the latest Python bindings, along with explanations of +what happens inside the session layer when a participant is added or removed +from the channel (see [Group management](./slim-group.md)). + +### Create a Group Session + +To create a group session, you need to configure the session with a topic +name and specify reliability and security settings. Here is an +example: + +```python +# Assume local_app is an initialized application instance +session = await local_app.create_session( + slim_bindings.PySessionConfiguration.Group( + channel_name=chat_topic, + max_retries=5, + timeout=datetime.timedelta(seconds=5), + mls_enabled=True, + ) +) +``` + +Parameters: + +* `topic` (required, PyName): Channel/Topic name where all the messages are + delivered. +* `max_retries` (optional, int): Retry attempts for missing Acks. +* `timeout` (optional, timedelta): Wait per attempt for Ack before retry. + If `timeout` is not set the session is best‑effort. +* `mls_enabled` (optional, bool): Enable secure group MLS messaging. + +### Sending and Replying in a Group Session + +In a Group, the session targets a channel: all sends are delivered to all the current +participants. Use `publish` to send a message to all the participants in the group. + +```python +# Broadcast to the channel +await session.publish(b"hello") + +# Handle inbound messages +msg_ctx, data = await session.get_message() +print("channel received:", data.decode()) +``` + +### Invite a New Participant + +The creator of the session can invite a new participant to the channel using the `invite` +method after creating the session. + +```python +# After creating the session: +invite_name = slim_bindings.PyName("agntcy", "ns", "participant") +await local_app.set_route(invite_name) +await session.invite(invite_name) +``` + +Parameters: + +* `invite_name` (PyName): Identifier of the participant to add. + +Notice the `await local_app.set_route(invite_name)` command before the invite. +This instructs SLIM on how to forward a message with the specified name. +This has to be done by the application for every invite. + +When a moderator wants to add a new participant (e.g., an instance of App-C) to +a group session, the following steps occur. All the steps are visualized in +the diagram below: + +1. **Discovery Phase:** The moderator initiates a discovery request to find a + running instance of the desired application (App-C). This request is sent to + the SLIM Node, which forwards it via anycast to one of the App-C instances. + In the example, the message is forwarded to App-C/1 that replies with its + full identifier. The SLIM Node relays this reply back to the moderator. + +2. **Invitation:** The moderator sends an invite message for the discovered + instance (App-C/1) to the SLIM Node, which forwards it to App-C/1. Upon + receiving the invite, App-C/1 creates a new group session, subscribes to + the channel, and replies with its MLS (Messaging Layer Security) key + package. This reply is routed back to the moderator. + +3. **MLS State Update:** The moderator initiates an MLS commit to add App-C/1 + to the secure group. The message is sent using the channel name and so the + SLIM Node distributes this commit to all current participants (App-B/2 and + App-A/1), who update their MLS state and acknowledge the commit. The + moderator collects all acknowledgments. Once all acknowledgments are + received, the moderator sends an MLS Welcome message to App-C/1. App-C/1 + initializes its MLS state and acknowledges receipt. At the end of this + process, all participants (including the new one) share a secure group state + and can exchange encrypted messages on the group channel. If MLS is + disabled, the MLS state update and welcome step are skipped. + +```mermaid +sequenceDiagram + autonumber + + participant Moderator + participant SLIM Node + participant App-C/1 + participant App-C/2 + participant App-B/2 + participant App-A/1 + + Note over Moderator,App-A/1: Discovery + Moderator->>SLIM Node: Discover agntcy/ns/App-C + SLIM Node->>App-C/1: Discover agntcy/ns/App-C + App-C/1->>SLIM Node: Discover Reply from agntcy/ns/App-C/1 + SLIM Node->>Moderator: Discover Reply from agntcy/ns/App-C/1 + + Note over Moderator,App-A/1: Invite + Moderator->>SLIM Node: Invite agntcy/ns/App-C/1 + SLIM Node->>App-C/1: Invite agntcy/ns/App-C/1 + App-C/1->>App-C/1: Create new Group session + App-C/1->>SLIM Node: Subscribe to Channel + App-C/1->>SLIM Node: Invite Reply (MLS key package) + SLIM Node->>Moderator: Invite Reply (MLS key package) + Moderator->>Moderator: Update MLS state + + Note over Moderator,App-A/1: MLS State Update + Moderator->>SLIM Node: MLS commit (Add agntcy/ns/App-C/1) to Channel + par Handle MLS commit on App-C/1 + SLIM Node->>App-B/2: MLS commit (Add agntcy/ns/App-C/1) to Channel + App-B/2->>App-B/2: Update MLS state + App-B/2->>SLIM Node: Ack(MLS Commit) + SLIM Node->>Moderator: Ack(MLS Commit) + and Handle MLS commit on App-A/1 + SLIM Node->>App-A/1: MLS commit (Add agntcy/ns/App-C/1) to Channel + App-A/1->>App-A/1: Update MLS state + App-A/1->>SLIM Node: Ack(MLS Commit) + SLIM Node->>Moderator: Ack(MLS Commit) + end + Moderator->>SLIM Node: MLS Welcome agntcy/ns/App-C/1 + SLIM Node->>App-C/1: MLS Welcome agntcy/ns/App-C/1 + App-C/1->>App-C/1: Init MLS state + App-C/1->>SLIM Node: Ack(MLS Welcome) + SLIM Node->>Moderator: Ack(MLS Welcome) +``` + +### Remove a Participant + +A moderator can remove a participant from the channel using the `remove` +method after creating the session. + +This example shows how to remove a participant from a group session: + +```python +# To remove a participant from the session: +remove_name = slim_bindings.PyName("agntcy", "ns", "participant") +await session.remove(remove_name) +``` + +Parameter: + +* `remove_name` (PyName): Identifier of the participant to remove. + +When a moderator wants to remove a participant (e.g., App-C/1) from a group +session, the following steps occur. All the steps are visualized in the diagram +below: + +1. **MLS State Update:** The moderator creates an MLS commit to remove App-C/1 + from the secure group. This commit is sent to the group channel and the + SLIM Node distributes it to all current participants (App-C/1, App-B/2, and + App-A/1). Each participant updates its MLS state and acknowledges the + commit. The moderator collects all acknowledgments. In case the MLS is + disabled, this step is not executed. + +2. **Removal:** After the MLS state is updated, the moderator sends a remove + message to App-C/1. Upon receiving the remove message, App-C/1 unsubscribes + from the channel, deletes its group session, and replies with a + confirmation. The SLIM Node relays this confirmation back to the moderator. + At the end of this process, App-C/1 is no longer a member of the group + and cannot send or receive messages on the channel. + +```mermaid +sequenceDiagram + autonumber + + participant Moderator + participant SLIM Node + participant App-C/1 + participant App-B/2 + participant App-A/1 + + Note over Moderator,App-A/1: MLS State Update + Moderator->>SLIM Node: MLS commit (Remove agntcy/ns/App-C/1) to Channel + par Handle MLS commit on App-C/1 + SLIM Node->>App-C/1: MLS commit (Remove agntcy/ns/App-C/1) to Channel + App-C/1->>App-C/1: Update MLS state + App-C/1->>SLIM Node: Ack(MLS Commit) + SLIM Node->>Moderator: Ack(MLS Commit) + and Handle MLS commit on App-B/2 + SLIM Node->>App-B/2: MLS commit (Remove agntcy/ns/App-C/1) to Channel + App-B/2->>App-B/2: Update MLS state + App-B/2->>SLIM Node: Ack(MLS Commit) + SLIM Node->>Moderator: Ack(MLS Commit) + and Handle MLS commit on App-A/1 + SLIM Node->>App-A/1: MLS commit (Remove agntcy/ns/App-C/1) to Channel + App-A/1->>App-A/1: Update MLS state + App-A/1->>SLIM Node: Ack(MLS Commit) + SLIM Node->>Moderator: Ack(MLS Commit) + end + + Note over Moderator,App-A/1: Remove + Moderator->>SLIM Node: Remove agntcy/ns/App-C/1 + SLIM Node->>App-C/1: Remove agntcy/ns/App-C/1 + App-C/1->>SLIM Node: Unsubscribe from Channel + App-C/1->>App-C/1: Remove Group session + App-C/1->>SLIM Node: Remove Reply + SLIM Node->>Moderator: Remove Reply +``` + +### Group Example + +This [example](https://github.com/agntcy/slim/blob/slim-v0.6.0/data-plane/python/bindings/examples/src/slim_bindings_examples/group.py) demonstrates how to create a group session, invite participants, and (if enabled) establish an MLS group for end-to-end encryption. It also shows how to broadcast messages to all current members and handle inbound group messages. The associated [README](https://github.com/agntcy/slim/blob/slim-v0.6.0/data-plane/python/bindings/examples/src/slim_bindings_examples/README_group.md) shows more information and how to run the example using the Taskfile. diff --git a/mkdocs/mkdocs.yml b/mkdocs/mkdocs.yml index 5b13386..c46c62c 100644 --- a/mkdocs/mkdocs.yml +++ b/mkdocs/mkdocs.yml @@ -14,6 +14,9 @@ extra_css: markdown_extensions: - admonition + - attr_list + - def_list + - md_in_html - pymdownx.details - pymdownx.superfences - pymdownx.tabbed: @@ -134,4 +137,4 @@ theme: code: Roboto Mono favicon: assets/favicon.ico logo_light: assets/img/logo-light.svg - logo_dark: assets/img/logo-dark.svg \ No newline at end of file + logo_dark: assets/img/logo-dark.svg