Skip to content

RFC: export a graph as a Testimony Record, so a deployment can evidence what it knew and where it came from #1834

Description

@troybrandonc-bit

Opening this as an RFC per CONTRIBUTING.md before any PR.

The problem this solves for Graphiti users

Teams putting Graphiti behind decisions in regulated contexts get asked a question it currently has no portable answer to: how do you evidence what the system knew at a given moment, and where each fact came from?

Graphiti knows all of this internally. What it cannot do today is hand somebody a self-contained artifact that a reviewer can check without access to the graph, the driver, or Graphiti itself.

Why Graphiti is unusually well placed

I spent the last month reading eight agent memory and framework projects to see what each records about its own decisions. Graphiti came out the strongest of them, and mostly for things bi-temporality required for unrelated reasons:

  • EntityEdge.episodes already names the episodes a fact was extracted from, and episodes retain content rather than discarding it after extraction. Most systems in this class cannot say where a stored fact came from at all.
  • resolve_edge_contradictions stamps invalid_at and expired_at and keeps the edge, and generates embeddings for invalidated edges so they stay retrievable. Both sides of a disagreement survive.
  • created_at vs valid_at/invalid_at is already the distinction between when something was written and when it held.

So this is a translation of things already stored, not a change to how anything works.

Proposal

One new module, graphiti_core/utils/testimony_export.py, exposing:

async def export_testimony(graphiti, group_ids: list[str], excerpts: bool = False) -> str

Returns JSON Lines. No new dependency, stdlib plus graphiti_core, no existing code path touched.

Graphiti Record entry
EpisodicNode evidence, with a sha256 of content
EntityEdge belief, citing ev_{episode_uuid} for each entry in episodes
an invalidated/replacing pair conflict naming both sides, resolution: null
the export itself integrity, scheme: replay

Episode content is not copied into the record by default. The digest lets a reader show a cited episode is unchanged without the artifact becoming a second copy of what people said. excerpts=True opts in.

The record declares acts: false, since Graphiti records and derives but does not actuate. Nothing invents decision or approval entries.

The design question I actually want input on

Conflict detection is inferred, and I would rather have it exact.

resolve_edge_contradictions sets the older edge's invalid_at to the newer edge's valid_at, so I pair edges on the same source node and the same (name, target_uuid) where that equality holds:

if a.invalid_at == b.valid_at:
    pairs.append(tuple(sorted((a.uuid, b.uuid))))

This can miss a contradiction whose timestamps were adjusted afterwards, and can pair two edges whose intervals abut for unrelated reasons.

Would you consider a first-class field recording which edge invalidated another? It would make this exact, and it seems useful independently of any export: right now an edge invalidated by a contradiction and one whose validity simply lapsed are indistinguishable after the fact.

If you would rather not add a field, I am happy to keep the inference and document its limits, which is what the current implementation does.

A second, smaller question: I use f'{edge.name}::{edge.target_node_uuid}' as the proposition token, because two edges from one node with the same relation name to different targets are different claims and collapsing them would manufacture contradictions. If there is a more idiomatic stable identifier for a claim, I would rather use yours.

What this is

The Testimony Record is an open format for what an automated system believed, what contradicted it, and what it did. Four conformance levels and a validator that is one standard-library file you can copy and run yourself. Specification text CC BY 4.0, schema and tools MIT. Free to implement, nothing to sign, and no dependency on anything of mine.

I maintain it, so treat my view of its usefulness accordingly. It is also why I would rather be told the mapping is wrong here than have it merged politely.

Status

Implemented, with 8 unit tests using unittest.mock against the real models and no database. ruff check and ruff format --check clean at your settings. The export validates at the format's top level for a record-only system.

Happy to open the PR against this issue if the design is acceptable, or to change the approach first. I can also just leave the design here if you would prefer to implement it yourselves.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions