Skip to content

Commit e9136eb

Browse files
flyersworderclaude
andauthored
feat: OssieSource reads Apache Ossie semantic models (v0.42.0) (#64)
* feat: OssieSource reads Apache Ossie semantic models (v0.42.0) Apache Ossie (incubating) — formerly Open Semantic Interchange — is the vendor-neutral spec for exchanging semantic models across analytics, AI, and BI platforms. It standardises what a metric *is*; this library enforces what an agent may *do* with it, so Ossie is a fourth input to the SemanticSource protocol rather than a competitor to enforcement. Four decisions define the adapter, each a place where the easy choice is a silent correctness bug: - Table keys drop the database qualifier. Relationship endpoints are "schema.table.column" and build_relationship_index recovers the table with one rsplit; a three-part key would break that contract. - Cardinality is derived from keys, not defaulted. Ossie never writes the join type down. - Composite joins are skipped with a warning, not split into one edge per column pair — the planner walks those edges, and each pair alone is wrong. Matches the CubeSource precedent. - Dialect is an opaque string, never checked against the enum: the accepted expression-language proposal adds Ossie_SQL_2026 and makes it the default. Governance vocabulary (owners, review dates, tiers, decompositions, drill_by, metric impacts) has no home in the spec — every $def sets additionalProperties: false — so it rides in custom_extensions under the AGENTIC_DATA_CONTRACTS vendor name and is revalidated on load. Every ai_context and every foreign vendor block is carried through get_extras() without interpretation. Also moves _parse_date to base.parse_review_date, now that two sources need it, and upgrades all locked dependencies. The pinned contract_digest guard still holds, so published bytes are unaffected. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: address review findings on OssieSource Ten findings from the branch review; eight were real defects. The one that mattered: cardinality was derived from the `from` side only, on the premise that Ossie's `to` is the one side "by construction". Nothing in the spec validates that, and RelationshipChecker._check_fan_out fires only on one_to_many — so a backwards-declared join read as one_to_one and silently disabled the row-multiplication warning on an aggregate. Now derived from both sides, with the spec's declaration honoured when the `to` dataset declares no keys at all (absence of a key is not evidence of fan-out, and downgrading every key-less model would flood the checker). Also: - Scalar tier/domains/filters in the vendor block were shredded by list(): "gold" became ['g','o','l','d']. Promoted as YamlSource does. - get_extras() always emitted ossie_custom_extensions, even empty. Verified to break freeze->rehydrate under expected_extras, and it added a noise key to every Ossie contract's canonical bytes. - Foreign extensions and ai_context were keyed globally but collected per model, so a two-model file silently lost the first of two same-named entries. Both are now keyed by model name first. - Extras were not JSON-normalized; a YAML-native date in an ai_context could reach canonical bytes unconverted. _normalize_extras/_jsonify promoted to base.jsonify_extras and shared. - json.loads on a YAML mapping payload raised TypeError and produced a "not valid JSON" warning about a non-problem. - An unresolvable dataset source vanished silently; a typo'd source turns drill_by column validation off rather than failing it. - expected_extras on a non-yaml source was silently dropped. Not fixed, documented instead: two models whose sources differ only by database qualifier still collide on the table key. That is inherent to the two-part key contract Relationship endpoints require, and it warns. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 5633cde commit e9136eb

15 files changed

Lines changed: 2210 additions & 601 deletions

File tree

CHANGELOG.md

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,48 @@
22

33
All notable changes to this project will be documented in this file.
44

5+
## [0.42.0] - 2026-08-16
6+
7+
### Added
8+
9+
- **`OssieSource` — read semantics from an Apache Ossie (incubating) model.** [Apache Ossie](https://ossie.apache.org/), formerly Open Semantic Interchange, is the vendor-neutral spec for exchanging semantic models across analytics, AI, and BI platforms; it entered the Apache Incubator with 50+ participating organizations behind it. Declare it like any other source:
10+
11+
```yaml
12+
semantic:
13+
source:
14+
type: ossie
15+
path: "./semantic/model.yml"
16+
```
17+
18+
The strategic read is that Ossie standardises what a metric *is*, while this library enforces what an agent may *do* with it. Nothing in the spec or its roadmap validates a generated query against a policy — their `validation/` checks models against the JSON Schema, and the planned reference engine *compiles* semantic queries into SQL. So Ossie is a fourth input to `SemanticSource`, not a competitor to the enforcement layer, and the protocol was built for exactly this.
19+
20+
Four decisions define the adapter, each one a place where the easy choice is a silent correctness bug:
21+
22+
- **Table keys drop the database qualifier.** Ossie sources are `database.schema.table`; our keys are two-part because `Relationship` endpoints are `schema.table.column` and `build_relationship_index` recovers the table with a single `rsplit`. A three-part key would make every endpoint four deep. Collisions are logged; query-backed datasets register no table.
23+
- **Cardinality is derived from both sides.** Ossie never writes the join type down — it is implicit in the keys. The spec documents `to` as the one side, but nothing validates it, and trusting it is not free: `RelationshipChecker._check_fan_out` fires only on `one_to_many`, so reading a backwards-declared join as `one_to_one` silently disables the row-multiplication warning on an aggregate. The type is read off `(from_columns are a key, to_columns are a key)`. Keys are optional in Ossie and their absence is not evidence of fan-out, so when the `to` dataset declares no keys at all the spec's declaration stands — otherwise every key-less model would flood the checker with false positives.
24+
- **Composite joins are skipped, not split.** Ossie carries parallel `from_columns`/`to_columns` lists; our `Relationship` has single-column endpoints. One edge per column pair would assert two joins that are each individually wrong, and the join planner walks those edges. The skip is logged with the relationship's name, matching the precedent `CubeSource` set for Cube's `AND`-chained joins.
25+
- **Dialect is an opaque string.** Resolution is deterministic as the spec requires — caller's choice, then `ANSI_SQL`, then `Ossie_SQL_2026`, then the first declared entry — but never validated against the enum. The accepted expression-language proposal adds `Ossie_SQL_2026` and makes it the default, so a closed check would break on the next spec bump for no benefit.
26+
27+
- **Governance vocabulary round-trips through Ossie's `custom_extensions`.** Ownership, review dates, tiers, domains, `decompositions`, `drill_by`, and the metric-impact graph have no home in the Ossie spec — every `$def` in `osi-schema.json` sets `additionalProperties: false`, so they cannot be smuggled in as extra keys. They ride in the spec's own escape hatch under the `AGENTIC_DATA_CONTRACTS` vendor name, whose `data` is a JSON *string* per the spec. Restored `decompositions` / `drill_by` pass through the same `validate_decompositions` / `validate_drill_by` as `YamlSource`, so a bad identity still fails loudly at load.
28+
29+
This makes `OssieSource` the second source after `YamlSource` to populate `business_owner` / `operational_owner` / `last_reviewed` and `decompositions` / `drill_by`; dbt and Cube still leave them unset.
30+
31+
- **`ai_context` and foreign vendor blocks are carried, not interpreted.** Every `ai_context` in the model (Ossie's AI-grounding channel — instructions, synonyms, example questions, in either its string or object form) reaches `get_extras()["ossie_ai_context"]`, and every non-`AGENTIC_DATA_CONTRACTS` vendor block reaches `get_extras()["ossie_custom_extensions"]`. This is the boundary `YamlSource.get_extras` already draws: the framework carries extras and, on request, places them in the prompt, but never interprets, validates, indexes, or computes over them. In particular `search_metrics` still matches on name and description only — Ossie synonyms do not silently change retrieval.
32+
33+
A foreign vendor's malformed JSON payload is carried verbatim and logged rather than raised on. Another vendor's typo must not stop this library from enforcing a contract.
34+
35+
Both sections are keyed by **semantic-model name first**. Ossie's top level is a list and it namespaces entity names per model, so a file with two models — each declaring a `customer` dataset, or each carrying a block from the same vendor — would otherwise silently lose the first. Both are omitted entirely when empty rather than emitted as `{}`: extras ride into the contract's inline snapshot and its canonical bytes, so an always-present empty dict would add a noise key to every Ossie contract's digest and would trip a contract declaring `expected_extras` on rehydrate.
36+
37+
- **`expected_extras` on a non-`yaml` source now warns instead of being silently dropped.** The policy is threaded only into `YamlSource`. An Ossie source *does* produce extras, so declaring it there looks like it should work; without a warning the author believes strict mode is on, and the mismatch surfaces much later as a load failure on a frozen contract.
38+
39+
### Changed
40+
41+
- **Shared parser helpers promoted to `semantic/base.py`.** `_normalize_extras` / `_jsonify` became `jsonify_extras`. Extras ride into `contract_canonical_bytes` through `json.dumps`, so every source that carries them needs the same date coercion and JSON-safety check — a YAML-native date inside an Ossie `ai_context` would otherwise reach a published ARD attestation unconverted. The error now names the parser that raised it.
42+
43+
- **`_parse_date` became `parse_review_date`.** Two sources now parse review dates, and the helper's whole reason for existing is a trap worth stating once: `datetime` must be checked before `date` because it subclasses it, or downstream `date - datetime` staleness arithmetic raises `TypeError`. A second copy would have been a second chance to get that ordering wrong. It sits alongside the other shared helpers (`build_relationship_index`, `fuzzy_search_metrics`, the validators) rather than being reached into privately from a sibling module.
44+
45+
- **All locked dependencies upgraded.** Notably `sqlglot` 30.14.0 → 30.17.0 (Layer 1 static analysis), `pydantic-ai-slim` 2.22.0 → 2.31.0, `starlette` 1.3.1 → 1.6.0, and `xxhash` 3.8.1 → 4.0.0. The pinned `contract_digest` guard in `tests/test_ard/test_catalog_entry.py` still holds, so published contract bytes and every existing ARD attestation are unaffected.
46+
547
## [0.41.1] - 2026-08-11
648

749
### Fixed

README.md

Lines changed: 75 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ You teach agents your business domains, metrics, and governance rules upfront
1919
- **Resource governance built in** — per-session cost, retry, row, and token budgets, and wall-clock limits.
2020
- **Per-caller row/column security** — allow/deny tables and filter values by principal, for multi-user bots.
2121
- **Framework-agnostic** — plain-function tools for the Claude Agent SDK, LangChain/deepagents, Pydantic AI, or no framework at all.
22-
- **Bring your own semantics** — read metrics from dbt, Cube, or inline YAML.
22+
- **Bring your own semantics** — read metrics from dbt, Cube, Apache Ossie, or inline YAML.
2323

2424
### Without a contract vs. with one
2525

@@ -652,9 +652,9 @@ tables:
652652
type: VARCHAR
653653
```
654654
655-
`tier`, `indicator_kind`, and `domains` are all optional. For dbt and Cube sources, these fields live under the metric's `meta:` block and are read through the same field names.
655+
`tier`, `indicator_kind`, and `domains` are all optional. For dbt and Cube sources, these fields live under the metric's `meta:` block and are read through the same field names. For Ossie, they live in the model's `custom_extensions` block (see [Apache Ossie](#apache-ossie) below).
656656

657-
**Ownership & review cadence (optional).** `business_owner` / `operational_owner` (always *teams*, not individuals — owners outlive any one person) and `last_reviewed` declare who owns a metric's definition vs. its data health, and when it was last vetted. `last_reviewed` feeds `DataContract.find_stale()` (see [Governance Staleness](docs/architecture.md#governance-staleness)) and surfaces in `lookup_metric` / `lookup_domain` as a `stale` flag so the agent can disclose drift at query time. The same three fields are accepted on a `domain`. They are read from the **YAML source** today; dbt/Cube metrics default to unset.
657+
**Ownership & review cadence (optional).** `business_owner` / `operational_owner` (always *teams*, not individuals — owners outlive any one person) and `last_reviewed` declare who owns a metric's definition vs. its data health, and when it was last vetted. `last_reviewed` feeds `DataContract.find_stale()` (see [Governance Staleness](docs/architecture.md#governance-staleness)) and surfaces in `lookup_metric` / `lookup_domain` as a `stale` flag so the agent can disclose drift at query time. The same three fields are accepted on a `domain`. They are read from the **YAML source** and from an **Ossie** model's `custom_extensions` block; dbt/Cube metrics default to unset.
658658

659659
**dbt** — point to a `manifest.json`:
660660
```yaml
@@ -709,6 +709,76 @@ cubes:
709709

710710
Joins whose SQL doesn't match the single-equality pattern (composite keys with `AND`-chained equalities) or whose target cube can't be resolved by name are skipped silently — fall back to declaring those in your contract YAML via `YamlSource`.
711711

712+
### Apache Ossie
713+
714+
[Apache Ossie (incubating)](https://ossie.apache.org/) — formerly Open Semantic Interchange — is the vendor-neutral spec for exchanging semantic models across analytics, AI, and BI tools. Point at a model file:
715+
716+
```yaml
717+
semantic:
718+
source:
719+
type: ossie
720+
path: "./semantic/model.yml"
721+
```
722+
723+
Ossie standardises what a metric *is*; this library enforces what an agent may *do* with it. So the spec is a strict subset of the vocabulary here — an Ossie `Metric` carries only `name`, `expression`, `description`, `datatype`, and `ai_context`.
724+
725+
| Ossie | Read as |
726+
| --- | --- |
727+
| `datasets[].source` + `fields[]` | Table schemas. A three-part `database.schema.table` is keyed on its trailing `schema.table`; a query-backed dataset registers no table |
728+
| `metrics[].expression.dialects[]` | `sql_expression`, resolved deterministically — your `dialect=` first, then `ANSI_SQL`, then `Ossie_SQL_2026`, then the first declared entry |
729+
| `relationships[]` | `Relationship`, with cardinality *derived from both sides* — see below |
730+
| `ai_context` (anywhere) | Carried into `get_extras()["ossie_ai_context"]`, keyed by model then entity kind, never interpreted |
731+
| `custom_extensions[]` | Ours read as real vocabulary; every other vendor's carried into `get_extras()["ossie_custom_extensions"]`, keyed by model then vendor |
732+
733+
**Cardinality.** Ossie never writes the join type down; it is implied by which endpoints are keys. The spec documents `to` as the one side, but nothing validates that — and trusting it is not free, because `RelationshipChecker` fires its fan-out warning only on `one_to_many`. So both sides are checked:
734+
735+
| `from_columns` a key | `to_columns` a key | Type |
736+
| --- | --- | --- |
737+
| ✓ | ✓ | `one_to_one` |
738+
| ✗ | ✓ | `many_to_one` |
739+
| ✓ | ✗ | `one_to_many` |
740+
| ✗ | ✗ | `many_to_many` |
741+
742+
Keys are optional in Ossie, and their absence is not evidence of fan-out — when the `to` dataset declares no keys at all, the spec's declaration stands and the join reads as `many_to_one`.
743+
744+
Composite-key relationships are **skipped with a warning** rather than split into one edge per column pair — our `Relationship` has single-column endpoints, and splitting would assert two joins that are each individually wrong. Declare those in a `YamlSource` overlay.
745+
746+
**Governance fields.** Ownership, review dates, tiers, decompositions, and the metric-impact graph have no home in the Ossie spec, and every `$def` in its JSON Schema sets `additionalProperties: false` — so they cannot be added as extra keys. They ride in the spec's own escape hatch, `custom_extensions`, under this project's vendor name:
747+
748+
```yaml
749+
custom_extensions:
750+
- vendor_name: AGENTIC_DATA_CONTRACTS
751+
data: |
752+
{
753+
"metrics": {
754+
"total_sales": {
755+
"business_owner": "revenue-analytics",
756+
"operational_owner": "data-platform",
757+
"last_reviewed": "2026-05-01",
758+
"tier": ["gold"],
759+
"domains": ["sales"],
760+
"decompositions": [
761+
{"operator": "sum", "operands": ["total_profit", "total_cost"]}
762+
],
763+
"drill_by": [
764+
{"dimension": "region", "column": "public.customer.c_region"}
765+
]
766+
}
767+
},
768+
"metric_impacts": [
769+
{"from": "total_cost", "to": "total_profit",
770+
"direction": "negative", "confidence": "verified",
771+
"evidence": "Margin bridge reconciled 2026-Q1."}
772+
]
773+
}
774+
```
775+
776+
Ossie stores extension payloads as a JSON *string*, so `data` is JSON nested inside YAML (a plain YAML mapping is accepted too). Restored decompositions and `drill_by` go through the same validators as `YamlSource`, failing loudly at load, and a bare `"tier": "gold"` is promoted to a list exactly as `YamlSource` does. Another vendor's malformed payload is carried verbatim and logged rather than raised on — a neighbour's typo must not stop your contract from being enforced.
777+
778+
`expected_extras` does not apply to an Ossie source: its extras are synthesized by the parser under fixed keys, not authored as free top-level sections. Declaring it on a non-`yaml` source logs a warning rather than being silently dropped.
779+
780+
> The spec is at `0.2.0.dev0` with no tagged releases yet, so expect churn. `dialect` is deliberately treated as an opaque string and never validated against the enum, since the accepted expression-language proposal adds `Ossie_SQL_2026` and makes it the default.
781+
712782
## Table Relationships
713783

714784
Define join paths so the agent knows how to combine tables correctly:
@@ -849,7 +919,7 @@ await trace.callable({
849919
# "kind": "identity", "operator": "product"}
850920
```
851921

852-
Today `decompositions` / `drill_by` are declared directly in YAML contracts; dbt/Cube extraction and a variance-diagnosis tool are deferred.
922+
Today `decompositions` / `drill_by` are declared directly in YAML contracts or in an Ossie model's `custom_extensions`; dbt/Cube extraction and a variance-diagnosis tool are deferred.
853923

854924
## Validating a verified-examples corpus
855925

@@ -1205,7 +1275,7 @@ DATA_PLUGIN_PATH=/tmp/kwp/data \
12051275

12061276
**Does it execute my SQL?** Only `run_query` does, and only after validation passes (plus an optional EXPLAIN dry-run). `inspect_query` validates without executing, and forbidden operations (DELETE/DROP/UPDATE/…) are blocked before they ever reach the database.
12071277

1208-
**Do I have to use dbt or Cube?** No. Author metrics inline in a `semantic.yml` with `YamlSource`. dbt (`manifest.json`) and Cube are supported if you already have them — the agent-facing behavior is identical regardless of source.
1278+
**Do I have to use dbt or Cube?** No. Author metrics inline in a `semantic.yml` with `YamlSource`. dbt (`manifest.json`), Cube, and Apache Ossie are supported if you already have them — the agent-facing behavior is identical regardless of source.
12091279

12101280
**Does it work without the Claude Agent SDK?** Yes. The tools are plain async functions usable from LangChain/deepagents, Pydantic AI, or directly; the example agents fall back to a no-SDK demo mode.
12111281

0 commit comments

Comments
 (0)