Local Codex MCP harness: contracts, persistent RAG memory, raw traces, verification records, governance policy and PASS/FLAG/BLOCK audits, observability reports, harness profiles, eval runs, Meta-Harness-lite promotion records, natural-language harness specs, MCP resources/prompts, multi-client installer, and completion gates.

Evaluate AI agents with Unix-style pipeline commands. Schema-driven adapters for any CLI agent, trajectory capture, pass@k metrics, and multi-run comparison.

LLM-powered clinical extraction + structured evals. Prompt strategies, hallucination detection, and per-field F1 scoring.

Deterministic synthetic two-party conversation corpus generator for testing AI scoring systems.

Verification-native local coding agent runtime with eval gates, memory, subagents, and model profiles.

YAML-driven evaluation harness for WhatsApp RAG bots

A lightweight workbench for dataset-driven agent and LLM evaluation.

Can Computer-Use Agents manage buy-side procurement operations? A benchmark across live e-commerce, with multi-model adapters (Northstar, OpenAI, Claude, Gemini), Kernel-hosted browsers, and Harbor/ATIF-v1.6 trajectory export.

Form ADV Part 2A intelligence + peer benchmarking — LangGraph, hybrid retrieval, eval harness

Production-style LLM evaluation harness for structured clinical extraction — compares prompt strategies across accuracy, cost, and hallucination.

Tolerant apply_patch for LLM-generated diffs, plus an eval harness for code agents.

Slack Q&A bot template — ingests Slack threads + markdown docs into Pinecone, then an n8n workflow retrieves, re-ranks on metadata, and posts answers back in-thread. Includes contextual retrieval and an eval harness.

Compare OpenClaw setups against the same scenario suite. Run prompts across multiple configurations, capture answers, latency, token usage, tool calls, and file reads, then generate a single comparison report.

A boring, config-driven harness for evaluating AI systems. One YAML drives the run, the trace is the source of truth. Offline, backtesting, and online-eval modes — works with any agent, RAG, or code-modifying system.

Binary-criteria evaluation harness for Claude skills with planned extension to plugins, agents, and MCP servers. Score every change yes/no across 7 layers — package integrity, trigger quality, functional quality, regression protection, baseline value, model variance, rollout safety. Never gradients.