Skip to content

Releases: najeed/ai-agent-eval-harness

Release list

First Release of MultiAgent-Verify

First Release of MultiAgent-Verify Pre-release
Pre-release

Choose a tag to compare

View all tags
@najeed najeed released this 02 Apr 23:33
v1.1
d1edd4a

Initial Stable Release Notes - MultiAgent-Verify v1.1.0
Announcing the first major release of MultiAgent-Verify, the enterprise-grade reliability framework for AI agents. This release establishes a new standard for industrial-scale agent evaluation, security verification, and high-fidelity dataset generation.

Key Highlights
Industrial-Grade Evaluation Engine

  • Unified Testing Framework: A robust, protocol-agnostic execution engine for assessing agent performance across http, local, and socket interfaces.
  • Agent Eval Specification (AES): Standardized benchmarking for multi-agent systems, including complexity levels and agent topologies.
  • Pass@K Evaluation: Scientific assessment of agent success rates with support for multiple attempts and consistency analysis.

Industrial Dataproc Engine

  • Cross-Sector Synthesis: High-signal dataset generation for 16+ industrial sectors, including Finance, Healthcare, Energy, and Telecom.
  • Gold-Standard Alignments: Native support for industrial standards such as SEC XBRL, CMS Hospital Data, and WHO indicators.
  • Parity Verification: Built-in mathematical validation to ensure synthetic datasets maintain real-world statistical veracity.

Security & Integrity

  • Adversarial Mutation: Generate adversarial scenario variants (typos, injections, ambiguity) to stress-test agent robustness.
  • Verification Protocol: SHA-256 integrity checks and manifest validation for all execution traces and reports.
  • Failure Taxonomy: A formal classification system for diagnosing agentic root causes and identifying systemic weaknesses.

Visual Debugger & REST API

  • Console API: A powerful REST interface for programmatically interacting with the scenario catalog and evaluation results.
  • Visual Suite: Premium HTML reporting and real-time debugger for visualizing agent trajectories and event streams.

Comprehensive Ecosystem

  • Modular Plugin System: Extensible architecture for adding custom metrics, adapters, and industrial providers.
  • Searchable Catalog: Deep keyword search across a vast library of industrial scenarios and failure edge-cases.
Loading
0 Join discussion