Visualize LLM outputs against datasets, manually annotate results, and run automated evaluations to algorithmically optimize prompts.
-
Updated
Nov 22, 2025 - TypeScript
#
llm-evals
Here are 27 public repositories matching this topic...
An implementation of the Anthropic's paper and essay on "A statistical approach to model evaluations"
-
Updated
Oct 6, 2025 - Python
Create an evaluation framework for your LLM based app. Incorporate it into your test suite. Lay the monitoring foundation.
workshop llm llms llmops llm-eval llm-test llm-evaluation-framework llm-evaluation-metrics llm-monitoring llm-testing llm-evals
-
Updated
May 6, 2025 - Jupyter Notebook
A framework for evaluating large language models (LLMs) across a variety of tasks.
nlp benchmark ai evaluation-framework ai-evaluation llm llm-evaluation llm-as-a-judge g-eval llm-evals
-
Updated
Mar 18, 2026 - Python
Squeeze your model with pressure prompts to see if its behavior leaks.
reliability evaluation calibration alignment quality-assurance metamorphic-testing ai-safety trustworthiness hallucinations prompt-engineering llm-eval llm-evals
-
Updated
Mar 1, 2026 - Python
Detecting Relational Boundary Erosion in AI systems. A framework for testing whether models maintain honest, calibrated, and appropriate boundaries.
python yaml calibration alignment metamorphic-testing model-evaluation ai-safety red-teaming prompt-injection hallucination-detection llm-evals evaluation-harness
-
Updated
Feb 22, 2026 - Python
Evaluation patterns, release gates, and anti-hallucination techniques for developer-focused AI workflows.
-
Updated
Mar 27, 2026 - Python
In this we evaluate the LLM responses and find accuracy
-
Updated
Jul 8, 2025 - Python
Local-first LLM evaluation runner with baselines, caching, markdown reports, and CI-friendly quality, latency, and cost gates.
-
Updated
Apr 9, 2026 - Python
A practical workbench for prompt, model, and mocked workflow evaluation with repeatable benchmarks, structured graders, and agent episode traces.
-
Updated
Mar 15, 2026 - Python
Behavioral evaluation framework for sentience-, emotion-, and welfare-related AI claims, with anti-sandbagging analysis.
alignment ai-safety llm-evals model-welfare benchmark-integrity behavioral-evaluation sentience-evals evaluation-research
-
Updated
Mar 24, 2026 - Python
Agentic research pipeline with local retrieval, structured evaluation, conditional revision, and traceable outputs using Groq.
python cli retrieval evaluation ai-agents groq llm prompt-engineering research-agent agentic-workflow llm-evals
-
Updated
Mar 23, 2026 - TypeScript
Evaluation and reliability harness for agentic LLM systems, with task success, latency, cost, retries, fallback routing, and failure taxonomy.
benchmarking intelligence dashboard latency reliability evaluation observability ai-agents failure-analysis cost-tracking llm-evals fallback-routing
-
Updated
Apr 1, 2026 - Python
Lightweight CI-native regression and behavior-aware evaluation toolkit for black-box agent workflows.
-
Updated
Apr 10, 2026 - Python
Reproducible evaluation suite for LLM behavior research: epistemic pathology, delegated introspection, and temporal consciousness diagnostics
python alignment language-models reproducibility ai-safety interpretability research-engineering ai-alignment research-tools rlhf llm-evaluation behavioral-research llm-evals epistemic-alignment temporal-consciousness epistemic-pathology
-
Updated
Nov 16, 2025 - Python
Horizon-Eval: evaluation-integrity framework for portable long-horizon agent benchmarks, with QA gates, trajectory auditing, replayable run bundles, and safety-gap analysis.
evaluation alignment reproducibility trajectory-analysis ai-safety safety-engineering llm-evals agent-evals benchmark-integrity
-
Updated
Mar 21, 2026 - Python
CLI release gate for structured AI changes.
-
Updated
Mar 12, 2026 - TypeScript
Germany-based AI/backend engineer. Shipping production quality loops, build logs, and practical engineering notes.
-
Updated
Apr 1, 2026
Browser-based LLM evaluation tool. Test prompts and models on your own data with multi-model judge council, cost tracking, and ranked results.
-
Updated
Apr 10, 2026 - TypeScript
Improve this page
Add a description, image, and links to the llm-evals topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the llm-evals topic, visit your repo's landing page and select "manage topics."