Evaluating ChatGPT’s Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulness

[ACL'24] Official Implementation of the paper "Direct Evaluation of Chain-of-Thought in Multi-hop Reasoning with Knowledge Graphs"(https://aclanthology.org/2024.findings-acl.168)

FaithScore: Fine-grained Evaluations of Hallucinations in Large Vision-Language Models

Evaluating the faithfulness of long-context language models

Code and data for the ACL 2024 Findings paper "Do LVLMs Understand Charts? Analyzing and Correcting Factual Errors in Chart Captioning"

The Dataset and Official Implementation for <Discursive Socratic Questioning: Evaluating the Faithfulness of Language Models’ Understanding of Discourse Relations> @ ACL 2024

Koishi's Day 2024 Paper (NeurIPS 2024): An advanced persona-driven role-playing system with global faithfulness quantification and optimization. In memory of the Koishi's Day of 2024.

About The corresponding code from our paper " Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoning" . Do not hesitate to open an issue if you run into any trouble!

Local-first evaluation framework for RAG systems and AI Agents. 18+ metrics, CLI + SDK, framework-agnostic. The pytest of AI evaluation.

RAG evaluation and observability lab.

7-layer RAG framework that eliminates semantic drift + context poisoning. Faithfulness 0.94 vs 0.71 Naive RAG. +13.6 EM on multi-hop QA.

Novel data representation leading to granular citations and higher accuracy

Geometric LLM grounding verification — deterministic, auditable, no second LLM. Python library for measuring how faithfully model outputs reflect their sources.

A new training framework for Trustworthy Large Reasoning Models

On the evaluation of deep learning interpretability methods for medical images under the scope of faithfulness

[EMNLP 2023] A Causal View of Entity Bias in (Large) Language Models

Official PyTorch implementation of Faithfulness Serum (ACL Main 2026) - a training-free method that improves the faithfulness of LLM explanations by guiding generation with attribution-based signals.

FIFA: Unified Faithfulness Evaluation Framework for Text-to-Video and Video-to-Text Generation

Causal Chain-of-Thought step faithfulness evaluation harness — quantifies whether reasoning steps actually drive model outputs or are decorative

Explanation-faithfulness auditing for vision-language models