🔍 Hallucination Detector

A beautiful web app for detecting factual inconsistencies (hallucinations) in AI-generated text using Vectara's Hughes Hallucination Evaluation Model (HHEM).

✨ Features

• Real-time hallucination detection - Compare AI-generated text against source material
• Consistency scoring - Get a probability score (0-1) indicating factual consistency
• Sentence-level analysis - Identify exactly which sentences may contain hallucinations
• Beautiful UI - Modern, responsive interface with visual feedback
• Fast inference - Runs locally using Hugging Face Transformers

🚀 Quick Start

Prerequisites

• Python 3.9 or higher
• pip (Python package manager)

Installation

1. Clone or navigate to the project directory:

   cd /Users/hindy/Desktop/Hallucination_Detector

2. Create a virtual environment (recommended):

   python -m venv venv
   source venv/bin/activate  # On Windows: venv\Scripts\activate

3. Install dependencies:

   pip install -r requirements.txt

4. Run the application:

   python backend.py

5. Open your browser and navigate to:

   http://localhost:8000
   

📖 How It Works

The Vectara HHEM model evaluates factual consistency between two texts:

1. Source Text (Premise) - The ground truth or reference information
2. Generated Text (Hypothesis) - The AI-generated text to verify

The model returns a score between 0 and 1:

• Score > 0.5 → Text is likely consistent with the source
• Score ≤ 0.5 → Text likely contains hallucinations

Example Use Case

Source Text:

Albert Einstein was born in Ulm, Germany in 1879. He developed the theory of relativity and won the Nobel Prize in Physics in 1921.

Generated Text (Consistent):

Einstein, born in 1879 in Ulm, is famous for his theory of relativity.

Generated Text (Hallucinated):

Einstein was born in 1885 in Munich and won the Nobel Prize for his work on quantum mechanics.

🔧 API Reference

POST /detect

Analyze text for hallucinations.

Request Body:

{
  "source_text": "The original/reference text...",
  "generated_text": "The AI-generated text to check...",
  "check_sentences": true
}

Response:

{
  "overall_score": 0.8542,
  "is_hallucinated": false,
  "confidence_percentage": 85.42,
  "sentence_results": [
    {
      "sentence": "First sentence...",
      "consistency_score": 0.92,
      "is_hallucinated": false
    }
  ],
  "summary": "✅ The generated text appears factually consistent with the source."
}

GET /health

Check if the service is running and model is loaded.

🛠️ Technology Stack

• Backend: FastAPI (Python)
• ML Model: vectara/hallucination_evaluation_model
• ML Framework: Hugging Face Transformers, PyTorch
• Frontend: Vanilla HTML/CSS/JavaScript
• NLP: NLTK (for sentence tokenization)

📝 Notes

• First run will download the model (~500MB) from Hugging Face
• GPU acceleration is automatically used if available (CUDA)
• For production use, consider adding authentication and rate limiting
• The model works best with English text

📚 References

• Vectara HHEM Model
• Vectara Hallucination Leaderboard
• Vectara Documentation

📄 License

This project is for demonstration purposes. The Vectara HHEM model is licensed under Apache 2.0.

---

Built with ❤️ using FastAPI and Hugging Face Transformers