Skip to content

Conversation

nerdalert
Copy link
Member

RAG Demo:

  • Adds RAG APIs to docling-serve to create a RAG service.
  • llama-index for categorical document collections.
  • Instructlab quantized model is used for generations.
  • Two collections are created, one via URL and one via file
    uploading to via the Instructlab UI.
  • Collections are then queried and returned with the Answer
    to the query along with the sources and metadata from the
    vector DB.
  • Do a negative test to ensure if the document does not
    contain information matching the query it does not
    hallucinate an answer.

** All components are running locally on a MAC M1.
The PDF ingestions are fast forwarded as they take about 60s each.**

Backend RAG service code will be posted to the docling repo and link here.

Demo video:

rag-demo-v2-oct21.mp4

Signed-off-by: Brent Salisbury <[email protected]>
@nerdalert
Copy link
Member Author

The backend code is posted here docling-project/docling-serve#9

@vishnoianil vishnoianil added the demo PR that contains Demo related changes label Nov 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
demo PR that contains Demo related changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants