Add pluggable LLM backend and Layer 3 fallback for query optimization - #115
Open
sarthaksarthak9 wants to merge 4 commits into
Open
Add pluggable LLM backend and Layer 3 fallback for query optimization#115sarthaksarthak9 wants to merge 4 commits into
sarthaksarthak9 wants to merge 4 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This implements AI query optimization pipeline by introducing a pluggable LLM backend and integrating it as the final fallback layer for suggestion generation. It adds a
BaseLLMBackendabstraction with implementations for Ollama, OpenAI, and a Mock backend for testing, allowing the backend to be selected through theCDB_LLM_BACKENDconfiguration.An LLM analyzer has been added to build a schema-aware system prompt, invoke the configured backend, validate the returned response, and convert valid recommendations into
Suggestionobjects. All LLM-generatedsafe_sqlstatements go through the same allow-list validation as rule-engine suggestions before being persisted, ensuring they satisfy the existing safety guarantees.The optimization collector has also been updated to follow a rules-first, LLM-as-fallback workflow. The LLM is only invoked for candidate queries that have already exceeded the latency or I/O thresholds and for which the rule engine produced no optimization suggestions. Suggestions generated by the LLM are assigned a default confidence score of 0.5, reflecting that they are intended to complement—not replace—the existing deterministic rule engine by covering optimization opportunities that are not recognized by the current set of rules.