fix: Handle None response in Evaluator to prevent crashes #105
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Bug Fix: Handle None Response in Evaluator
Problem Description
When the Evaluator receives a
None
response fromextract_json_from_response
, it currently lacks proper error handling, leading toTypeError
crashes during runtime. This issue affects the stability of the application, especially when processing malformed JSON responses from LLM.Root Cause Analysis
The absence of a null check for
extract_json_from_response
return value allowsjson.loads(None)
to be called, which raisesTypeError: the JSON object must be str, bytes or bytearray, not NoneType
.Solution Description
None
responses fromextract_json_from_response
logger.error
tologger.debug
for prompt response loggingCode Changes
File:
evaluator.py
(lines 80-87)Verification Steps
extract_json_from_response
returnsNone
Affected Areas
evaluator.py
module: Updated to includeNone
response handlingImpact
Testing
None
response scenariosValueError
is raised with descriptive messageReferences
Checklist
None
responses in Evaluator