Describe the bug
In deepeval/prompt/utils.py, the structured output schema builder detects nested Pydantic models with BaseModel in annotation.__bases__. This only works when the field type directly subclasses BaseModel. If a project uses an intermediate base class (e.g. class DerivedPayload(BasePayload) where BasePayload(BaseModel)), the check fails and the field falls back to STRING type.
To Reproduce
from pydantic import BaseModel
class BasePayload(BaseModel):
value: int
class DerivedPayload(BasePayload):
extra: str
class MyOutput(BaseModel):
payload: DerivedPayload # This gets typed as STRING instead of OBJECT
# Because:
print(BaseModel in DerivedPayload.__bases__) # False
print(BaseModel in DerivedPayload.__mro__) # True
Root Cause
deepeval/prompt/utils.py L237-252:
elif (
hasattr(annotation, "__bases__")
and BaseModel in annotation.__bases__
):
field_type = "OBJECT"
Expected behavior
Any Pydantic BaseModel subclass (including indirect inheritance) should be recognized as an OBJECT type in generated schemas.
Suggested Fix
Use issubclass(annotation, BaseModel) or check __mro__ instead of __bases__, consistent with how array item models are already handled elsewhere in the same file:
elif (
hasattr(annotation, "__mro__")
and issubclass(annotation, BaseModel)
):
field_type = "OBJECT"
Describe the bug
In
deepeval/prompt/utils.py, the structured output schema builder detects nested Pydantic models withBaseModel in annotation.__bases__. This only works when the field type directly subclassesBaseModel. If a project uses an intermediate base class (e.g.class DerivedPayload(BasePayload)whereBasePayload(BaseModel)), the check fails and the field falls back toSTRINGtype.To Reproduce
Root Cause
deepeval/prompt/utils.pyL237-252:Expected behavior
Any Pydantic
BaseModelsubclass (including indirect inheritance) should be recognized as anOBJECTtype in generated schemas.Suggested Fix
Use
issubclass(annotation, BaseModel)or check__mro__instead of__bases__, consistent with how array item models are already handled elsewhere in the same file: