5748 Filter out nested serializer fields in the API and defer the filtered fields. #5973
+765
−89
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR leverages the new mixin
NestedDynamicFieldsMixin
fordrf-dynamic-fields
to enable filtering of nested fields, as introduced in dbrgn/drf-dynamic-fields#44This allows requests like:
GET /api/rest/v4/docket-entries/?fields=id,recap_documents__plain_text,recap_documents__id
to filter both parent and nested fields:
Similarly, the
omit
parameter can be used:GET /api/rest/v4/docket-entries/?omit=entry_number,recap_documents__plain_text
This mixin has been added to parent serializers that render nested objects and to their respective child serializers:
Docket Entries:
DocketEntrySerializer
RECAPDocumentSerializer
Financial Disclosures:
FinancialDisclosureSerializer
AgreementSerializer
DebtSerializer
InvestmentSerializer
GiftSerializer
NonInvestmentIncomeSerializer
PositionSerializer
ReimbursementSerializer
SpouseIncomeSerializer
People:
PersonSerializer
EducationSerializer
Deferred fields.
To optimize query performance by deferring filtered fields from the request, this PR introduces two additional mixins:
RetrieveFilteredFieldsMixin
Retrieves the filtered fields from the
fields
oromit
query parameters, allowing theViewSet
to access them and defer them accordingly.DeferredFieldsMixin
Overrides the
get_queryset
method to defer fields based on the filtered request fields. Key behaviors:Conflict with
select_related
:You cannot defer a field that is also included in
select_related
. When this happens, theselect_related
field is removed from the deferred list.Prioritizing the deferred field by removing the
select_related
entry is not ideal, because serializers may reference that relation indirectly (e.g., via a property), making it hard to predict whether it's needed. For example in thedocket-entries
endpoint,select_related("docket")
is used. Thedocket
field is referenced directly in the serializer and also indirectly inRECAPDocument.get_absolute_url
.Original deferred fields preserved:
Any fields already deferred in the original
queryset
are preserved and combined with user-defined ones.*
only()
:Django prioritizes
only()
overdefer()
. If the original queryset usesonly()
, user-specified fields to defer are ignored.Ordering
prefetch_related
:Custom
Prefetch
objects (used for nested field deferring) must appear before the originalprefetch_related()
lookups to avoid conflicts.Nested relationship keys:
Foreign key fields that link nested objects to their parent must not be deferred, even if omitted in the response. Deferring them would trigger extra queries.
For example,
recap_documents__docket_entry_id
is required for joining RECAPDocuments to DocketEntries, even if omitted from the payload. If the user includes this field in the omit list, it is removed from the payload but ignored during the deferring process.No support for
.values()
or.annotate()
:If the base queryset uses
.values()
or annotations, the deferring logic is ignored, as our current ViewSets were this logic was applied don’t use them.Here an example of how
DeferredFieldsMixin
works for the request:api/rest/v4/docket-entries/?fields=id,recap_documents__plain_text,recap_documents__id
DocketEntry
(parent) selects only the id field. However, due to select_related("docket"), all docket fields are also selected:RECAPDocuments
(nested) select only theid
,plain_text
, anddocket_entry_id
fields:In this PR I've added these mixins
RetrieveFilteredFieldsMixin
andDeferredFieldsMixin
to the following viewsets:AudioViewSet
DocketViewSet
DocketEntryViewSet
RECAPDocumentViewSet
CourtViewSet
OpinionClusterViewSet
OpinionViewSet
OpinionsCitedViewSet
TagViewSet
OriginatingCourtInformationViewSet
FinancialDisclosureViewSet
AgreementViewSet
DebtViewSet
GiftViewSet
InvestmentViewSet
NonInvestmentIncomeViewSet
PositionViewSet
ReimbursementViewSet
SpouseIncomeViewSet
PersonViewSet
PositionViewSet
RetentionEventViewSet
EducationViewSet
SchoolViewSet
PoliticalAffiliationViewSet
SourceViewSet
ABARatingViewSet
For now,
drf-dynamic-fields
is temporarily installed from my forked repository. Once the upstream PR is merged and a new release is published, we can update the dependency to point back to the official package.Let me know what do you think.