Skip to content

5748 Filter out nested serializer fields in the API and defer the filtered fields. #5973

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

albertisfu
Copy link
Contributor

This PR leverages the new mixin NestedDynamicFieldsMixin for drf-dynamic-fields to enable filtering of nested fields, as introduced in dbrgn/drf-dynamic-fields#44

This allows requests like:
GET /api/rest/v4/docket-entries/?fields=id,recap_documents__plain_text,recap_documents__id
to filter both parent and nested fields:

Screenshot 2025-07-14 at 7 54 12 p m

Similarly, the omit parameter can be used:
GET /api/rest/v4/docket-entries/?omit=entry_number,recap_documents__plain_text

Screenshot 2025-07-14 at 7 55 02 p m

This mixin has been added to parent serializers that render nested objects and to their respective child serializers:

Docket Entries:

  • DocketEntrySerializer
  • RECAPDocumentSerializer

Financial Disclosures:

  • FinancialDisclosureSerializer
  • AgreementSerializer
  • DebtSerializer
  • InvestmentSerializer
  • GiftSerializer
  • NonInvestmentIncomeSerializer
  • PositionSerializer
  • ReimbursementSerializer
  • SpouseIncomeSerializer

People:

  • PersonSerializer
  • EducationSerializer

Deferred fields.

To optimize query performance by deferring filtered fields from the request, this PR introduces two additional mixins:

RetrieveFilteredFieldsMixin
Retrieves the filtered fields from the fields or omit query parameters, allowing the ViewSet to access them and defer them accordingly.

DeferredFieldsMixin
Overrides the get_queryset method to defer fields based on the filtered request fields. Key behaviors:

  • Conflict with select_related:
    You cannot defer a field that is also included in select_related. When this happens, the select_related field is removed from the deferred list.
    Prioritizing the deferred field by removing the select_related entry is not ideal, because serializers may reference that relation indirectly (e.g., via a property), making it hard to predict whether it's needed. For example in the docket-entries endpoint, select_related("docket") is used. The docket field is referenced directly in the serializer and also indirectly in RECAPDocument.get_absolute_url.

  • Original deferred fields preserved:
    Any fields already deferred in the original queryset are preserved and combined with user-defined ones.

  • *only():
    Django prioritizes only() over defer(). If the original queryset uses only(), user-specified fields to defer are ignored.

  • Ordering prefetch_related:
    Custom Prefetch objects (used for nested field deferring) must appear before the original prefetch_related() lookups to avoid conflicts.

  • Nested relationship keys:
    Foreign key fields that link nested objects to their parent must not be deferred, even if omitted in the response. Deferring them would trigger extra queries.
    For example, recap_documents__docket_entry_id is required for joining RECAPDocuments to DocketEntries, even if omitted from the payload. If the user includes this field in the omit list, it is removed from the payload but ignored during the deferring process.

  • No support for .values() or .annotate():
    If the base queryset uses .values() or annotations, the deferring logic is ignored, as our current ViewSets were this logic was applied don’t use them.

Here an example of how DeferredFieldsMixin works for the request:
api/rest/v4/docket-entries/?fields=id,recap_documents__plain_text,recap_documents__id

DocketEntry (parent) selects only the id field. However, due to select_related("docket"), all docket fields are also selected:

Screenshot 2025-07-14 at 8 32 50 p m

RECAPDocuments (nested) select only the id, plain_text, and docket_entry_id fields:
Screenshot 2025-07-14 at 8 34 01 p m

In this PR I've added these mixins RetrieveFilteredFieldsMixin and DeferredFieldsMixin to the following viewsets:

AudioViewSet

DocketViewSet
DocketEntryViewSet
RECAPDocumentViewSet
CourtViewSet
OpinionClusterViewSet
OpinionViewSet
OpinionsCitedViewSet
TagViewSet
OriginatingCourtInformationViewSet

FinancialDisclosureViewSet
AgreementViewSet
DebtViewSet
GiftViewSet
InvestmentViewSet
NonInvestmentIncomeViewSet
PositionViewSet
ReimbursementViewSet
SpouseIncomeViewSet

PersonViewSet
PositionViewSet
RetentionEventViewSet
EducationViewSet
SchoolViewSet
PoliticalAffiliationViewSet
SourceViewSet
ABARatingViewSet

For now, drf-dynamic-fields is temporarily installed from my forked repository. Once the upstream PR is merged and a new release is published, we can update the dependency to point back to the official package.

Let me know what do you think.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: To Do
Development

Successfully merging this pull request may close these issues.

2 participants