-
Notifications
You must be signed in to change notification settings - Fork 18
Feature: add bulk_save to HistoryModel #408
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Conversation
Unit tests in social_protection module due to dependency requirements
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds a bulk_save class method to the HistoryModel for efficiently creating and updating multiple instances in a single atomic transaction. Instances without an id are created, while those with an id are updated.
- Adds
bulk_saveclass method supporting batch operations with configurable batch size - Implements atomic transaction handling to ensure all-or-nothing semantics
- Handles version tracking and timestamp management for both create and update operations
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| update_fields = [f for f in to_update[0].__dict__.keys() | ||
| if not f.startswith('_') and f not in exclude_fields] |
Copilot
AI
Nov 15, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Accessing to_update[0] without checking if to_update is empty will raise an IndexError. This code is inside the if to_update: block but relies on at least one element existing. While the if to_update: check prevents the block from executing when empty, the logic assumes all instances have identical field sets, which may not hold if the model has fields with null=True or if different subclasses are mixed. Consider extracting field names from the model's _meta.get_fields() instead of relying on instance __dict__.
| update_fields = [f for f in to_update[0].__dict__.keys() | |
| if not f.startswith('_') and f not in exclude_fields] | |
| update_fields = [ | |
| field.name for field in cls._meta.get_fields() | |
| if ( | |
| field.concrete | |
| and not field.auto_created | |
| and not (field.many_to_many or field.one_to_many) | |
| and field.name not in exclude_fields | |
| ) | |
| ] |
| setattr(instance, field, value) | ||
| instance.user_updated = user | ||
| instance.date_updated = now | ||
| instance.version = F('version') + 1 |
Copilot
AI
Nov 15, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Using F('version') + 1 with bulk_update will not work as expected. The F expression is meant for database-level updates in .update() calls, but bulk_update saves the current Python object values to the database. The F expression will be serialized as-is rather than being evaluated as SQL. This will either fail or set an incorrect value. The version increment logic at lines 288-292 attempts to compensate by re-fetching versions, but the initial assignment is still problematic. Set instance.version to the current value + 1 instead: instance.version = instance.version + 1.
| instance.version = F('version') + 1 | |
| instance.version = instance.version + 1 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@weilu can you double check that ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think the use of F('version') itself is problematic as proven by passing unit test in openimis/openimis-be-social_protection_py#119, but we can switch to instance.version for clarity
| return new_instance | ||
|
|
||
| @classmethod | ||
| def bulk_save(cls, data_list, user, batch_size=100): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can it be default at None and retrieve it from the get_current_user() ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Models shouldn't be aware of current user as that's at the controller layer.
This allows for efficient update and create of multiple instances based on 'id' field – those without supplied id values are created, otherwise updated.
All operations are atomic - either all succeed or all fail.
Unit tests in social_protection module due to dependency requirements