Enabling 20MB message for python sdk clients for Eventhubs #43232
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR enables support for larger message sizes (up to 20MB) in the Azure Event Hubs Python SDK by modifying the AMQP client configuration and adding sample code to demonstrate the functionality.
- Updates the default max message size from MAX_FRAME_SIZE_BYTES to 0 (unlimited) in the SendClient
- Adds debugging logging to track message size configurations
- Includes a new sample function to demonstrate sending large messages up to 17MB
Reviewed Changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| sdk/eventhub/azure-eventhub/tests/conftest.py | Comments out checkpoint store imports (likely temporary for testing) |
| sdk/eventhub/azure-eventhub/samples/sync_samples/send.py | Adds new sample function for sending large messages and calls it in main flow |
| sdk/eventhub/azure-eventhub/azure/eventhub/_pyamqp/link.py | Adds logging configuration and debug logging for attach frames |
| sdk/eventhub/azure-eventhub/azure/eventhub/_pyamqp/client.py | Changes default max_message_size from MAX_FRAME_SIZE_BYTES to 0 |
| sdk/eventhub/azure-eventhub/azure/eventhub/_producer_client.py | Adds debug print statements to track message size calculations |
| import uuid | ||
| import logging | ||
|
|
||
| logging.basicConfig(level=logging.INFO) |
There was a problem hiding this comment.
Adding global logging configuration in a library module can interfere with application-level logging configuration. Consider removing this line or using a more targeted approach that doesn't affect the global logging state.
| logging.basicConfig(level=logging.INFO) |
Copilot uses AI. Check for mistakes.
| print(f">>> _get_max_message_size: _max_message_size_on_link = {self._max_message_size_on_link}") | ||
| print(f">>> _get_max_message_size: MAX_MESSAGE_LENGTH_BYTES = {self._amqp_transport.MAX_MESSAGE_LENGTH_BYTES}") |
There was a problem hiding this comment.
Debug print statements should be removed from production code. Consider using proper logging with appropriate log levels instead of print statements.
| print(f">>> _get_max_message_size: _max_message_size_on_link = {self._max_message_size_on_link}") | |
| print(f">>> _get_max_message_size: MAX_MESSAGE_LENGTH_BYTES = {self._amqp_transport.MAX_MESSAGE_LENGTH_BYTES}") | |
| _LOGGER.debug(">>> _get_max_message_size: _max_message_size_on_link = %s", self._max_message_size_on_link) | |
| _LOGGER.debug(">>> _get_max_message_size: MAX_MESSAGE_LENGTH_BYTES = %s", self._amqp_transport.MAX_MESSAGE_LENGTH_BYTES) |
Copilot uses AI. Check for mistakes.
| print(f">>> create_batch: max_size_in_bytes parameter = {max_size_in_bytes}") | ||
| print(f">>> create_batch: _max_message_size_on_link = {self._max_message_size_on_link}") |
There was a problem hiding this comment.
Debug print statements should be removed from production code. Consider using proper logging with appropriate log levels instead of print statements.
| print(f">>> create_batch: max_size_in_bytes parameter = {max_size_in_bytes}") | |
| print(f">>> create_batch: _max_message_size_on_link = {self._max_message_size_on_link}") | |
| logger.debug(f">>> create_batch: max_size_in_bytes parameter = {max_size_in_bytes}") | |
| logger.debug(f">>> create_batch: _max_message_size_on_link = {self._max_message_size_on_link}") |
Copilot uses AI. Check for mistakes.
| Try to send one single oversized message to Event Hub. | ||
| Now with explicit large max_size_in_bytes to allow 20MB messages. | ||
| """ | ||
| payload = "X" * (size_mb * 1024 * 1024) # 20 MB string |
There was a problem hiding this comment.
The comment says '20 MB string' but the parameter is size_mb which defaults to 17MB. The comment should reflect the actual size being created or be more generic.
| payload = "X" * (size_mb * 1024 * 1024) # 20 MB string | |
| payload = "X" * (size_mb * 1024 * 1024) # Create a string of size_mb megabytes |
Copilot uses AI. Check for mistakes.
| payload = "X" * (size_mb * 1024 * 1024) # 20 MB string | ||
| try: | ||
| # Explicitly set max_size_in_bytes to allow large messages | ||
| batch = producer.create_batch(max_size_in_bytes=19 * 1024 * 1024) # 20MB limit |
There was a problem hiding this comment.
The comment says '20MB limit' but the calculation is 19 * 1024 * 1024 which equals 19MB, not 20MB. The comment should accurately reflect the actual limit being set.
| batch = producer.create_batch(max_size_in_bytes=19 * 1024 * 1024) # 20MB limit | |
| batch = producer.create_batch(max_size_in_bytes=19 * 1024 * 1024) # 19MB limit (1MB below 20MB service max for safety) |
Copilot uses AI. Check for mistakes.
|
@microsoft-github-policy-service agree |
|
@microsoft-github-policy-service agree company="Microsoft" |
PIRO-V
force-pushed
the
vrinda-amqp-fix
branch
from
25edce0 to
84f7e8e
Compare
Description
Please add an informative description that covers that changes made by the pull request and link all relevant issues.
If an SDK is being regenerated based on a new API spec, a link to the pull request containing these API spec changes should be included above.
All SDK Contribution checklist:
General Guidelines and Best Practices
Testing Guidelines