Improve async translation trigger after transcription#16
Improve async translation trigger after transcription#16mandira15 wants to merge 2 commits intosusiai:masterfrom
Conversation
|
I am a first time contributor , please give me feedbacks for further improvement and learning. |
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Triggers translation asynchronously immediately after transcription to reduce latency for live interpretation and avoid delayed/indirect translation starts.
Changes:
- Start a background thread to run
process_translation()right after a transcript is produced. - Gate translation trigger on
translate_tobeing present andtranscript_event["translated"]not being set.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| # Trigger translation asynchronously after transcription | ||
| if translate_to and not transcript_event.get("translated", False): | ||
| threading.Thread( | ||
| target=process_translation, | ||
| args=(transcript_event,), | ||
| daemon=True | ||
| ).start() |
There was a problem hiding this comment.
Starting a brand-new threading.Thread per transcript can lead to unbounded thread creation under load (e.g., many chunks/users), increasing context switching and memory usage. Prefer a bounded concurrency mechanism such as a shared ThreadPoolExecutor (or the project’s existing background job system, e.g., Celery/RQ) to cap parallelism and improve operational stability.
| if translate_to and not transcript_event.get("translated", False): | ||
| threading.Thread( | ||
| target=process_translation, | ||
| args=(transcript_event,), | ||
| daemon=True | ||
| ).start() |
There was a problem hiding this comment.
The background thread receives transcript_event by reference. If the main thread continues to mutate transcript_event (or reuses it) after starting the thread, process_translation() may observe inconsistent state or race with updates like translated. Consider passing an immutable payload (or a shallow/deep copy of the needed fields) and using a synchronization/atomic mechanism for the translated status to prevent duplicate or inconsistent translations.
| threading.Thread( | ||
| target=process_translation, | ||
| args=(transcript_event,), | ||
| daemon=True |
There was a problem hiding this comment.
Daemon threads can be terminated abruptly during worker shutdown/reload (e.g., Django dev server autoreload, gunicorn worker restart), which may leave translations incomplete without retries. If translation completion is important, use non-daemon execution with a graceful shutdown strategy, or (preferably) a durable background job queue so work survives process restarts.
|
Thanks for the detailed feedback! You're right about the potential issues with unbounded thread creation and shared mutable state. I initially chose threading for simplicity and low-latency triggering, but I see how this can impact scalability under load. I'll refactor this to use a bounded ThreadPoolExecutor and pass an immutable copy of transcript_event to avoid race conditions. Also exploring integration with a more robust background job system if needed. I'll surely improve this at the earliest. |
This pull request improves the translation workflow by triggering translation asynchronously immediately after a transcript is generated. Previously, translation was triggered indirectly and could be delayed depending on transcript queue processing. With this change, translation is started in a separate background thread right after transcription, ensuring faster and more reliable translation handling.
The update ensures that translation is only triggered when a valid translate_to language is provided and avoids duplicate translation by checking the existing translation status. The use of a daemon thread ensures non-blocking execution and maintains the responsiveness of the transcription pipeline.
Motivation and Context
The existing implementation delayed translation until older transcript chunks were processed or relied on indirect triggers. This caused inconsistent translation timing and affected real-time interpretation performance.
This change improves real-time responsiveness by:
• Triggering translation immediately after transcription
• Ensuring non-blocking execution using asynchronous threading
• Preventing duplicate translation using proper checks
This is especially important for live interpretation use cases where low latency is critical.
How Has This Been Tested?
This change was tested locally using the SUSI Translator Django development server.
Testing steps performed:
• Started Django server using python manage.py runserver
• Sent POST request to /api/transcribe endpoint via Swagger UI
• Verified transcription pipeline execution
• Confirmed translation trigger occurs asynchronously
• Ensured no crashes, blocking, or duplicate translations
The system continues to process transcription and translation without affecting existing functionality.
Types of changes
Checklist: