Conversation
RG-04
commented
Mar 30, 2026
RG-04
commented
- Add Latency tracking for HTTP synthesizers
- Prevents early return in generate_http_loop (bug)
- Prevents setting cache on null responses from the synthesizer implementations, usually arising due to API errors
| await self._push_stream(message) | ||
| else: | ||
| self.internal_queue.put_nowait(copy.deepcopy(message)) | ||
| super().push(message) |
There was a problem hiding this comment.
BaseSynthesizer.push() is async def — doesn't this need an await? Without it the coroutine is created but never executed, so the message never reaches internal_queue and _generate_http_loop will hang on get().
| await self._push_stream(message) | ||
| else: | ||
| self.internal_queue.put_nowait(copy.deepcopy(message)) | ||
| super().push(message) |
There was a problem hiding this comment.
The original code did copy.deepcopy(message) before enqueuing. BaseSynthesizer.push() does a plain put_nowait(message) with no copy. Since _generate_http_loop mutates meta_info in-place (sets format, text, mark_id, etc.), this could corrupt the caller's original dict. Should the deepcopy be preserved here?
| self.first_chunk_generated = False | ||
| self.synthesized_characters = 0 | ||
| self.model = "default" | ||
| self.current_turn_start_time = None |
There was a problem hiding this comment.
current_turn_id is set in _stamp_turn_start (line 108) and read in _record_turn_latency (line 116) but isn't initialized here. Might be worth adding self.current_turn_id = None alongside current_turn_start_time for safety.
2 participants
