feat: Add comprehensive rate limit handling across API providers #161
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Add new model identifiers to AVAILABLE_LLMS - Extend create_client for new model support - Add environment variable validation - Implement model-specific response handling - Update requirements.txt with google-cloud-aiplatform Co-Authored-By: Erkin Alp Güney <[email protected]>
Co-Authored-By: Erkin Alp Güney <[email protected]>
Co-Authored-By: Erkin Alp Güney <[email protected]>
It is quite difficult for weak models to do so based on the default template, as one error in replaced text can lead to termination of editing in the *SEARCH/REPLACE mode*. Splitting the text into segments and enabling "whole" mode editing helps solve this problem. The approach was tested on the "2d_diffusion" task using the "groq/llama3-8b-8192".
…n with segmented templates Co-Authored-By: Erkin Alp Güney <[email protected]>
…on support Co-Authored-By: Erkin Alp Güney <[email protected]>
…rs, and template compatibility Co-Authored-By: Erkin Alp Güney <[email protected]>
- Add retry limit and better error handling in extract_json_between_markers - Replace assert statements with try-catch blocks across all files - Add proper error messages and recovery mechanisms - Prevent infinite loops when JSON extraction fails Fixes SakanaAI#154 Co-Authored-By: Erkin Alp Güney <[email protected]>
- Add RateLimitHandler class for managing rate limits - Implement provider-specific request queues and locks - Add proper error handling and logging - Extend backoff patterns to all API providers - Add user feedback during rate limiting Fixes SakanaAI#155 Co-Authored-By: Erkin Alp Güney <[email protected]>
|
Thanx |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add comprehensive rate limit handling across API providers
This PR implements robust rate limit handling across all API providers used in the AI-Scientist framework, addressing the continuous retry issue (#155).
Changes
Implementation Details
rate_limit.pymodule for rate limit handlingTesting
The changes have been tested by:
Impact
These changes make the system more robust by:
Fixes #155
Link to Devin run: https://app.devin.ai/sessions/2ec43d6fe7a84849a348753167e5a895