Skip to content

Fix rate limit sleep duration calculation#1168

Open
Krishnachaitanyakc wants to merge 1 commit intoPythagora-io:mainfrom
Krishnachaitanyakc:fix/rate-limit-sleep-duration
Open

Fix rate limit sleep duration calculation#1168
Krishnachaitanyakc wants to merge 1 commit intoPythagora-io:mainfrom
Krishnachaitanyakc:fix/rate-limit-sleep-duration

Conversation

@Krishnachaitanyakc
Copy link
Copy Markdown

Summary

Fixes #1121

The rate limit handler in BaseLLMClient.__call__ used wait_time.seconds to compute the sleep duration, which is incorrect for timedelta objects that span multiple days. Python's timedelta.seconds only returns the seconds component within the current day (0-86399), not the total number of seconds:

  • timedelta(days=1) (daily rate limit reset) has .seconds = 0, causing a zero-second sleep that immediately retries and gets rate-limited again in a busy loop
  • A negative timedelta from clock drift (e.g. timedelta(seconds=-5) = timedelta(days=-1, seconds=86395)) has .seconds = 86395, causing a ~24-hour sleep

Changes

  • core/llm/base.py: Replace wait_time.seconds with int(wait_time.total_seconds()) and clamp the result to a 1-3600 second range. This prevents both zero-second busy loops and excessively long waits (over 1 hour) that make the tool unusable. Also adds log.info() so rate limit sleeps are always logged.
  • tests/llm/test_rate_limit.py: Add 8 new test cases covering normal durations, large values (capped at 3600s), daily rate limits, negative timedeltas, fractional seconds, and the case where rate_limit_sleep returns None.

Test plan

  • All 101 existing + new tests pass (python3 -m pytest tests/llm/ -v)
  • Verify with an actual rate-limited API key that the sleep duration is correct and within bounds
  • Confirm that insufficient-funds RateLimitError (where rate_limit_sleep returns None) still raises APIError without sleeping

Replace timedelta.seconds with int(total_seconds()) in the rate limit
handler to correctly compute sleep durations. The .seconds attribute
only returns the seconds component within the current day (0-86399),
which produces incorrect values for timedeltas spanning days (e.g.
daily rate limits returning timedelta(days=1) would sleep 0 seconds)
or negative timedeltas from clock drift (timedelta(seconds=-5) would
sleep ~86395 seconds).

The sleep duration is now clamped to a 1-3600 second range to prevent
both zero-second busy loops and excessively long waits that make the
tool unusable. Also adds a log.info() call so rate limit sleeps are
always visible in logs regardless of error_handler availability.

Fixes Pythagora-io#1121
@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Mar 25, 2026

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Rate limit Timeout Sleep WAY too long

2 participants