[feat] Implement Advanced Throttling & Rate Limiting with Dynamic Adjustment

[DhanashreePetare]

Description:

The current throttler system in iDDS lacks enforcement and intelligent rate limiting:

Missing Features:

• Dynamic rate limiting based on system load
• Token bucket / sliding window algorithms
• Priority queue for request ordering
• Per-User and Per-VO rate limits
• Backpressure handling with auto-scaling
• 429 Too Many Requests responses

Current Gap:
throttlers.py only has basic add/get operations. No enforcement logic exists to actually limit requests based on configured thresholds.

What Needs to be Done:

1. Add rate limiter enforcer in Clerk agent
2. Implement token bucket algorithm
3. Create priority queue management
4. Support per-VO/user limits from database
5. Add rate limit headers to API responses
6. Detect system overload and reject gracefully

Why It Matters:
Essential for multi-tenant production systems to prevent resource exhaustion and ensure fair resource allocation across users and organizations.

[AggarwalKashvi]

Hello! I would like to contribute to this issue.

I have experience with Python, including threading, and I can work with SQLAlchemy . I will ensure correct handling of API errors.
I understand the issue and I can implement this with the existing iDDS architecture

can I help with this?

[RitikaSen27]

Hello,

I am interested in the work on this problem. I have read the existing implementation of throttlers.py, and I have the sense that it is only a simple add/get implementation and nothing beyond that.

I intend to introduce a real rate-limiting enforcer into the Clerk agent whereby the requests are either accepted or rejected when they are made. The method that I am planning is:

Add a token-bucket rate limiter to allow bursting and steady-state restrictions.

Impose per-user and per-VO rate limits based on policies retrieved out of the database (cached in memory)

Allow request ordering based on priority such that high priority workflows do not get starved.

Identify system overload (e.g. queue depth / active workflows) and de-prioritize with a backpressure (i.e. reject or deprioritize low-priority requests).

Send correct 429 Too many Requests responses and default rate-limit correctors.

I will introduce it in phases and begin with the token bucket and Clerk enforcement hook, then priority handling, and system conscious backpressure.

I want to ask whether this solution is in the direction of the project or there are certain design limitations that I should observe prior to the commencement.

Thanks

[DhanashreePetare]

@AggarwalKashvi, and @RitikaSen27, I don't have these permissions. I opened it but want to get the issue verified by the authors first. @wguanicedew, can you help?