-
-
Notifications
You must be signed in to change notification settings - Fork 23
Allow list of available xcache servers to be downloaded #1133
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds support for dynamically retrieving xcache server lists from a Virtual Placement Service (VPS) instead of relying on a static configuration. The implementation periodically checks the VPS liveness service to get an updated list of available servers, with configurable check intervals and automatic fallback handling.
- Introduces VPS-based xcache server discovery with liveness checking
- Adds configurable check intervals (default 30 minutes) for server list updates
- Provides Helm chart configuration options for VPS site specification
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.
File | Description |
---|---|
transformer_manager.py | Implements VPS server validation logic and dynamic cache prefix updates |
values.yaml | Adds configuration options for VPS site, check interval, and liveness URL |
configmap.yaml | Maps VPS configuration values to environment variables |
import time | ||
import urllib3 | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Imports should be placed at the top of the file rather than inside functions. Move the time and urllib3 imports to the module level.
import time | |
import urllib3 |
Copilot uses AI. Check for mistakes.
Add
values.yaml
options to allow the specification of a "Virtual Placement Service" site which gives a list of xcache servers to use. In principle this includes also a liveness test, so servers that go down should be removed dynamically from the list we use for transforms. This will be checked if the list hasn't been updated within a certain interval, by default 30 minutes. This new option can fully replace the server list given bycachePrefix
.This is not my favorite way to implement this, but since gunicorn spawns multiple workers which have distinct configuration data, this is the simplest way of ensuring that all app instances will eventually update. It would conceptually be cleaner to store global configuration data that might be updated in a single location (Redis?) but that's a much bigger refactoring.