Skip to content

Allow list of available xcache servers to be downloaded #1133

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: develop
Choose a base branch
from
Open

Allow list of available xcache servers to be downloaded #1133

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a3b35cd
Update xcache list internally
ponyisi Aug 3, 2025
94e1254
Add documentation on new options
ponyisi Aug 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion docs/deployment/reference.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,10 @@ parameters for the [rabbitMQ](https://github.com/bitnami/charts/tree/master/bitn
| `minio.auth.rootPassword` | Password key to log into minio | leftfoot1 |
| `minio.apiIngress.enabled` | Should minio chart deploy an ingress to the service? | false |
| `minio.apiIngress.hostname` | Hostname associate with ingress controller | nil |
| `transformer.cachePrefix` | Prefix string to stick in front of file paths. Useful for XCache | |
| `transformer.cachePrefix` | Prefix string to stick in front of file paths. Useful for XCache | nil |
| `transformer.cacheVPSSite` | Specify a Virtual Placement Service site whose XCaches we should use. Will update automatically if list changes | nil |
| `transformer.cacheVPSCheckInterval` | How frequency should the Virtual Placement Service be consulted for the list of XCaches (in seconds) | 1800 |
| `transformer.cacheVPSLivenessURL` | URL from which Virtual Placement Service site info can be obtained | https://vps.cern.ch/liveness |
| `transformer.autoscaler.enabled` | Enable/disable horizontal pod autoscaler for transformers | True |
| `transformer.autoscaler.cpuScaleThreshold` | CPU percentage threshold for pod scaling | 30 |
| `transformer.autoscaler.minReplicas` | Minimum number of transformer pods per request | 1 |
Expand Down
5 changes: 5 additions & 0 deletions helm/servicex/templates/app/configmap.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,11 @@ data:
{{- else }}
TRANSFORMER_CACHE_PREFIX = ""
{{- end }}
{{- if .Values.transformer.cacheVPSSite }}
TRANSFORMER_CACHE_VPS_SITE = "{{ .Values.transformer.cacheVPSSite }}"
TRANSFORMER_CACHE_VPS_INTERVAL = {{ .Values.transformer.cacheVPSCheckInterval }}
TRANSFORMER_CACHE_VPS_LIVENESS_URL = "{{ .Values.transformer.cacheVPSLivenessURL }}"
{{- end }}
TRANSFORMER_AUTOSCALE_ENABLED = {{- ternary "True" "False" .Values.transformer.autoscaler.enabled }}
TRANSFORMER_CPU_LIMIT = {{ .Values.transformer.cpuLimit }}
TRANSFORMER_MEMORY_LIMIT = "{{ .Values.transformer.memoryLimit }}"
Expand Down
6 changes: 6 additions & 0 deletions helm/servicex/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,12 @@ transformer:
# comma-separated list of values.
# Do not put root:// in these values.
cachePrefix: null
# Alternatively, provide the virtual placement service site name to look up
# xCache servers from
cacheVPSSite: null
cacheVPSCheckInterval: 1800
cacheVPSLivenessURL: https://vps.cern.ch/liveness


autoscaler:
cpuScaleThreshold: 30
Expand Down
31 changes: 30 additions & 1 deletion servicex_app/servicex_app/transformer_manager.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -220,7 +220,11 @@ def create_job_object(
env = env + [env_var_instance_name]

# provide each pod with an environment var holding cache prefix path
if "TRANSFORMER_CACHE_PREFIX" in current_app.config:
if (
"TRANSFORMER_CACHE_PREFIX" in current_app.config
or "TRANSFORMER_CACHE_VPS_SITE" in current_app.config
):
TransformerManager.validate_caches()
env += [
client.V1EnvVar(
"CACHE_PREFIX", value=current_app.config["TRANSFORMER_CACHE_PREFIX"]
Expand Down Expand Up @@ -410,6 +414,31 @@ def persistent_volume_claim_exists(self, claim_name, namespace):
return False
return False

@staticmethod
def validate_caches():
import time
import urllib3

Comment on lines +419 to +421
Copy link
Preview

Copilot AI Aug 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Imports should be placed at the top of the file rather than inside functions. Move the time and urllib3 imports to the module level.

Suggested change
import time
import urllib3

Copilot uses AI. Check for mistakes.

if not (thissite := current_app.config.get("TRANSFORMER_CACHE_VPS_SITE", None)):
return

lastchecktime = current_app.config.get("TRANSFORMER_CACHE_VPS_LASTCHECK", 0)
ttl = current_app.config.get("TRANSFORMER_CACHE_VPS_INTERVAL", 30 * 60)
if (now := time.time()) - lastchecktime > ttl:
current_app.config["TRANSFORMER_CACHE_VPS_LASTCHECK"] = now
vps_server = current_app.config["TRANSFORMER_CACHE_VPS_LIVENESS_URL"]
try:
sitedata = urllib3.request("GET", vps_server).json()
if thissite not in sitedata:
current_app.logger.error(f"{thissite} is not in VPS liveness data")
servers = sorted(
[_["address"] for _ in sitedata[thissite].values() if _["live"]]
)
current_app.logger.info(f"Live xCache servers are {servers}")
current_app.config["TRANSFORMER_CACHE_PREFIX"] = ",".join(servers)
except Exception as e:
current_app.logger.error(f"Exception retrieving site data: {e}")

@staticmethod
def create_hpa_object(request_id, max_workers):
target = client.V1CrossVersionObjectReference(
Expand Down