Add multi-URL support and per-resolution url param to Hub Resolver

  Implements per-resolution  parameter
  for explicit hub selection and ConfigMap-based URL lists with ordered
  fallback for cluster-level defaults.

  - Add  param for per-resolution hub endpoint override
  - Add  /  ConfigMap keys for
    ordered URL lists with first-success-wins fallback
  - Use TrimRight for trailing slash stripping on ConfigMap URLs
  - Add unit tests for fallback, url override, error formatting
  - Add e2e tests for url param and version constraint resolution
  - Update docs with new param, ConfigMap options, and URL precedence

Signed-off-by: ab-ghosh <abghosh@redhat.com>

Author: ab-ghosh

config/resolvers/hubresolver-config.yaml

@@ -32,3 +32,12 @@ data:
   default-kind: "task"
   # the default hub source to pull the resource from.
   default-type: "artifact"
+  # Ordered list of Artifact Hub API URLs to try. First successful response wins.
+  # If not set, the ARTIFACT_HUB_API env var or default (https://artifacthub.io) is used.
+  # artifact-hub-urls: |
+  #   - https://internal-hub.example.com/
+  #   - https://artifacthub.io/
+  # Ordered list of Tekton Hub API URLs to try. First successful response wins.
+  # If not set, the TEKTON_HUB_API env var is used.
+  # tekton-hub-urls: |
+  #   - https://api.hub.tekton.dev/

docs/hub-resolver.md

@@ -18,6 +18,7 @@ Use resolver type `hub`.
 | `kind`           | Either `task` or `pipeline` (Optional)                                        | Default: `task`                                                     |
 | `name`           | The name of the task or pipeline to fetch from the hub                        | `golang-build`                                             |
 | `version`        | Version or a Constraint (see [below](#version-constraint) of a task or a pipeline to pull in from. Wrap the number in quotes!   | `"0.5.0"`, `">= 0.5.0"`                                                    |
+| `url`            | Custom hub API endpoint to query instead of the cluster-configured default (Optional). Must be an absolute HTTP or HTTPS URL. Overrides all other URL configuration (ConfigMap URL lists, environment variables, and defaults). | `https://internal-hub.example.com`                        |
 
 The Catalogs in the Artifact Hub follows the semVer (i.e.` <major-version>.<minor-version>.0`) and the Catalogs in the Tekton Hub follows the simplified semVer (i.e. `<major-version>.<minor-version>`). Both full and simplified semantic versioning will be accepted by the `version` parameter. The Hub Resolver will map the version to the format expected by the target Hub `type`.
 
@@ -44,6 +45,8 @@ for the name, namespace and defaults that the resolver ships with.
 | `default-artifact-hub-pipeline-catalog`| The default artifact hub catalog from where to pull the resource for pipeline kind.  | `tekton-catalog-pipelines`               |
 | `default-kind`              | The default object kind for references.              | `task`, `pipeline`     |
 | `default-type`              | The default hub from where to pull the resource.     | `artifact`, `tekton`   |
+| `artifact-hub-urls`         | Ordered YAML list of Artifact Hub API URLs to try. First successful response wins. If not set, the `ARTIFACT_HUB_API` env var or default is used. | See [below](#configuring-multiple-hub-urls) |
+| `tekton-hub-urls`           | Ordered YAML list of Tekton Hub API URLs to try. First successful response wins. If not set, the `TEKTON_HUB_API` env var is used. | See [below](#configuring-multiple-hub-urls) |
 
 ### Configuring the Hub API endpoint
 
@@ -128,6 +131,70 @@ spec:
   # overall will not succeed without those parameters.
 ```
 
+### Task Resolution from a Private Hub
+
+```yaml
+apiVersion: tekton.dev/v1beta1
+kind: TaskRun
+metadata:
+  name: private-hub-task-reference
+spec:
+  taskRef:
+    resolver: hub
+    params:
+    - name: url
+      value: https://internal-hub.example.com
+    - name: catalog
+      value: my-team-catalog
+    - name: type
+      value: artifact
+    - name: kind
+      value: task
+    - name: name
+      value: my-task
+    - name: version
+      value: "1.0.0"
+```
+
+When the `url` parameter is not specified, the resolver falls back to the
+cluster-configured defaults: first checking the ConfigMap URL lists
+(`artifact-hub-urls` / `tekton-hub-urls`), then the environment variables
+(`ARTIFACT_HUB_API` / `TEKTON_HUB_API`). This allows Pipeline authors to
+explicitly choose which hub instance to query on a per-resolution basis, which
+is useful in multi-team environments or air-gapped deployments where resources
+are hosted on private hub instances.
+
+### Configuring Multiple Hub URLs
+
+You can configure multiple hub API URLs in the `hubresolver-config` ConfigMap.
+The resolver will try each URL in order and return the first successful result.
+This is useful for layered catalogs, multi-team environments, or air-gapped
+deployments where resources may exist on different hub instances.
+
+```yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: hubresolver-config
+  namespace: tekton-pipelines-resolvers
+data:
+  artifact-hub-urls: |
+    - https://internal-hub.example.com/
+    - https://artifacthub.io/
+  tekton-hub-urls: |
+    - https://internal-tekton-hub.example.com/
+    - https://api.hub.tekton.dev/
+```
+
+**URL precedence** (highest to lowest):
+1. Per-resolution `url` parameter — single URL override for a specific resolution request
+2. ConfigMap URL lists (`artifact-hub-urls` / `tekton-hub-urls`) — tried in order, first success wins
+3. Environment variable (`ARTIFACT_HUB_API` / `TEKTON_HUB_API`) — single URL fallback
+4. Default URL (`https://artifacthub.io` for artifact type)
+
+If the ConfigMap URL list keys are not set (commented out by default), the
+resolver behaves exactly as before, using the environment variable or default URL.
+
 ### Version constraint
 
 Instead of a version you can specify a constraint to choose from. The constraint is a string as documented in the [go-version](https://github.com/hashicorp/go-version) library.

pkg/resolution/resolver/hub/config.go

@@ -16,6 +16,13 @@ limitations under the License.
 
 package hub
 
+import (
+	"fmt"
+	"strings"
+
+	"sigs.k8s.io/yaml"
+)
+
 // ConfigTektonHubCatalog is the configuration field name for controlling
 // the Tekton Hub catalog to fetch the remote resource from.
 const ConfigTektonHubCatalog = "default-tekton-hub-catalog"
@@ -35,3 +42,31 @@ const ConfigKind = "default-kind"
 // ConfigType is the configuration field name for controlling
 // the hub type to pull the resource from.
 const ConfigType = "default-type"
+
+// ConfigArtifactHubURLs is the configuration field name for controlling
+// the Artifact Hub API URLs to fetch remote resources from.
+// Value is a YAML list of URLs, tried in order; first success wins.
+const ConfigArtifactHubURLs = "artifact-hub-urls"
+
+// ConfigTektonHubURLs is the configuration field name for controlling
+// the Tekton Hub API URLs to fetch remote resources from.
+// Value is a YAML list of URLs, tried in order; first success wins.
+const ConfigTektonHubURLs = "tekton-hub-urls"
+
+// parseURLList parses a YAML list string from a ConfigMap value into
+// a slice of URL strings. Each URL is trimmed of whitespace and trailing slashes.
+// Returns nil if the input is empty or not a valid YAML list.
+func parseURLList(yamlList string) ([]string, error) {
+	yamlList = strings.TrimSpace(yamlList)
+	if yamlList == "" {
+		return nil, nil
+	}
+	var urls []string
+	if err := yaml.Unmarshal([]byte(yamlList), &urls); err != nil {
+		return nil, fmt.Errorf("failed to parse URL list: %w", err)
+	}
+	for i, u := range urls {
+		urls[i] = strings.TrimRight(strings.TrimSpace(u), "/")
+	}
+	return urls, nil
+}

pkg/resolution/resolver/hub/params.go

@@ -48,3 +48,9 @@ const ParamCatalog = "catalog"
 
 // ParamType is the parameter defining what the hub type to pull the resource from.
 const ParamType = "type"
+
+// ParamURL is the parameter defining a custom hub API endpoint to use
+// instead of the cluster-configured default. When specified, it overrides
+// the ARTIFACT_HUB_API or TEKTON_HUB_API environment variable based on the
+// resolution type.
+const ParamURL = resource.ParamURL

pkg/resolution/resolver/hub/resolver.go

@@ -22,6 +22,7 @@ import (
 	"fmt"
 	"io"
 	"net/http"
+	"net/url"
 	"slices"
 	"strings"
 
@@ -135,12 +136,34 @@ func Resolve(ctx context.Context, params []pipelinev1.Param, tektonHubURL, artif
 		return nil, fmt.Errorf("failed to validate params: %w", err)
 	}
 
+	// Determine ordered list of hub URLs to try.
+	// Precedence: url param (Option A) > ConfigMap YAML list (Option B) > env var URL.
+	var urls []string
+	if paramURL := paramsMap[ParamURL]; paramURL != "" {
+		// Per-resolution URL override takes highest precedence.
+		urls = []string{strings.TrimSuffix(paramURL, "/")}
+	} else {
+		conf := framework.GetResolverConfigFromContext(ctx)
+		switch paramsMap[ParamType] {
+		case ArtifactHubType:
+			urls = resolveHubURLs(conf, artifactHubURL, ConfigArtifactHubURLs)
+		case TektonHubType:
+			urls = resolveHubURLs(conf, tektonHubURL, ConfigTektonHubURLs)
+		}
+	}
+
+	if len(urls) == 0 {
+		return nil, fmt.Errorf("no hub URL configured for type %s", paramsMap[ParamType])
+	}
+
 	if constraint, err := goversion.NewConstraint(paramsMap[ParamVersion]); err == nil {
-		chosen, err := resolveVersionConstraint(ctx, paramsMap, constraint, artifactHubURL, tektonHubURL)
+		chosen, constraintURL, err := resolveVersionConstraintWithFallback(ctx, paramsMap, constraint, urls)
 		if err != nil {
 			return nil, err
 		}
 		paramsMap[ParamVersion] = chosen.String()
+		// Pin subsequent fetch to the same hub that satisfied the constraint.
+		urls = []string{constraintURL}
 	}
 
 	resVer, err := resolveVersion(paramsMap[ParamVersion], paramsMap[ParamType])
@@ -149,33 +172,7 @@ func Resolve(ctx context.Context, params []pipelinev1.Param, tektonHubURL, artif
 	}
 	paramsMap[ParamVersion] = resVer
 
-	// call hub API
-	switch paramsMap[ParamType] {
-	case ArtifactHubType:
-		url := fmt.Sprintf(fmt.Sprintf("%s/%s", artifactHubURL, ArtifactHubYamlEndpoint),
-			paramsMap[ParamKind], paramsMap[ParamCatalog], paramsMap[ParamName], paramsMap[ParamVersion])
-		resp := artifactHubResponse{}
-		if err := fetchHubResource(ctx, url, &resp); err != nil {
-			return nil, fmt.Errorf("fail to fetch Artifact Hub resource: %w", err)
-		}
-		return &ResolvedHubResource{
-			URL:     url,
-			Content: []byte(resp.Data.YAML),
-		}, nil
-	case TektonHubType:
-		url := fmt.Sprintf(fmt.Sprintf("%s/%s", tektonHubURL, TektonHubYamlEndpoint),
-			paramsMap[ParamCatalog], paramsMap[ParamKind], paramsMap[ParamName], paramsMap[ParamVersion])
-		resp := tektonHubResponse{}
-		if err := fetchHubResource(ctx, url, &resp); err != nil {
-			return nil, fmt.Errorf("fail to fetch Tekton Hub resource: %w", err)
-		}
-		return &ResolvedHubResource{
-			URL:     url,
-			Content: []byte(resp.Data.YAML),
-		}, nil
-	}
-
-	return nil, fmt.Errorf("hub resolver type: %s is not supported", paramsMap[ParamType])
+	return fetchResourceWithFallback(ctx, paramsMap, urls)
 }
 
 // ResolvedHubResource wraps the data we want to return to Pipelines
@@ -367,13 +364,29 @@ func validateParams(ctx context.Context, paramsMap map[string]string, tektonHubU
 			return fmt.Errorf("kind param must be one of: %s", strings.Join(supportedKinds, ", "))
 		}
 	}
+
+	if paramURL, ok := paramsMap[ParamURL]; ok && paramURL != "" {
+		u, err := url.ParseRequestURI(paramURL)
+		if err != nil || u.Scheme == "" || u.Host == "" {
+			return fmt.Errorf("url param must be a valid absolute URL: %s", paramURL)
+		}
+		if u.Scheme != "http" && u.Scheme != "https" {
+			return fmt.Errorf("url param must be a valid http(s) URL: %s", paramURL)
+		}
+	}
+
+	hasURLOverride := paramsMap[ParamURL] != ""
 	if hubType, ok := paramsMap[ParamType]; ok {
 		if hubType != ArtifactHubType && hubType != TektonHubType {
 			return fmt.Errorf("type param must be %s or %s", ArtifactHubType, TektonHubType)
 		}
 
-		if hubType == TektonHubType && tektonHubURL == "" {
-			return errors.New("please configure TEKTON_HUB_API env variable to use tekton type")
+		if hubType == TektonHubType && tektonHubURL == "" && !hasURLOverride {
+			conf := framework.GetResolverConfigFromContext(ctx)
+			configURLs, _ := parseURLList(conf[ConfigTektonHubURLs])
+			if len(configURLs) == 0 {
+				return errors.New("please configure TEKTON_HUB_API env variable to use tekton type")
+			}
 		}
 	}
 
@@ -384,10 +397,73 @@ func validateParams(ctx context.Context, paramsMap map[string]string, tektonHubU
 	return nil
 }
 
-func resolveVersionConstraint(ctx context.Context, paramsMap map[string]string, constraint goversion.Constraints, artifactHubURL, tektonHubURL string) (*goversion.Version, error) {
+// resolveHubURLs returns the ordered list of hub URLs to try.
+// Precedence: ConfigMap YAML list > env var URL.
+func resolveHubURLs(conf map[string]string, envVarURL, configKey string) []string {
+	if yamlStr, ok := conf[configKey]; ok && strings.TrimSpace(yamlStr) != "" {
+		urls, err := parseURLList(yamlStr)
+		if err == nil && len(urls) > 0 {
+			return urls
+		}
+	}
+	if envVarURL != "" {
+		return []string{envVarURL}
+	}
+	return nil
+}
+
+// fetchResourceFromURL fetches a resource from a single hub URL.
+func fetchResourceFromURL(ctx context.Context, paramsMap map[string]string, baseURL string) (framework.ResolvedResource, error) {
+	switch paramsMap[ParamType] {
+	case ArtifactHubType:
+		url := fmt.Sprintf(fmt.Sprintf("%s/%s", baseURL, ArtifactHubYamlEndpoint),
+			paramsMap[ParamKind], paramsMap[ParamCatalog], paramsMap[ParamName], paramsMap[ParamVersion])
+		resp := artifactHubResponse{}
+		if err := fetchHubResource(ctx, url, &resp); err != nil {
+			return nil, fmt.Errorf("fail to fetch Artifact Hub resource: %w", err)
+		}
+		return &ResolvedHubResource{
+			URL:     url,
+			Content: []byte(resp.Data.YAML),
+		}, nil
+	case TektonHubType:
+		url := fmt.Sprintf(fmt.Sprintf("%s/%s", baseURL, TektonHubYamlEndpoint),
+			paramsMap[ParamCatalog], paramsMap[ParamKind], paramsMap[ParamName], paramsMap[ParamVersion])
+		resp := tektonHubResponse{}
+		if err := fetchHubResource(ctx, url, &resp); err != nil {
+			return nil, fmt.Errorf("fail to fetch Tekton Hub resource: %w", err)
+		}
+		return &ResolvedHubResource{
+			URL:     url,
+			Content: []byte(resp.Data.YAML),
+		}, nil
+	}
+	return nil, fmt.Errorf("hub resolver type: %s is not supported", paramsMap[ParamType])
+}
+
+// fetchResourceWithFallback tries each URL in order, returns the first successful result.
+// When there is only one URL, error messages are preserved exactly as before.
+func fetchResourceWithFallback(ctx context.Context, paramsMap map[string]string, urls []string) (framework.ResolvedResource, error) {
+	var errs []error
+	for _, baseURL := range urls {
+		result, err := fetchResourceFromURL(ctx, paramsMap, baseURL)
+		if err != nil {
+			errs = append(errs, err)
+			continue
+		}
+		return result, nil
+	}
+	if len(errs) == 1 {
+		return nil, errs[0]
+	}
+	return nil, fmt.Errorf("failed to fetch resource from any configured hub URL: %w", errors.Join(errs...))
+}
+
+// resolveVersionConstraintFromURL resolves a version constraint from a single hub URL.
+func resolveVersionConstraintFromURL(ctx context.Context, paramsMap map[string]string, constraint goversion.Constraints, baseURL string) (*goversion.Version, error) {
 	var ret *goversion.Version
 	if paramsMap[ParamType] == ArtifactHubType {
-		allVersionsURL := fmt.Sprintf("%s/%s", artifactHubURL, fmt.Sprintf(
+		allVersionsURL := fmt.Sprintf("%s/%s", baseURL, fmt.Sprintf(
 			ArtifactHubListTasksEndpoint,
 			paramsMap[ParamKind], paramsMap[ParamCatalog], paramsMap[ParamName]))
 		resp := artifactHubListResult{}
@@ -409,12 +485,11 @@ func resolveVersionConstraint(ctx context.Context, paramsMap map[string]string,
 				if ret != nil && ret.GreaterThan(checkV) {
 					continue
 				}
-				// TODO(chmouel): log constraint result in controller
 				ret = checkV
 			}
 		}
 	} else if paramsMap[ParamType] == TektonHubType {
-		allVersionsURL := fmt.Sprintf("%s/%s", tektonHubURL,
+		allVersionsURL := fmt.Sprintf("%s/%s", baseURL,
 			fmt.Sprintf(TektonHubListTasksEndpoint,
 				paramsMap[ParamCatalog], paramsMap[ParamKind], paramsMap[ParamName]))
 		resp := tektonHubListResult{}
@@ -433,7 +508,6 @@ func resolveVersionConstraint(ctx context.Context, paramsMap map[string]string,
 				if ret != nil && ret.GreaterThan(checkV) {
 					continue
 				}
-				// TODO(chmouel): log constraint result in controller
 				ret = checkV
 			}
 		}
@@ -444,6 +518,26 @@ func resolveVersionConstraint(ctx context.Context, paramsMap map[string]string,
 	return ret, nil
 }
 
+// resolveVersionConstraintWithFallback tries each URL in order for version constraint resolution.
+// Returns the resolved version and the URL that satisfied the constraint, so the caller can
+// pin subsequent fetches to the same hub.
+// When there is only one URL, error messages are preserved exactly as before.
+func resolveVersionConstraintWithFallback(ctx context.Context, paramsMap map[string]string, constraint goversion.Constraints, urls []string) (*goversion.Version, string, error) {
+	var errs []error
+	for _, baseURL := range urls {
+		ver, err := resolveVersionConstraintFromURL(ctx, paramsMap, constraint, baseURL)
+		if err != nil {
+			errs = append(errs, err)
+			continue
+		}
+		return ver, baseURL, nil
+	}
+	if len(errs) == 1 {
+		return nil, "", errs[0]
+	}
+	return nil, "", fmt.Errorf("failed to resolve version constraint from any configured hub URL: %w", errors.Join(errs...))
+}
+
 func isSupportedKind(kindValue string) bool {
 	return slices.Contains[[]string, string](supportedKinds, kindValue)
 }