ai/live: Remote signer discovery

[j0sh]

This adds an optional remote discovery feature to the remote signer node. Client using the remote signer can now discover orchestrators and their supported capabilities on the network.

Usage

• Remote signer node gets a -remoteDiscovery flag. This is only usable in remote signer mode for now and is opt-in. All the other gateway-side orchestrator configuration flags are supported here: -orchWebhookUrl, -orchAddr , -orchBlocklist, -extraNodes etc.

• If the gateway is using a remote signer (via -remoteSignerUrl) and has no other orchestrator sources (eg, webhook, orchestrator list) then it will fall back to using the remote signer's discovery endpoint.

• The discovery endpoint is GET /discover-orchestrators. It returns a JSON list of [{"address":...,"capabilities":[...]}] where the address is the URI the orchestrator can be reached at, and the capabilities is a list of capabilities the orchestrator supports, eg live-video-to-video/streamdiffusion. There is also an optional caps query parameter to return only the subset of orchestrators matching the capability (exact string match). Multiple caps parameters can be included (OR).

Remote Discovery Implementation

• Remote discovery refreshes its orchestrator list (via GetOrchestrator calls) every minute or so, to maintain an up-to-date view of orchestrator capabilities
• It builds a map of capabilities => orchestrators during each refresh for quick capability lookups
• Price filtering happens during refresh, so it doesn't return orchestrators that will be later rejected by the remote signer due to price

Supporting Changes

• DB Discovery pool and Webhook Discovery pool are updated to use a "builder pattern" of config structs + initializer, matching the regular orchestrator pool. The existing constructor interfaces are left untouched, but the plumbing underneath uses the new builder pattern. This is so we can support two things: the IgnoreCapacityCheck and the [TODO: DB Discovery refresh]

• Remote discovery uses the GetOrchestrators() code instead of re-implementing half of what it does. However, to avoid returning capacity errors during the call (it is capabilities we want), we need to thread through the IgnoreCapacityCheck flag a couple places from the webhook, DB discovery, etc.

• Tests, docs, etc.

[codecov]

Codecov Report

❌ Patch coverage is 62.54545% with 103 lines in your changes missing coverage. Please review.
✅ Project coverage is 32.66951%. Comparing base (f117b44) to head (c96a93d).
⚠️ Report is 2 commits behind head on master.

Files with missing lines	Patch %	Lines
cmd/livepeer/starter/starter.go	3.92157%	49 Missing ⚠️
server/remote_discovery.go	68.70229%	34 Missing and 7 partials ⚠️
server/remote_signer.go	71.87500%	8 Missing and 1 partial ⚠️
discovery/db_discovery.go	90.00000%	1 Missing and 2 partials ⚠️
cmd/livepeer/starter/flags.go	0.00000%	1 Missing ⚠️

Additional details and impacted files

@@                 Coverage Diff                 @@
##              master       #3866         +/-   ##
===================================================
+ Coverage   32.47906%   32.66951%   +0.19045%     
===================================================
  Files            170         171          +1     
  Lines          41673       41929        +256     
===================================================
+ Hits           13535       13698        +163     
- Misses         27113       27199         +86     
- Partials        1025        1032          +7     

Files with missing lines	Coverage Δ
discovery/discovery.go	84.12698% <100.00000%> (+0.34320%)	⬆️
discovery/wh_discovery.go	67.67677% <100.00000%> (+4.88607%)	⬆️
cmd/livepeer/starter/flags.go	0.00000% <0.00000%> (ø)
discovery/db_discovery.go	67.08075% <90.00000%> (+1.98008%)	⬆️
server/remote_signer.go	62.54072% <71.87500%> (+1.08617%)	⬆️
server/remote_discovery.go	68.70229% <68.70229%> (ø)
cmd/livepeer/starter/starter.go	21.85386% <3.92157%> (-0.64088%)	⬇️

... and 5 files with indirect coverage changes

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f117b44...c96a93d. Read the comment docs.

Files with missing lines	Coverage Δ
discovery/discovery.go	84.12698% <100.00000%> (+0.34320%)	⬆️
discovery/wh_discovery.go	67.67677% <100.00000%> (+4.88607%)	⬆️
cmd/livepeer/starter/flags.go	0.00000% <0.00000%> (ø)
discovery/db_discovery.go	67.08075% <90.00000%> (+1.98008%)	⬆️
server/remote_signer.go	62.54072% <71.87500%> (+1.08617%)	⬆️
server/remote_discovery.go	68.70229% <68.70229%> (ø)
cmd/livepeer/starter/starter.go	21.85386% <3.92157%> (-0.64088%)	⬇️

... and 5 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.