feat: run test suites with YAML test files and config-driven gateway probing - #60
Merged
Conversation
SteveLauC
marked this pull request as draft
July 31, 2026 11:51
SteveLauC
force-pushed
the
fix_probe_gateway
branch
from
July 31, 2026 11:52
1b2fc19 to
6a2fb21
Compare
…probing The test runner only looked for loadgen.dsl and silently fell back to paths relative to the loadgen config, while env variables defined in a per-test loadgen.yml were never substituted because template rendering used the top-level config's global environment, leaving request URLs pointing at "not_found" hosts. Gateway startup probing was equally fragile: it dialed two ports from LR_GATEWAY_HOST/LR_GATEWAY_API_HOST and treated the gateway as ready once either answered, which raced with multi-entry configurations and with the API listener coming up before the entry. The runner now accepts either loadgen.dsl or loadgen.yml per test case (DSL preferred) and renders $[[env.X]] against merged environments, following the framework precedence OS env > per-test env > top-level env without mutating the global app config. A gateway.yml remains optional; when present, its listening addresses (the api listener plus every enabled entry) are derived by parsing it with the framework config loader so templates and configs.template references resolve exactly as the gateway binary resolves them, and the gateway is considered ready only once all of them accept connections. The gateway subprocess runs with the test directory as its working directory, startup failures include the captured gateway output, aborted tests surface their reason in the result summary, and request logging prints the final URL after template substitution instead of the raw template.
SteveLauC
force-pushed
the
fix_probe_gateway
branch
from
July 31, 2026 11:58
ef41b4c to
4613d3f
Compare
SteveLauC
marked this pull request as ready for review
July 31, 2026 12:00
medcl
approved these changes
Jul 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The test runner only looked for loadgen.dsl and silently fell back to paths relative to the loadgen config, while env variables defined in a per-test loadgen.yml were never substituted because template rendering used the top-level config's global environment, leaving request URLs pointing at "not_found" hosts. Gateway startup probing was equally fragile: it dialed two ports from LR_GATEWAY_HOST/LR_GATEWAY_API_HOST and treated the gateway as ready once either answered, which raced with multi-entry configurations and with the API listener coming up before the entry.
The runner now accepts either loadgen.dsl or loadgen.yml per test case (DSL preferred) and renders $[[env.X]] against merged environments, following the framework precedence OS env > per-test env > top-level env without mutating the global app config. A gateway.yml remains optional; when present, its listening addresses (the api listener plus every enabled entry) are derived by parsing it with the framework config loader so templates resolve exactly as the gateway binary resolves them, and the gateway is considered ready only once all of them accept connections. The gateway subprocess runs with the test directory as its working directory, startup failures include the captured gateway output, aborted tests surface their reason in the result summary, and request logging prints the final URL after template substitution instead of the raw template.
Standards checklist