Skip to content

feat: run test suites with YAML test files and config-driven gateway probing - #60

Merged
medcl merged 1 commit into
mainfrom
fix_probe_gateway
Aug 1, 2026
Merged

feat: run test suites with YAML test files and config-driven gateway probing#60
medcl merged 1 commit into
mainfrom
fix_probe_gateway

Conversation

@SteveLauC

@SteveLauC SteveLauC commented Jul 31, 2026

Copy link
Copy Markdown
Member

The test runner only looked for loadgen.dsl and silently fell back to paths relative to the loadgen config, while env variables defined in a per-test loadgen.yml were never substituted because template rendering used the top-level config's global environment, leaving request URLs pointing at "not_found" hosts. Gateway startup probing was equally fragile: it dialed two ports from LR_GATEWAY_HOST/LR_GATEWAY_API_HOST and treated the gateway as ready once either answered, which raced with multi-entry configurations and with the API listener coming up before the entry.

The runner now accepts either loadgen.dsl or loadgen.yml per test case (DSL preferred) and renders $[[env.X]] against merged environments, following the framework precedence OS env > per-test env > top-level env without mutating the global app config. A gateway.yml remains optional; when present, its listening addresses (the api listener plus every enabled entry) are derived by parsing it with the framework config loader so templates resolve exactly as the gateway binary resolves them, and the gateway is considered ready only once all of them accept connections. The gateway subprocess runs with the test directory as its working directory, startup failures include the captured gateway output, aborted tests surface their reason in the result summary, and request logging prints the final URL after template substitution instead of the raw template.

Standards checklist

  • The PR title is descriptive
  • The commit messages are semantic
  • Necessary tests are added
  • Updated the release notes
  • Necessary documents have been added if this is a new feature
  • Performance tests checked, no obvious performance degradation

@SteveLauC
SteveLauC marked this pull request as draft July 31, 2026 11:51
@SteveLauC
SteveLauC force-pushed the fix_probe_gateway branch from 1b2fc19 to 6a2fb21 Compare July 31, 2026 11:52
…probing

The test runner only looked for loadgen.dsl and silently fell back to
paths relative to the loadgen config, while env variables defined in a
per-test loadgen.yml were never substituted because template rendering
used the top-level config's global environment, leaving request URLs
pointing at "not_found" hosts. Gateway startup probing was equally
fragile: it dialed two ports from LR_GATEWAY_HOST/LR_GATEWAY_API_HOST
and treated the gateway as ready once either answered, which raced
with multi-entry configurations and with the API listener coming up
before the entry.

The runner now accepts either loadgen.dsl or loadgen.yml per test case
(DSL preferred) and renders $[[env.X]] against merged environments,
following the framework precedence OS env > per-test env > top-level
env without mutating the global app config. A gateway.yml remains
optional; when present, its listening addresses (the api listener plus
every enabled entry) are derived by parsing it with the framework
config loader so templates and configs.template references resolve
exactly as the gateway binary resolves them, and the gateway is
considered ready only once all of them accept connections. The gateway
subprocess runs with the test directory as its working directory,
startup failures include the captured gateway output, aborted tests
surface their reason in the result summary, and request logging prints
the final URL after template substitution instead of the raw template.
@SteveLauC
SteveLauC force-pushed the fix_probe_gateway branch from ef41b4c to 4613d3f Compare July 31, 2026 11:58
@SteveLauC
SteveLauC marked this pull request as ready for review July 31, 2026 12:00
@medcl
medcl merged commit e98ddd3 into main Aug 1, 2026
11 of 14 checks passed
@medcl
medcl deleted the fix_probe_gateway branch August 1, 2026 03:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants