fix: CLI error reporting for cmd? and cd with absolute paths - #129
Merged
Conversation
Add comprehensive automated test suite for milk-cli: - tests/cli/cli_robustness_tests.milk: 109 annotated test cases covering basic parsing, help/discovery, builtins, missing/wrong arguments, variables, arithmetic, flow control, user functions, command chaining, shell features (pipes, redirects, here-strings, command substitution), error conditions, set flags, readonly/declare. - tests/cli/run_cli_robustness_tests.sh: Bash test runner that feeds tests one-by-one to milk-cli, captures output, checks for crashes/hangs/missing error messages, and prints a summary report. - .agents/workflows/cli-robustness-test.md: Workflow for running the test suite. Initial run: 105 pass, 4 MISSING_ERROR (no error message printed for: cmd? with unknown command, cd to nonexistent directory, mem.mk2Dim with no args, mem.rm with no args).
Fix two CLI error reporting bugs found by the robustness test suite: 1. help_command() (cmd?) now returns RETURN_FAILURE when the queried command is not found. Previously it always returned RETURN_SUCCESS even when printing "does not exist", misleading the caller. 2. Extend tokenizer raw-string bypass to treat arguments starting with '/' as raw strings (same as '-' prefixed arguments). This fixes cd /absolute/path where the '/' was misinterpreted as division by cli_parse(), causing a silent parse error that skipped command execution entirely. Update test expectations: mem.mk2Dim and mem.rm with no arguments correctly create a local FPS with defaults — this is expected FPS behavior, not an error. Also add 'does not exist' to the runner grep pattern for error detection.
oguyon
added a commit
that referenced
this pull request
May 21, 2026
* feat: add CLI robustness test suite Add comprehensive automated test suite for milk-cli: - tests/cli/cli_robustness_tests.milk: 109 annotated test cases covering basic parsing, help/discovery, builtins, missing/wrong arguments, variables, arithmetic, flow control, user functions, command chaining, shell features (pipes, redirects, here-strings, command substitution), error conditions, set flags, readonly/declare. - tests/cli/run_cli_robustness_tests.sh: Bash test runner that feeds tests one-by-one to milk-cli, captures output, checks for crashes/hangs/missing error messages, and prints a summary report. - .agents/workflows/cli-robustness-test.md: Workflow for running the test suite. Initial run: 105 pass, 4 MISSING_ERROR (no error message printed for: cmd? with unknown command, cd to nonexistent directory, mem.mk2Dim with no args, mem.rm with no args). * fix: CLI error reporting for cmd? and cd with absolute paths Fix two CLI error reporting bugs found by the robustness test suite: 1. help_command() (cmd?) now returns RETURN_FAILURE when the queried command is not found. Previously it always returned RETURN_SUCCESS even when printing "does not exist", misleading the caller. 2. Extend tokenizer raw-string bypass to treat arguments starting with '/' as raw strings (same as '-' prefixed arguments). This fixes cd /absolute/path where the '/' was misinterpreted as division by cli_parse(), causing a silent parse error that skipped command execution entirely. Update test expectations: mem.mk2Dim and mem.rm with no arguments correctly create a local FPS with defaults — this is expected FPS behavior, not an error. Also add 'does not exist' to the runner grep pattern for error detection.
oguyon
added a commit
that referenced
this pull request
May 21, 2026
* feat: add CLI robustness test suite Add comprehensive automated test suite for milk-cli: - tests/cli/cli_robustness_tests.milk: 109 annotated test cases covering basic parsing, help/discovery, builtins, missing/wrong arguments, variables, arithmetic, flow control, user functions, command chaining, shell features (pipes, redirects, here-strings, command substitution), error conditions, set flags, readonly/declare. - tests/cli/run_cli_robustness_tests.sh: Bash test runner that feeds tests one-by-one to milk-cli, captures output, checks for crashes/hangs/missing error messages, and prints a summary report. - .agents/workflows/cli-robustness-test.md: Workflow for running the test suite. Initial run: 105 pass, 4 MISSING_ERROR (no error message printed for: cmd? with unknown command, cd to nonexistent directory, mem.mk2Dim with no args, mem.rm with no args). * fix: CLI error reporting for cmd? and cd with absolute paths Fix two CLI error reporting bugs found by the robustness test suite: 1. help_command() (cmd?) now returns RETURN_FAILURE when the queried command is not found. Previously it always returned RETURN_SUCCESS even when printing "does not exist", misleading the caller. 2. Extend tokenizer raw-string bypass to treat arguments starting with '/' as raw strings (same as '-' prefixed arguments). This fixes cd /absolute/path where the '/' was misinterpreted as division by cli_parse(), causing a silent parse error that skipped command execution entirely. Update test expectations: mem.mk2Dim and mem.rm with no arguments correctly create a local FPS with defaults — this is expected FPS behavior, not an error. Also add 'does not exist' to the runner grep pattern for error detection.
oguyon
added a commit
that referenced
this pull request
May 22, 2026
* feat: add CLI robustness test suite Add comprehensive automated test suite for milk-cli: - tests/cli/cli_robustness_tests.milk: 109 annotated test cases covering basic parsing, help/discovery, builtins, missing/wrong arguments, variables, arithmetic, flow control, user functions, command chaining, shell features (pipes, redirects, here-strings, command substitution), error conditions, set flags, readonly/declare. - tests/cli/run_cli_robustness_tests.sh: Bash test runner that feeds tests one-by-one to milk-cli, captures output, checks for crashes/hangs/missing error messages, and prints a summary report. - .agents/workflows/cli-robustness-test.md: Workflow for running the test suite. Initial run: 105 pass, 4 MISSING_ERROR (no error message printed for: cmd? with unknown command, cd to nonexistent directory, mem.mk2Dim with no args, mem.rm with no args). * fix: CLI error reporting for cmd? and cd with absolute paths Fix two CLI error reporting bugs found by the robustness test suite: 1. help_command() (cmd?) now returns RETURN_FAILURE when the queried command is not found. Previously it always returned RETURN_SUCCESS even when printing "does not exist", misleading the caller. 2. Extend tokenizer raw-string bypass to treat arguments starting with '/' as raw strings (same as '-' prefixed arguments). This fixes cd /absolute/path where the '/' was misinterpreted as division by cli_parse(), causing a silent parse error that skipped command execution entirely. Update test expectations: mem.mk2Dim and mem.rm with no arguments correctly create a local FPS with defaults — this is expected FPS behavior, not an error. Also add 'does not exist' to the runner grep pattern for error detection.
DasVinch
pushed a commit
that referenced
this pull request
May 29, 2026
* feat: add CLI robustness test suite Add comprehensive automated test suite for milk-cli: - tests/cli/cli_robustness_tests.milk: 109 annotated test cases covering basic parsing, help/discovery, builtins, missing/wrong arguments, variables, arithmetic, flow control, user functions, command chaining, shell features (pipes, redirects, here-strings, command substitution), error conditions, set flags, readonly/declare. - tests/cli/run_cli_robustness_tests.sh: Bash test runner that feeds tests one-by-one to milk-cli, captures output, checks for crashes/hangs/missing error messages, and prints a summary report. - .agents/workflows/cli-robustness-test.md: Workflow for running the test suite. Initial run: 105 pass, 4 MISSING_ERROR (no error message printed for: cmd? with unknown command, cd to nonexistent directory, mem.mk2Dim with no args, mem.rm with no args). * fix: CLI error reporting for cmd? and cd with absolute paths Fix two CLI error reporting bugs found by the robustness test suite: 1. help_command() (cmd?) now returns RETURN_FAILURE when the queried command is not found. Previously it always returned RETURN_SUCCESS even when printing "does not exist", misleading the caller. 2. Extend tokenizer raw-string bypass to treat arguments starting with '/' as raw strings (same as '-' prefixed arguments). This fixes cd /absolute/path where the '/' was misinterpreted as division by cli_parse(), causing a silent parse error that skipped command execution entirely. Update test expectations: mem.mk2Dim and mem.rm with no arguments correctly create a local FPS with defaults — this is expected FPS behavior, not an error. Also add 'does not exist' to the runner grep pattern for error detection.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a comprehensive CLI robustness test suite and fixes two error reporting bugs discovered by it.
CLI robustness test suite (new)
tests/cli/cli_robustness_tests.milk— 109 annotated test cases covering parsing, built-in commands, variables, arithmetic, flow control, user functions, command chaining, shell features, and error conditions.tests/cli/run_cli_robustness_tests.sh— Bash test runner that feeds test blocks tomilk-clivia FIFO mode, captures output, and categorizes results as PASS/FAIL/MISSING_ERROR/CRASH/HANG..agents/workflows/cli-robustness-test.md— Workflow for running the test suite.Bug fixes
cmd?with unknown command —help_command()inCLIcore_help.cnow returnsRETURN_FAILUREwhen the queried command is not found. Previously it always returnedRETURN_SUCCESSeven after printing "does not exist".cd /absolute/pathsilent failure — Extended the tokenizer raw-string bypass inCLIcore_UI.cto treat/-prefixed arguments as raw strings (same as-prefixed arguments). The/in absolute paths was being interpreted as division bycli_parse(), causing a silent parse error that skipped command execution entirely.Test expectation corrections
mem.mk2Dimandmem.rmwith no arguments correctly create a local FPS with default values — updated test expectations from#EXPECT:ERRto#EXPECT:OK.does not existto the runner's error-detection grep pattern.Prompt Summary
User requested a comprehensive CLI test suite to identify bad behavior (crashes, missing error messages). The test suite was created and run, revealing 4 MISSING_ERROR cases. Two were real bugs (fixed here), two were false positives (test expectations corrected).
AI Authorship