SYS TO expression argument passing

[dansanderson]

Is your feature request related to a problem? Please describe.
The SYS command calls an ML subroutine, with minimal ability to pass arguments from BASIC to the routine. Today, BASIC 65's SYS supports positional arguments to set the CPU registers, four byte values plus status flags: SYS <addr>,<a>,<x>,<y>,<z>,<s> These values can be BASIC expressions, but must evaluate to byte values. We also have a SYS TO variant with labeled parameters and additional features.

A fun thing to do on a C64 is to exploit parser internals to access raw characters after the end of the SYS command, to effectively pass raw program text to a subroutine that's expecting it. This trick involves manipulating the txtptr internal variable, and either diving deeper into the parser to evaluate expressions or more likely just pull tokens out of the program until the end of the command.

This trick is less useful in BASIC 65 because SYS accepts more arguments than BASIC 2, and we're (well, I'm) wary of exposing parser internals while we're still fixing bugs and adding features. I also think there's opportunity for making this a more useful, formal built-in feature, e.g. to support passing evaluated expressions to the subroutine instead of raw tokens.

Describe the solution you'd like
Give SYS TO special syntax for two argument groups, one that supports the current address followed by optional labeled arguments, and an optional second group consisting of a comma-delimited list of arbitrary expressions that evaluate to any of the supported value types:

SYS TO <sys-arg-group> [<value-arg-group>]

sys-arg-group: <addr-expr>[,<label>(<expr>)]*
label: A|X|Y|Z|S|I|O

value-arg-group: <expr>[,<expr>]*

Populate a data structure in unused variable memory at fretop, including in the null case, containing zero or more evaluated expressions, in argument order. Include support for string expressions—which means carefully allocating string memory and adjusting fretop and the list location accordingly. Document fretop. The subroutine uses fretop to determine if and how many value args were provided (first byte is count), and access their values in fixed-length records that indicate their type and value (32-bit real, integer, or string value pointer and length). On subroutine return, strings are garbage collected, and the data structure ignored (and potentially overwritten by BASIC later).

Example uses:

TCP=$40000: CONNECT=3
SYS TO TCP-CONNECT HOST$, PORT

Extreme weird case:

SYS TO $3100,A($FF),O($420) 237,"MY STUFF",A($FE)

Describe alternatives you've considered
The space between the sys-arg-group and value-arg-group is pretty subtle. We must disallow accidents like:

SYS TO $3100,A($FF),O($420),237,"MY STUFF",A($FE)

One alternative is to forgo the space and wrap value args in a repeatable labeled argument:

SYS TO TCP-CONNECT,Z(HOST$),Z(PORT)

SYS TO $3100,A($FF),O($420),Z(237),Z("MY STUFF"),Z(A($FE))

This topic is part of a larger conversation about making BASIC extensible. This isn't too far removed from a proposal to bind subroutines to custom keywords with built-in argument parsing, though it wins out on simplicity.

Additional context
Naturally, this does not provide for arbitrary syntax after the SYS command the way the C64 technique does. I think its usefulness in passing evaluated expressions outweighs this, and I don't see a good way to provide custom syntax without just exposing the entire parser. Of course, some would like us to just do that, but then we wouldn't have been able to add features like SYS TO in the first place. :P