MVP for externalized Python

[tonvanbart]

This PR adds support for externalized Python and syntax highlighting and code completion for KSML Python files:

• code: snippets can reference external file:
• globalCode: snippets can reference external file:
• customer modules can be put in a configurable directory and will be available for import
• Python interface stub module for code completion and syntax highlighting in Python editors
• tests for the above
• added documentation page on external Python files

[KasparMetsa]

Good job

[KasparMetsa]

Please remove.

[KasparMetsa]

No validation for malformed file: references. Users might accidentally type file: (empty), file: foo.py (space after colon), or file:foo.py extra text.

Without validation, these produce confusing IOException messages instead of clear format errors.

Suggested fix:

var location = code[0].substring("file:".length()).trim();
if (location.isEmpty()) {
    throw new TopologyException("Empty file path after 'file:' prefix");
}

[tonvanbart]

Leaving for now until Monday's discussion.

[jeroenvandisseldorp]

I'm not in favour of this language style. It requires special interpretation by KSML, where I would hope that we can do something similar through the following:

type: predicate
code: |
  import test-filter-external-python-function
expression: values_is_blue(key, value)

If we change anything in the language at all for this feature, then I suggest to add an import tag per function, which would look like this:

type: predicate
import: test-filter-external-python-function
expression: values_is_blue(key, value)

We could even interpret the import tag as "string or list of strings" in KSML, possibly generating more import statements before the code block:

type: predicate
import:
  - time
  - test-filter-external-python-function
expression: values_is_blue(key, value)

This way, we would let KSML generate the "import sourcefile" statement as part of the function's code itself, making it more Python driven again instead of KSML interpreter driven.

[tonvanbart]

IMHO this would make things more confusing for Python programmers, not clearer. They would be used to import module in the top of their Python source but now will have to remember that import: module in KSML generates the import for them.
Also, when editing the source in an external editor, the IDE does not have the import and won't be able to provide type hints (and likely will flag statements referring to the import as being in error)

[jeroenvandisseldorp]

Yes I agree. Adding the import in KSML would be my second-best option, but still preferable to the file: tag in the code. My first preference would be to make it work using just import module in Python code.

[tonvanbart]

You have that option: configure modulesDirectory and import the module in your code and have that be the only way. Then drop file: support completely.
Maybe we should make clear what we want first, instead of trying to resolve it via PR discussions?