How to build/test changes in the compiler/runtimes?

[Mingun]

It seems that the algorithm, outlined by me in the kaitai-io/kaitai_struct_doc#31, no longer works because there is no script build-jvm-compiler (which is called from tests/build-compiler) in the project tree. So I can't test my changes in kaitai-io/kaitai_struct_compiler#197.

@GreyCat, @generalmimon, can you assist me in testing my changes?

[generalmimon]

because there is no script build-jvm-compiler (which is called from tests/build-compiler) in the project tree

AFAIK there's never been a script named build-jvm-compiler, do you mean sbt compilerJVM/stage (build-compiler:7)?

It seems that the algorithm, outlined by me in the kaitai-io/kaitai_struct_doc#31, no longer works

What exactly doesn't work on these commands (developers.adoc:212-226)?

To test changes in compiler and runtimes, run following commands (for example, for Java runtime):

# Assumed, that this will be done
# git clone --recursive https://github.com/kaitai-io/kaitai_struct.git
# cd kaitai_struct
cd tests
./build-compiler
# Convert all test `.ksy` files into all target languages (including Java),
# which will give you tests/compiled/java
./build-formats
# will run all tests for Java; results will be in `test_out/java`
./run-java

Also, see readme of https://github.com/kaitai-io/kaitai_struct_tests project.

They all look good to me, I'm running them all the time.

[Mingun]

AFAIK there's never been a script named build-jvm-compiler, do you mean sbt compilerJVM/stage (build-compiler:7)?

Yes, exactly. I looked at travis-fold more carefully -- it turns out that these are the names of folded groups, and not the names of scripts 🤦

What exactly doesn't work on these commands (developers.adoc:212-226)?

There was a lot of things mixed and I didn't know what specifically didn't work. Now I known that, in fact, only #211 interfered.

So my instruction also works for me.

[generalmimon]

@Mingun

Yes, exactly. I looked at travis-fold more carefully -- it turns out that these are the names of folded groups, and not the names of scripts

I actually agree with you that the travis-fold usage here can be very confusing for newcomers. I also remember wondering about what it does when I wasn't familiar with the project, and I needed to look into the travis-fold script to see what it does.

I looked for the travis-fold usage in the whole project, and it looks like a forgotten experiment that never actually worked. The point is to make some section of the build log collapsible. This makes sense, because the kaitai_struct Travis CI build log tends to be very long (~ 10000 lines) and making it possible to collapse improves the user experience while reviewing the log.

One thing is that the build-compiler script is actually no longer run by any CI build for a while - it's replaced by a manual sbt invocation, which does the compiler packaging (kaitai_struct / .travis.yml:28):

  - sbt compile compilerJVM/stage fastOptJS buildNpmJsFile buildNpmPackage compilerJVM/debian:packageBin compilerJVM/universal:packageBin

If it's not run by Travis and it's only run locally on developers' machines, there's no point in leaving it there :)

Other script translator-tests:7 is still used, but the Travis folding doesn't work anyway, as you can see in https://travis-ci.com/github/kaitai-io/kaitai_struct/jobs/429467053#L1146. This is probably because despite that there is the opening travis_fold:start:build-jvm-compiler statement, the travis_fold:end:build-jvm-compiler never occurs (see https://api.travis-ci.com/v3/job/429467053/log.txt). This issue may be caused by sbt test failing, so that it returns an error exit code and prevents the ending tag (travis-fold:15) from printing.