Scala 3 Install

[sean-russell]

Hi,

I am planning to use coderunner for some assessments in a new course in Scala. I asked the tech guy who manages our schools moodle server and the VM that is hosting the Jobe server. The problem we ran into is that when compiling a test program I get the following (root cause) error:
"Caused by: java.nio.file.AccessDeniedException: /home/jobe00/.cache"

Clearly we have some permissions set that are preventing the job run from creating these cache files. When permissions are given for that folder, there is just another one, and another one.

Have you any knowledge of others that have installed Scala 3 on a Jobe server and got it running? If yes, do you know how they did it? If no, do you have any suggestions for an approach to resolving this problem?

Regards
Sean

[trampgeek]

The jobe user accounts are deliberately created without a home directory to prevent various security exploits. So /home/jobe00 shouldn't actually exist. Did you create it manually? If so, that's probably not a good idea unless all other approaches fail. Certainly you would need to add code to empty it at the end of a run, or successive users could share results, data, etc. That code would need to go into the Jobe source code (LanguageTask.php or runguard.c).

I just pushed an update to Jobe (v2.2.1) that adds to the environment a variable HOME that references a temporary directory that is created on each run, originally for use by Python's Matplotlib module. If Scala is using the HOME variable to find the user's home directory (which I think it should be), that should solve your problems.

I haven't yet rebuilt jobeinabox, so if you're using that you'll need to rebuild it from the dockerfile.

Please post back to let us know what happened.

[sean-russell]

I was having trouble working through this with the technical support person in my school, so I installed a Jobe server instance on a server I could use to test it and even at the latest version the same problem still persists.

I'm trying another angle at the minute by contacting the scala-cli maintainers and seeing if there is any config or flag that I can set that would cause all of the operations to be carried out only in the current working directory. I will update you if anything productive comes out of that discussion.

Regards
Sean

[trampgeek]

A couple of things.

Firstly, have you read the thread on Scala in Jobe here: https://coderunner.org.nz/mod/forum/discuss.php?d=722?

Secondly, I had a quick play myself right now. I was able to get a rudimentary and totally-untested Scala question type going as follows:

1. I gave up on using Coursier to install scala (it does heaps of caching) and switched to using apt to install it.
2. I had to add the '-nc' option to the run command, i.e. something like "scala -nc prog.scala"
3. I had to set the question type memory limit to 0 and numprocs to 200 (or some number greater than 20, anyway) to get the JVM (a terrible resource hog) to run.
4. I had to increase the time limit e.g. to 10 seconds.

I attach my question prototype. But please realise this is just a quick prototype. There are no doubt lots of other java options you should be considering.

PROTOTYPE_scala_program-20251006-1133.xml

[sean-russell]

Firstly, have you read the thread on Scala in Jobe here:

I actually hadn't. The problem with that solution is that it uses Scala version 2, while I am using Scala version 3 which does not appear to be available in apt. I then tried to install it using SDKMAN, with the problem being that this only installs a local version. I then tried installing this for the jobe user, but this still didn't do it for the numbered jobe users.

I am currently looking at just downloading and installing the binaries, which in principle is relatively easy but I need to modify the PATH variable for the jobe users to add the location of the scala binaries for that to work. This has gotten me much further, and I am avoiding the path issues byu having put the binary somewhere accessible to all users and by using the absolute path when I execute the coed. But I am now running into memory issues that I can't seem to solve regardless of the amount of memory I throw into the compilation command...

I am executing the compiler (running just scala gives the same result as it initiates the compiler) with the following command:

output = subprocess.check_output(['/usr/local/share/scala/bin/scalac',"-J-Xss128m","-J-Xms1024m", "-J-Xmx4096m",'-J-XX:MaxMetaspaceSize=4096m', 'main.scala'], stderr=subprocess.STDOUT, universal_newlines=True)

This is the result I am currently seeing:

[3.018s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 131072k, guardsize: 0k, detached.
[3.019s][warning][os,thread] Failed to start the native thread for java.lang.Thread "Thread-0"

  Exception while compiling main.scala

  An unhandled exception was thrown in the compiler.
  Please file a crash report here:
  https://github.com/scala/scala3/issues/new/choose
  For non-enriched exceptions, compile with -Xno-enrich-error-messages.


     while compiling: <no file>
        during phase: parser
                mode: Mode()
     library version: version 2.13.16
    compiler version: version 3.3.6
            settings: -J<flag> List(-Xss128m, -Xms1024m, -Xmx4096m, -XX:MaxMetaspaceSize=4096m) -classpath /usr/local/share/scala/lib/scala-library-2.13.16.jar:/usr/local/share/scala/lib/scala3-library_3-3.3.6.jar:/usr/local/share/scala/lib/scala-asm-9.8.0-scala-1.jar:/usr/local/share/scala/lib/compiler-interface-1.10.4.jar:/usr/local/share/scala/lib/scala3-interfaces-3.3.6.jar:/usr/local/share/scala/lib/scala3-compiler_3-3.3.6.jar:/usr/local/share/scala/lib/tasty-core_3-3.3.6.jar:/usr/local/share/scala/lib/scala3-staging_3-3.3.6.jar:/usr/local/share/scala/lib/scala3-tasty-inspector_3-3.3.6.jar:/usr/local/share/scala/lib/jline-reader-3.27.1.jar:/usr/local/share/scala/lib/jline-terminal-3.27.1.jar:/usr/local/share/scala/lib/jline-terminal-jni-3.27.1.jar:/usr/local/share/scala/lib/jline-native-3.27.1.jar -d /tmp/scala3-scripting16385101161451712121

Exception in thread "main" java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
	at java.base/java.lang.Thread.start0(Native Method)
	at java.base/java.lang.Thread.start(Thread.java:1526)
	at dotty.tools.dotc.transform.Pickler.runOn(Pickler.scala:150)
	at dotty.tools.dotc.Run.runPhases$1$$anonfun$1(Run.scala:315)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at scala.collection.ArrayOps$.foreach$extension(ArrayOps.scala:1324)
	at dotty.tools.dotc.Run.runPhases$1(Run.scala:308)
	at dotty.tools.dotc.Run.compileUnits$$anonfun$1(Run.scala:348)
	at dotty.tools.dotc.Run.compileUnits$$anonfun$adapted$1(Run.scala:357)
	at dotty.tools.dotc.util.Stats$.maybeMonitored(Stats.scala:69)
	at dotty.tools.dotc.Run.compileUnits(Run.scala:357)
	at dotty.tools.dotc.Run.compileSources(Run.scala:261)
	at dotty.tools.dotc.Run.compile(Run.scala:246)
	at dotty.tools.dotc.Driver.doCompile(Driver.scala:37)
	at dotty.tools.scripting.ScriptingDriver.compileAndRun(ScriptingDriver.scala:22)
	at dotty.tools.scripting.Main$.process(Main.scala:39)
	at dotty.tools.MainGenericRunner$.run$1(MainGenericRunner.scala:250)
	at dotty.tools.MainGenericRunner$.process(MainGenericRunner.scala:270)
	at dotty.tools.MainGenericRunner$.main(MainGenericRunner.scala:281)
	at dotty.tools.MainGenericRunner.main(MainGenericRunner.scala)

This was with TimeLimit (secs) set to 15 and MemLimit (MB) set to 0 to disable the limit.

I feel like the solution is close, and would welcome another nudge in the right direction.

Regards
Sean

P.S. As I was just about to post this I read the section on this page (https://www.domjudge.org/docs/old_manual/judging.html) about the JVM and memory limits. So it occurred to me that if the GC is the potential culprit, then maybe I should change that...

output = subprocess.check_output(['/usr/local/share/scala/bin/scala',"-J-Xss128m","-J-Xms1024m", "-J-Xmx4096m",'-J-XX:MaxMetaspaceSize=4096m','-J-XX:+UseSerialGC', 'main.scala'], stderr=subprocess.STDOUT, universal_newlines=True)

This does both the compilation and execution, as when I tried to do them separately, the time taken was even longer!
But as of this moment I have a basic hello world in Scala 3 that can be successfully tested!

Thanks for your help and I'll update if I come up with any changes that can improve the overall process

Regards
Sean (Again)