Native Code Generation, Compilation & Native function calling

[Nickyesulol]

Luau is a scripting language which is rich in utility and an improvement to Lua.
From my understanding, lune is a runtime of luau, which does not focus on embeddability, but for utility with scripts on a developers machine or even being shipped.

The current problem is, that you would also have to ship the runtime and manually set everything up.
Additionally, it would be a hassle of files and dirs (depending on how you ship it) instead of one nice executable.
There are already projects for Lua, which solve this problem by packing a runtime with the scripts (https://github.com/luvit/luvi, https://github.com/samyeyo/rtc). I believe, that here luau has an advantage due to the Native Code Generation included in the project (although i'm not sure if it would actually work how I imagine it, due to me not knowing how all the magic actually works). Another drawback is, that luau was designed as a sandboxed scripting language, lune not being sandboxed, I believe it would be a really powerful move to include the ability to call native functions from libraries (best if both static and dynamic, but dynamic preferred). This would enable people to create dedicated libraries and environments for platform and native dependent use cases. With how luau is optimized for performance and the easy syntax and the advantages of type checking, it would enable awesome projects. One use case could be machine learning, just as python does it. Luau would have the advantage of performance. The only hurdle here would be the ecosystem, but it could probably also somehow be fixed by making a transpiler or cooperability layer similar to mojolang.

[CompeyDev]

Additionally, it would be a hassle of files and dirs (depending on how you ship it) instead of one nice executable.

There is the lune build command, which allows you to compile to a standalone executable that you can ship to a user. It also allows for cross-compilation.

I believe it would be a really powerful move to include the ability to call native functions from libraries (best if both static and dynamic, but dynamic preferred)

It isn't possible to have the ability to static binaries for any runtime. I'm not sure what you mean by that at all. Lune is not a compiler, it can't link your Luau code against native code; it can only dynamically load libraries into memory and expose them to you. FFI is already being worked on, see #243.

Luau would have the advantage of performance. The only hurdle here would be the ecosystem, but it could probably also somehow be fixed by making a transpiler or cooperability layer similar to mojolang.

This is an interesting idea, and I've looked into Luau being used for machine learning, and it is pretty promising, but this is out of Lune's scope.

[Nickyesulol]

Sorry, I oversaw the build command, thank you for bringing my attention to it.

It isn't possible to have the ability to static binaries for any runtime. I'm not sure what you mean by that at all. Lune is not a compiler, it can't link your Luau code against native code; it can only dynamically load libraries into memory and expose them to you.

I'm not sure if the build command bundles the runtime or actually manages to turn it into native machine code, if the latter, wouldn't it be possible to statically link it? I'm not sure if that is what codegen does, but from what I've read, is that codegen (the one from the original luau project) turns luau into native assembly and then executes it. If it's that, wouldn't it be possible to link them? Even then, that's not a priority and it's good to hear that FFI is already being worked on. I love what you guys are doing :)

[CompeyDev]

Native codegen is close to what you've described, although it isn't something which is done portably, but rather right before execution on Luau's end. Even so, it would not be within Lune's scope to implement that. Luau technically does not compile to any machine code, JIT may however sometimes do so.

lune build only bundles the runtime, that's correct. This is the typical approach almost all runtimes for most languages follow. What you're looking for is not a runtime, but a separate project which behaves as a different compiler frontend for Luau code.

[Nickyesulol]

So if I understand correctly, it turns it into the platform's assembly and then during runtime turns that into machine code? Thank you for explaining everything you had until now, I'm just curious a bit about the technical specifications about Codegen, because it is a really interesting concept. Do you mind explaining it a bit in detail, I have not find really any articles about it.