Well that's kinda disappointing. I expected something more in 2025, like directly generating asm like a lot of languages are starting to do.
And your article makes it ambiguous whether it's from the Lua authors or grad students. I assume it started out just the students and then the Lua authors joined in?
One of Lua's goals has been extreme portability, and the main implementation works on anything that has a C compiler, going to the extreme of compiling cleanly on C89, C99, and even compiling as C++ (no extern "C"). Remember that Lua is popular in the embedded space too, so this is a big feature.
Pallene isn't designed to be a new native language on its own. Pallene is designed to be a companion language for Lua, specializing in a subset of performance.
But as importantly, Pallene isn't just compiling to C. Pallene is generating C code that directly manipulates the underlying Lua internals, which are in C.
The research thesis is that many bottlenecks are due to boxing and unboxing going through an FFI. Memory safety also incurs overhead. Python is an extreme example of how excruciatingly slow this can be, but even Lua incurs costs for this. A core tenant of the Pallene compiler is that it can generate C code that gets to cheat like crazy. Pallene gets to directly access Lua internals and things like arrays directly manipulate underlying C arrays deep inside, which sidesteps boxing/unboxing. The compiler can do the analysis to make sure it doesn't cheat in a way that is unsafe. Finally, the C optimizer now also has a chance to perform optimizations. And now operations such as crunching math on arrays of numbers may get much faster because now you get generated code that is more CPU friendly and may benefit more from prefetching and cache locality.
Pallene is built from the the extreme compatibility goals as Lua since it is designed to work with it. It it only depends on any C compiler and Lua itself. If you get Lua compiled, then you can get Pallene working. That means any existing project that uses Lua (5.4) could start adding Pallene modules to their project for new features or to try to improve performance in key areas. Since Pallene just outputs a Lua modules, it looks like any other Lua module implemented in C, so it won't create new portability constraints that you didn't have before. This is different than say LuaJIT, where not all platforms may allow JIT or you may be targeting a new CPU architecture that LuaJIT does not support.
Both Teal and Pallene were started by grad students of Roberto's. Since Roberto has started giving talks himself on Pallene, I'm assuming they are joining in.
I'm Pallene's lead maintainer. Currently the code is maintained by me, my students and other open sou rce collaborators. We collaborate with Roberto over some Pallene-related research, specially about the type system semantics, but he isn't an active committer.
If it goes further and generates native C control flow statements when possible ("if", "for", native functions, native function calls, etc), I think it could be an omni-level language, generating basically Lua statements when dynamic and C when not, and mixing them all within the same program, entirely controlled by how much type information you give it (and how you use tables and metatables).
Alas, as of last month we changed Pallene's compiler and it now generates C gotos for control flow. ( ^ _ ^ メ ) It helped with certain optimizations...
We switched from a hierarchical internal representation to a flat one, with gotos. Made it easier to write textbook otimization algorithms. In theory we could try to reconstruct the if and while statements when it's time to emit C, but gotos are more straightforward now.
This sounds like it could compete with V8 at some point in the next few years. Honestly it could be a game changer. Looking forward to watching its progress.
Also, I think it's important to say that ECS is not the only composition option, you can just use "EC" (or also CES): while on ECS the behavior runs on systems, on "EC"/"CES" the behavior runs on components, which is the case on Unity (by default) and O3DE.
I believe the mentioned Scott Bilas famous talk about composition follows this "EC" model instead of ECS, except that he calls it Game Object instead of Entity.
Forgive me if I'm wrong about this, still trying to get my head around ECS concepts, but isn't the whole point that data and behaviours are uncoupled? Whereas Unity's Game Objects both hold data and are composed of behaviours (rather than being data-only objects that are operated on from with-out by systems)?
Yes, I made this comment because sometimes I see some devs calling Unity an ECS (although it has an ECS option, but that's not the default mode most unity games uses).
Just to say that ECS is not the only composition option: Entity with composed behaviors is also an option (and the most traditional one I believe), and it does not have the infamous complexity of "pure" ECS in my opinion.
Just as no battle plan survives contact with the enemy, no design pattern survives contact with a specific implementation. You will often have to compromise purity for the sake of efficiency, the quirks of a specific language, performance, or in the case of a framework a consistent and general purpose API.
His "Sensible ECS" is basically what I did on my (simple 2D) game, just simple structs with components, and functions that operates on these components.
That works well, although it does not features parallelism, but still managed to run on a old laptop that can at most run Windows 7.
That's because they share a common origin from Typed Lua and Titan languages:
https://teal-language.org/book/other_projects.html