Our experience with Delta Engine has been that it's way more resource hungry than the JVM code it replaced. It doesn't handle resource exhaustion well at all; lots of crashing and deadlock when nearing full resource utilization.
I would love to have something more resource efficient than Spark on JVM, but Delta Engine isn't there yet.
I would love to have something more resource efficient than Spark on JVM, but Delta Engine isn't there yet.