This topic was brought up many times in the past. Most of the time it was suggested to use speculative approaches to detect blowups, but I haven't actually seen any attempts to predict them before supercompilation. For example, while I was writing Mazeppa, I've seen many times that blowups occur because some function call gives insufficient information that is subsequently pattern-matched by another function, and since it cannot be properly pattern-matched at compile-time, a lot of code duplication takes place.

I'm leaning towards a kind of "abstract interpretation pass" before supercompilation to approximate "good" and "bad" interactions between functions. After all, given that offline analyses exist even for harder problems such as detecting non-termination, why cannot we understand how supercompilation gives us desirable results?

1. The approach involves magic numbers to implement heuristics. They worked for the samples provided by the authors, but will they work for large programs?

2. I think that there's a smarter approach to detect blowups. Specifically, there can be "good" and "bad" interactions in the form `f(g(...), ...)`, where `f` and `g` are functions; an interaction is good if `g` produces information that is known at compile-time (to be subsequently deconstructed by `f`), and bad if it doesn't. If the interaction is bad, `f` and `g` should be transformed separately.

In general, I see manual annotations as a low-level mechanism that can be used for predictable supercompilation. We haven't yet implemented heuristics that would guarantee predictability, but this is on our priority list.

> Are there any other notable differences between this approach and call-by-value constructors?

Yes, lazy constructors must be implemented differently than eager ones. The standard approach is to enclose constructor arguments under an environment of values, just as we implement closure functions. I don't think that supercompilation can remove all laziness from the constructors.

[1] Jonsson, Peter & Nordlander, Johan. (2011). Taming code explosion in supercompilation. PERM'11 - Proceedings of the 20th ACM SIGPLAN Workshop on Partial Evaluation and Program Manipulation. 33-42. 10.1145/1929501.1929507.

[1] https://www.youtube.com/watch?v=MYSFTtYc4P4

https://en.wikipedia.org/wiki/Cultural_legacy_of_Mazeppa

* Many compilers and transpilers available. Don't like LunarML? Maybe try Poly/ML. Or MLton. Or MLKit. Or SML/NJ. Etc. All of the above are pretty much complete and production quality, and there are quite a few more.

* The language has a formal specification which means there's a document formally specifying what all the constructs in the language should do. In contrast, if you wanted to build your own OCaml compiler/transpiler/interpreter, you'd basically have to follow whatever the main OCaml implementation does, which changes from one version to the next, so you'd always be playing catch-up.

* The formal specification hasn't changed since 1997. This means that once written, your SML code should work the same forever. You no longer have to worry about your programs (or their dependencies) breaking when you upgrade to a new version of OCaml.

* Compared to OCaml, your single-threaded programs will usually run faster if you compile them with MLton, mostly due to whole-program optimization / monomorphization and unboxed native integers, reals and arrays.

* Some SML implementations have interesting extensions, which you can use (at the cost of being tied to that implementation). For example, MLKit supports memory regions (i.e. more efficient memory management in some cases). SML# (which is not related to .NET) supports seamless integration with SQL and other interesting features. Some implementations support multi-threading natively and/or different forms of parallelism. Most of them also support some form of FFI to interface with C or other languages.

* SML programs are easier to port to CakeML than any other language, since CakeML is mostly an SML subset. CakeML not only allows you to formally verify that your program is correct (or that it has certain desirable properties), but it also has a compiler that is formally proven to be correct, so your compiled CakeML programs are (mostly) guaranteed to be free of miscompilation bugs as well.

And a minor point: operator arguments (as well as curried arguments) are evaluated left-to-right, not right-to-left like in OCaml.

I only sometimes use it as a "I would recommend this repo" -- how can one do that anyways, given that the repo could morph into something one would no longer recommend?

[1] L. Augustsson. Compiling Pattern Matching. In Functional Programming Languages and Computer Architecture, pages 368– 381, 1985.

[2] P. Wadler. Efficient Compilation of Pattern Matching. In S.L. Peyton Jones, editor, The Implementation of Functional Programming Languages, pages 78–103. Prentice Hall, 1987.

What a lot of people miss about that paper is that he wasn't just talking about goto statements. He was also making a more general observation about how more powerful and general programming language features are not necessarily desirable, because they tend to adversely impact developer productivity.

The reason I, as a user, prefer structured control flow statements over goto is not that I believe they are powerful. It's precisely because they are less powerful. The resulting constraints on how the program can be structured make it easier for me to read and reason about existing code. That makes maintaining code easier. It also makes optimization and static analysis easier. And it makes writing tests easier, too.

I have similar feelings about ADTs. The reason I prefer them to other ways of doing composite data types is not that I think they're more powerful. It's that they create constraints that tend to reduce the semantic complexity of the domain models people create in programming languages that use them. And that makes my job easier.

The corollary to that, though, is that I'm not actually all that hype about adding ADTs to existing languages. For reasons that are similar to how the mere availability of structured, reentrant function calls is small consolation in a codebase that's already riddled with goto statements. The real win doesn't come from using ADTs, it comes from not having to worry about all those other confusing overpowered things that aren't ADTs.

But they're hard to read for anyone who isn't an expert on not just the language but the type in question (c.f. Rust's Option() idioms all looks like line noise to newbies, etc...). And that's a bad trade.

In essence, this stuff is just Perl all over again. It's a language feature that prioritizes concision over comprehension. And I say that as someone who really likes coding in perl. But "people" don't like perl, and the community moved on, and the reasons are... the same reason that uptake in ADTs is lagging where the experts want it to be.

Pattern matching on the next level down is a power tool to be used with care.

Having used some pattern matching languages for quite some time, I find anything much deeper than that is a code smell at best and pathological at worst. Pattern matching creates coupling proportional to the depth/complexity/precision of the pattern match. The top-level coupling is often unavoidable; if you're pattern matching at all, you certainly care which "branch" you are on and there is likely no refactoring that away. But the danger rises rapidly the deeper in you go. It's just so easy to pattern match on a third-level part of the complex object when you really ought to be wrapping that behind a function somewhere, possibly itself emitting a sum type value.

... but if all you really need is that "top level" match, a lot of pattern matching syntax and features are not really necessary (if not positively dangerous).

Which is exactly how Perl apologia arguments went.

All Turing Complete programming languages are Turing equivalent to one another. Programs written in one language can be mechanically transformed into those written in another. This is irrelevant to the discussion of programming languages. The whole point of creating different programming languages is to explore different ways to express the same program!

There's also a great talk on the matter [1], if somebody is interested in formalities.

[1] https://www.youtube.com/watch?v=43XaZEn2aLc

This is one of the social factors of programming language design and it's one of the main reasons successful programming languages work so hard to establish a coherent philosophy and a set of best practices or idioms within the language. For similar reasons, I believe this is why "anything goes" languages such as LISP have struggled to gain widespread adoption: with no philosophy every programmer becomes an island unto themselves.

There are already two misconceptions.

First: "Lisp has no programming philosophies" and styles.

Not every program starts by zero. Since Lisp exists since the end 1950s, it has seen quite a lot in programming styles over the years and it may contain traces of several. Generally it may support more than one programming paradigm. For example during the Common Lisp standardization there was a wish to have a standardized object system. So instead of the multiple possible approaches (actors, message passing, prototype-based, ...), Common Lisp has just one: CLOS, the Common Lisp Object System. So, much of the object-oriented code written in CL is implemented in one particular object system: CLOS. Object Lisp, Flavors, LOOPs, Common Objects, and a bunch of other once had thus been replaced by one standard.

CLOS also defines a bunch of user-level macros: DEFCLASS, DEFMETHOD, DEFGENERIC, ... Everyone using CL & CLOS will use those macros.

Second: "every programmer becomes an island unto themselves". If we look at the way CLOS was designed: there was a core group of six people from three companies. Around that there was a mailing-list based communication with a large group of interested people. Early on a prototype was implemented as a portable implementation of CLOS. This was widely distributed among interested parties: implementors, companies, research groups, ... Then reports about the language extension and its layers were published, books were published, application & library code was published.

One of famous books coming out of this effort: "The Art of the Meta-Object Protocol". It contained also a toy implementation of CLOS in Common Lisp. Book and the implementation of CLOS (both the larger prototype and the toy implementation) showed in excellent quality how to write object-oriented Lisp code.

https://mitpress.mit.edu/9780262610742/the-art-of-the-metaob...

So, there are communities, which share code and coding styles. Not every programmer is alone and starts from zero.

You misquoted me. I said no philosophy, singular. In the programming language context, a philosophy is a convention or a standard. Just as many standards implies that there is no standard, many philosophies implies no philosophy.

Everything else you said is evidence for my premise. Hire 3 different programmers, one from each of the communities, and you might as well have 3 different programming languages. That’s not a standard. That’s not a philosophy. That’s anything goes!

Most of the Common Lisp code that is accessible via public repositories conforms to conventions and is understandable.

Lisp programmers are highly motivated toward encouraging collaboration, since there aren't that many people in Lisp where you can afford to be turning people away toward projects that are easier to get into.

Also, you can easily hire 3 developers and get 3 different languages in, oh, Java or JavaScript. One person is doing Kotlin, another one Scala, ...

Three C++ programmers in the same room could also not understand each other. The greybeard speaking only C++98 with a bit of 2003 doesn't grok the words coming out of the C++20 girl's mouth and so it goes.

That makes no sense.

> Hire 3 different programmers, one from each of the communities, and you might as well have 3 different programming languages.

Maybe not. They build on the same foundation, a language with a large standard, which is largely unchanged since three decades. A language which can be incrementally extended, without invalidating the rest. A language where extensions can be embedded, without invalidating the rest of the language or its tools. Many projects use SBCL (now itself 25 years old and only incrementally grown) and a bunch of core libraries for it.

> That’s not a standard. That’s not a philosophy. That’s anything goes!

Most languages support widely different software development practices. Take JavaScript: it includes imperative, object-oriented and functional elements (similar to Lisp). It has huge amounts of competing frameworks (many more than any Lisp), where many of them have been superseded and many of them are built on a multitude of other libraries. The developer can pick and choose. Each projects will be different from other projects, depending on which libraries and programming frameworks it uses - and which of those the developer re-invents.

Any half-way powerful language (C++ -> templates, Ruby -> meta objects, Java -> objects & byte code & reflection & class loader, Smalltalk -> meta objects, Rust -> macros, C++ -> language interpreters, Java -> external configuration languages, C -> macro processor, ...) has ways to adapt the language to a certain style & domain.

Any large Java framework defines new standards, new configuration mechanisms, new ways to use new Java features (lambdas, streams, ...). See the large list of features added to the Java language over time: https://en.wikipedia.org/wiki/Java_version_history For many of them there were competing proposals. Each Java code base will use some subset/superset of these features, depending on what is needed and what the personal preferences are. And then the Java architect in the project will not be satisfied and will invent yet another configuration system, this time not using XML or JSON for the syntax, but develop a new embedded scripting language for the JVM and integrate that with his software. I have seen Java architects which eventually didn't understand their own configuration system any more.

If you think a language like Common Lisp is special here, then in reality it is not. It's just slightly different in that it has extensibility as one of its philosophies and provides defined interfaces for that. There is one macro mechanism, which is powerful enough, that for decades it has not been replaced. Every syntactic extension mechanism will use this macro system, which is documented, stable and widely used.

Why do you think that Lisp is an "anything goes" language? What's your baseline? I think that C is no less an "anything goes" language, but with a much less pleasant UI.

> with no philosophy every programmer becomes an island unto themselves

Some people actually think that Lispers tend to be too philosophical

For all its faults, C is quite easy to read and understand, even by beginner C programmers. Yes, C also has macros but their clumsiness helps to discourage their use.

That's why the most important difference between C++ and Rust isn't some technicality even though the technical differences are huge, it's cultural. Rust has a Safety Culture and everything else is subservient to that difference.

Sugar matters, Rust's familiar looking loops are just sugar, it only "really" has a single way to do loops, the loop construct, an infinite loop you can break out of. But despite that, people deliberately write the other loops - and the linter strongly recommends that they write them, because the programs aren't just for machines to compile, they're for other humans to read, and a while let loop is an intuitive thing to read for example, so is the traditional for-each style iterator loop.

Surely any Turing complete PL can express a sum type? I can't imagine a language that can support products but not sums.

Better known as the Turing Tar Pit.

So, while you are formally right, the need of shortcuts in pattern matching is undeniable to me.

In addition to that, I don't always find the constructor/method approach more readable. It _can_ be readable, if you design your API with care; however, it is common to group related data into records in the FP world as well, which mimicks OOP constructors, in a sense.

If currying isn’t built-in, you can emulate it. In TypeScript you need to do it explicitly, which means anticipating that it will be used, and whenever I try that, I later decide that it’s needlessly obscure and rewrite the code some other way.

Even with built-in currying, you still need to anticipate usage by changing the order of the parameters, based on which ones you expect the callers to have first. It’s less general than writing an inline function with exactly the parameters you need. TypeScript has nice syntax for defining tiny, one-off functions and I think that’s better than having currying.

One side-effect of having currying in a language is that it discourages naming things, and I think that’s often bad for readability. A chain of method calls gives you a lot of names to look up and a place to put documentation.

However, I agree with the wording of the original comment: when you _pass_ some arguments to a constructor and then to a method, this is (at least conceptually) partial application.

Could you say more? I'm not sure I understand what you're saying here.

When a function needs a lot of parameters to do a calculation, these parameters can be arbitrarily grouped into records and supplied in any order. Languages have convenient syntaxes for certain ways of grouping parameters together.

For example, a closure creates a group of parameters in the form of a function.