I find that I reach for metaprogramming for two reasons.

First reason: I have been programmed into a corner. This needs to be tomorrow and a rewrite is out of the question and normally refactorings simply can't get the job done. I need to add methods at runtime or some equally nightmarish hack or the data won't be were is needs to be. I have done some like this twice both in Ruby, about has many times as I ever hope to need longjmp in my native language C++.

The other reason I reach for metaprogramming, is performance. In C++ template metaprograms used to be the only way to have what were easily calculable valuables baked into the final binary. Now there is constexper, which is still sort of black magic and many don't understand it at all. Either way being able to replace runtime code with a thing that looks like a function but actually resolves to a constant can solves certain categories of problems.

> Metaprogramming is often witchcraft.

This is an advice I frequently see, and I disagree with it strongly. For some reason, people are irrationally afraid of metaprogramming, as if it was a different kind of programming. It isn't. It's regular programming, following the very same rules, except that you're automating your own work now.

I don't know; maybe it's because mainstream languages absolutely suck at metaprogramming (if they allow it at all), so people are not used to it?

I mean, it's like someone calling algebra "witchcraft". It's not witchcraft, they just never bothered to learn it.

(Related, I see this approach of lowest-common-denominator programming being popular, and I believe it's very hurtful to everyone. Programming is a profession. If you don't understand some piece of code because it uses weird language features, suck it up and learn stuff, like a real professional.)

The problem isn't with writing or reading metaprogramming. It's in debugging issues in the program that the metaprogram writes. You have to then figure out what both the meta-code and the code does and how they interact to have a good understanding of the problem.

There's also some very powerful advantages to avoiding metaprogramming. Like, "grepping for the name of a token shows all instances where said token is relevant". This is an incredibly powerful tool, and you really miss it when poking around in new codebases with lots of magic.

Metaprogramming frequently reduces readability of the code. Almost all Linux kernel developers and embedded systems programmers are familiar with the linked list composed only of #define 's, and other metaprogramming solutions. It doesn't mean that seeing things like that are readable or easily traced to reproduce what happened.

Maybe you are so smart you can juggle it all in your head, as Torvalds is reputed to do with the Linux kernel. The rest of us don't have the ability to hold the mental model of both the primary layer and the metaprogrammed layer in our heads without a devotion of time that would exceed our schedules.

Thankfully in C-like languages, the compilers I've worked with will let you see the post-metaprogrammed output.

C++ templates and other metaprogramming features improve readability; I'm don't mean to cast shade on metaprogramming in general, just heavily obfuscated implementations.

I think that's the part of the blog post that resonated most with me. There seems to be this perverse belief that shortening code is always a positive, no matter how much black magic it requires.

Traceability seems to always be the first thing burnt at the stake to appease the god's desire to save 3 characters; which I find strange, because it's the most fundamental requirement of good code. If you can't work back through the code to find the source of a bug, your code is an unmaintainable pile of shit. No amount of proper whitespace use will save you.

I am currently in the catfish position and have been for a few years. In my experience the #1 reason that people reach for metaprogramming (maybe 90% of all cases) is that they have just learned about the technique in the language and have not yet learned about the readability (and often performance) disadvantages.

For background, right now I am working with an internal Python framework which creates key classes from mixins at runtime with Python's type() constructor. Figuring out what is going on at any given time is very difficult and interacting with it takes days longer than would be needed normally. The author is an inexperienced Python programmer working on a key library for a very large company and we are unable to throw his work away for organisational reasons (the framework team has a lot of political will). Not sure how to think about it. These people's poor quality work keeps me in a job but honestly it makes me feel bad for all the time and money that the client is wasting.

There's a fine line to walk between too much metaprogramming and too much boilerplate.

I often find that my wish to avoid metaprogramming results in to much cloned code, when a simple annotation/reflection or conventional setup would make life a lot easier.

Like everything in programming, using or avoiding meta-programming is a balancing act.

I like the C# approach of having a ton of syntactic sugar. I mean, if you look at what a basic `yield return` loop actually compiles to... That's some crazy metaprogramming going on. But by encoding it at the language level, they help assure that the metaprogramming is accurate and universal.

* Accurate: the final construct does what the meta-construct says it should.

* Universal: everyone speaks with the same meta-constructs.

The antithesis of universal is the Lisp Curse [0], and I think that's a natural conclusion of free metaprogramming, as opposed to the constrained metaprogramming that something like C# offers.

[0] http://winestockwebdesign.com/Essays/Lisp_Curse.html

I agree very much - but often it seems to me that younger or less experienced programmers reach for metaprogramming by default, and end up with an end-product that can be very hard to trace.

Sometimes it is a godsend.