Sorry but I cannot agree. I work on the JVM and this pattern is great for "hello world" webapps but breaks down pretty quickly. When large parts of your logic like metrics, security and routing is being done with metaprogramming you're really doing yourself a disservice.
If you don't like routing being done with metaprogramming, you can manually create routes yourself similar to this example[0]. I think the author is trying to show appealing APIs.
I've worked a lot with this pattern in Python (Flask), Java and Scala. I have found it a little inconvenient beyond the simple hello world app when dealing with Flask, but I haven't encountered this problem with Java/Scala. To date, I've worked on probably 4-5 separate applications using this pattern and it's worked quite well. I have also used Spring 4 on the JVM, which ... feels more flexible, but also has a lot of cognitive overhead, and so I'm not crazy about it. I believe Spring Boot follows closely after Jersey, most likely as a result of the pattern's popularity.
In my experience, it is pretty straightforward to build your resources (as Jersey calls them) into various classes without any issue, even for applications with many API endpoints.
As for metrics, security and routing, most of these are canonically managed using annotations in Java. In Spring, application context XML configurations are used, but they are very big and not close to their actual implementations, which makes them get out of control quickly (and inconvenient when they aren't out of control). It feels like losing either way.
I'm a big fan of Coda Hale's work on Dropwizard, which uses metrics, security and routing through annotations, and it works very well for large scale production applications. I use these techniques at scale at Bazaarvoice with great success.
To be fair, I won't go so far as to claim the pattern is imperfect. It's just the best pattern I've used for web services to date, even at scale.
I would love for you to expound more on your own experiences in more detail!
edit: clarification on where I've used the pattern
I've worked on everything you mention and more, and I find the simplicity of say node.js or SparkJava very refreshing when coming from metaprogramming frameworks. Once you go metaprogramming everything is suddenly metaprogramming. Data serialization is metaprogramming. Middleware is metaprogramming. Adding your own plugins to the framework becomes metaprogramming.
Also, because the framework wants to create your classes you cannot simply send in the objects dependencies into the constructor anymore. You find yourself in need of a DI framework instantiating your classes. And suddenly the data dependencies of your app become very vague. Adapters get registered based on the presence of JARs and/or classes in directories which are scanned in runtime.
You have reinvented the work of the compiler and the language, and what do you gain? A big mess where you no longer have programmatic control. I cannot easily start parts of my application for testing, I have to rely on Spring or Jersey having some sort of metaprogramming magic triggered by some magic annotation supplied by a JUnit<->Spring integration module. Suddenly modules like JUnit and Spring aren't combinable without metaprogramming. You see this trend spreading like wildfire: "Is your lib Spring compatible?". We should instead be asking ourself why the heck unrelated libs have to be compatible with one another. They are supposed to be orthogonal, that's the point! To see an example of how ridicilous this can get, have a look here: http://stackoverflow.com/questions/35957287/cucumber-testng-...
I wouldn't go as far as saying that frameworks like Spring or Jersey are unworkable. But I believe you could acheive the same level of service a LOT easier if you wouldn't go metaprogramming crazy. Just consider the amount of time wasted trying to figure out why something isn't wired correctly, why some plugin is loaded, why your component isn't picked up, why some property isn't resolved etc. With normal programming, you fire up the debugger and step through the code. Solved in 5 min. With metaprogramming, it's a struggle on a whole different level.
This is exactly the reason why I refuse to use Lombok. It literally mutates your bytecode! That's a disaster waiting to happen, and a potential debugging nightmare.
I think that there is a fair balance where metaprogramming has great utility. For example, I'm a huge proponent of Guice. And yes, sometimes it makes debugging issues a little more complicated. OTOH, without it, there's a lot of boilerplate. Admittedly, DI in Java is a workaround for an inherent Java problem (boilerplate). Maybe it wouldn't be so useful in other languages/ecosystems. I have written services both with and without DI, and prefer it, and haven't had significant issues debugging or managing it. But it must remain scoped.
As for library compatibility issues, that is more of a byproduct of dynamically linked libraries than metaprogramming. If you're writing in a JVM language, you're kind of stuck with this (OK, you could shade your library and all its dependencies to get away from it, if you want to deal with the bloat / impact to the JIT'er).
Having said all that, your statements are arguing against metaprogramming, and I'm not really arguing for or against metaprogramming in my original comment. Simply, I think the pattern of defining HTTP resources in the aforementioned way is very intuitive from a software development perspective. If it can be done without metaprogramming, then great. But the presence of metaprogramming doesn't, in my mind, preclude the effectiveness of the pattern.
Ok, I hear you. I agree somewhat, but not totally :) To me, the most important aspect of programming is to retain full control. I for instance want to be able to register my HTTP endpoints based on some data read from a file. I want to be able to start my app in server mode and run requests against it in any way I want during testing. What many of these frameworks do is to remove this power.
Bytecode modification is not too bad IMO, as long as it's within reason. I use a notnull annotation weaver to add notnull assertions. Not Lombok, that's too much for me as well. Should be another language really, like Kotlin.
Metaprogramming does have utility. But it has a tendency to go overboard. I actually find Jersey to be a good example of a nice and clean API for defining resources. I recently tried to integrate it into a small app where we do all the wiring manually for clarity. The problem is that it's hard to stay in programmatic control. You define a resource. You realize you need a Service in your resource. You realize that the only way of getting it into your object is to inject it with the JAX-RS annotations. And so suddenly your service has to be a JAX-RS @Singleton service. You have to make your whole app JAX-RS compatible. The framework takes control. This is really bad IMO. A library which requires control over your program.
This is not an intrinsic problem of metaprogramming, but IMO it's a big problem of Java as it stands today. We need to give up on frameworks, make sure that all libraries are combinable and that they don't take control of your app. Doing so would make everything a lot better.
I wish I could just do:
new Jersey(config).addRoutesIn(new HelloWorldResource(new MyBackendService(dbDriver))).addSerializer(new JsonSerializer()).start();
This would then utilize some reflection when routing requests, but not take over my entire app.
Regarding the libraries, my point was not that you get conflicting classes (DLL hell sort of thing), but that we're building multiple products which want to take over the world. JUnit wants to be in control, and so does Spring. So when you try to combine them, you end up with a problem. Can you start Spring programmatically from a JUnit test? Well, not really. You can, but it's very involved. So they want to provide you with a Spring-JUnit integration. This is a really bad idea, since every linear combination of libs has to get an integration, like the link I included in my post. If all these libraries just tried to stay out of the way of the programmer and just provide a programmatic API, there would be no need for such integrations.
To jump into a really interesting discussion: I totally get what the both of you are saying, but I think that the real problem are poor abstractions. In some sense SQL is code generation since it's a declarative way of describing what you want which the query planner figures out how to realize. At least to me, it's rarely a source of frustration. Why? Because it's a sound, proven abstraction based on a very solid foundation. In the frameworks that figure in this discussion, there's no such thing, and all the abstractions I've seen are leaky under some scenario.
That makes me prefer minimal, magic-free tools, because even if it's sometimes a source of extra effort, at least I won't have to spend time fighting the framework. The fear of boilerplate seems strange to me. I'd much rather spend the extra ten seconds of adding a route manually through a line of code than no time at all if it spares me having to spend an hour figuring out why my ContextProvider isn't being registered by the classpath scanner.
I had to think about your comment. Yes, SQL is a sort of code generation and you could argue that annotating classes also is code generation. But my main pain point is that most annotation frameworks are frameworks and assume that they are in control: that they start the app and that the application programmer must write their whole app according to the rules of the framework. That's like if a SQL client lib would assume that the whole app was written in SQL.
Your post totally resonated with me, particularly how once you go metaprogramming everything is suddenly metaprogramming. I work at a Spring shop and we joke about how we should just be called Senior Java Annotators. I find it ironic that Spring does all this machinery in order to instantiate your application, yet often you end up needed to use the Order http://docs.spring.io/spring/docs/current/javadoc-api/org/sp... annotation anyway to get things to wire up in the right way...at which point, it is kind of like, what even is the point of using Spring then? We waste tons of time figuring out what is getting configured, injected, instantiated, etc. to the point where I'd rather just define everything myself in the order I need it, so I know exactly what I'm getting and why. I miss normal programming, normal debugging, and normal application initialization.
And then there is the problem of CGLib wrapping e.g. for @Transactional and other annotations. Of course, it only happens for other services calling into your wrapped service. If you call your own methods from within the service, you're hitting just the service instance, not the wrapped instance, which causes all sorts of headaches. Yet people still take this disaster further with Lombok/bytecode manipulation and aspects.
Hey! A colleague who also sees the light! I've been pushing for more programmatic control in our apps due to the same problems you mention. I would actually consider myself one of the most versed in Spring at our shop. But while I see so many problems, many devs only see "the beauty of the minimal code". I wish I could understand what is blindsiding them from all the problems we have.
The latest incarnation of this problem is Spring Boot. It does some really far out stuff to your app, like making m separate metrics for each unique url of your webapp. That goes south pretty quickly. Not to mention how much longer the app takes to start, how much slower it is compared to coding the same functionality manually (x4), how much trouble people have getting the right mental model of how the app works (with instrumented transaction boundaries and stuff that you mention), how hard it is to debug (why did this endpoint return 404?) etc.
We did some POCs with vert.x and spark, and all of a sudden you see what type of overcomplicated mess we have been creating for ourselves.
I disagree. It's great syntax for controllers - place where you should read request data. Services is another story , but you will always have place where you read initial request data.
