I'm a member of the team that created Polylith, so feel free to ask questions or give feedback.
Polylith has been gaining momentum within the Clojure community. As an example, you can follow along with Sean Corfield's journey migrating the World Singles codebase to Polylith: https://corfield.org/blog/2021/02/23/deps-edn-monorepo/
Haven't had time to dive into the code yet, but the write-up is very thorough !
A couple points though:
- i wish you had made it clear from the start that the architecture applies to individual applications / services.
I wondered if "components" were made to represent only libraries of functions, or if they could be their own server running along side other services. You need to wait for the very end to get the idea that a polylith distributes / microservices architecture means "many polilths next to each other".
Which means that polylith might help organize the _code_ of a microservices monorepo, but will be of no help with the _execution_ problems of it (eg where should the data live, how many services do you have to traverse, how do you deal with desynchro and caching, etc... In my experience those are the thorny problems, not so much how your libs are laid out - but maybe that's because we already use similar ideas coming from "clean architecture", "hexagonal", etc..
- I am right to understand that most code in "components" should be concerned mostly with business domain, and free of "big" frameworks (spring / rails / etc..) and that only "bases" should be concerned with that ?
- what is the story when you need to fake a component that represents a distributed service ? The "component" folder will have an API,and an implementation that targets the real service ; but should it also provide a "fake" implementation ? Or would it be a different component sharing the same API ?
- Are there sides of the architecture that depends on being done with a dynamic language like closure ? Have you actually tried it with other languages ?
- The main idea is to allow bricks (as we call the "LEGO bricks"
- components and bases) to be put together into projects from where we generate artifacts (libraries, tools and different kind of services). Maybe we should have a sentence like that on top of the first page of the high level doc!
- The help the Polylith architecture provides when it comes to how to execute the code is that it makes it much easier to reorganise the code in production (by splitting, merging, or creating new services by reusing existing components) without affecting your single development experience (which is always only one single project). This is where the LEGO idea comes in. But you are right, it doesn't help you at all with the implementation of the code.
- A component can handle business logic, e.g. some part of our domain, others might manage integration with external systems, and others will be responsible for infrastructure features such as logging or persistence.
- The way you introduce a "fake" component (a component that can replace an existing component) is to create a new component that implement the same interface. So yes, it should be a new component that implements the same "API".
- We haven't tried it with other languages than Clojure, but the idea is applicable to other languages because it's not about the language syntax. David Vujic is working on tooling support for Python: https://davidvujic.blogspot.com/2022/02/a-fresh-take-on-mono...
I tried looking into Polylith a few months ago, when I first discovered it. I recall the feeling of experiencing information overload. Though, I am not a Clojure expert either, so that probably compounded my issue.
I believe it would be beneficial to supply more example projects. I found one, https://github.com/furkan3ayraktar/clojure-polylith-realworl..., however, it uses SQLite. Maybe an example which uses Postgres, and Redis for caching would be more real-world? Also, maybe a few deployment examples? Heroku, AWS, GCP, Kubernetes?
One question I have, how are ENV variable driven configurations handled? For example, if I need a `DATABASE_URL` etc. I recently ran into an issue, https://discord.com/channels/313110246643990528/313110246643..., in my own Clojure web service attempt where I could not use `def` to define the individual variables since they are evaluated at Uberjar build time. I eventually converted it to a `defn` but then it gets evaluated every time it's used.
Thanks! It was a bit nerve-racking to give traffic light ratings to existing architectures that people use and enjoy, but I hope we were fair in our assessment.
I really like the ideas behind Polylith. I'm building mainly data transformations in Apache Spark and I'm searching for some nice abstractions that could help to build cleaner and more maintainable code, have you tried it out in a pure data processing context as well?
Hi there! I mean to be constructive with my post and not harsh, so please take it as intended :) TLDR; show and tell is key. You can't just tell me about your project, you have to show me to make it real in my head.
I invested about 40 minutes into the video in TFA and another 10 minutes in the "nutshell" video you linked. I haven't heard of Polylith before today. I really enjoyed your presentation and I thought it was of a very high quality as far as the amount of practice you put in and the visuals you created to accompany your words.
However, I have to be honest and say that the quality of the presentation is really the only thing that kept me watching. I figured if you spent so much time on the presentation there must be something here worth hearing. although I did want to abandon the effort and move on at several points because I just had no idea what you were trying to convey.
Or let me put it this way: I did know what you were talking about, but I continually felt like I was missing something because what you were telling me was obvious to me.
In the 40 minute video some notes I had in my head were that:
- First of all, you need to cut out the guy who immediately says he forgets your name. That's not a good way to start off the video.
- But more importantly, you only start talking about Polylith at minute 8 of the talk. Really, I had to double check at a couple points to make sure I was watching the right video because you spent most of the first part of the presentation talking about a different project called Code city, and I started to think this was a presentation on that. What you do at minute 8 you should have done immediately.
- When you do finally introduce Polylith, you don't tell me anything specific or interesting. The words you use are quite generic, and would be used by almost any project to describe itself. All projects want to be simple, flexible, and productive.
- At 12 minutes in things started to click for me when you actually started getting more specific. I don't know what your YouTube statistics look like but I'm willing to bet by this point you've lost almost all of your viewers.
- At 27 minutes in I got really excited because the MC came back to say that we were going to move past concepts and theory and get a demo of the actual product. But what follows is not a demo at all, but further descriptions of a system.
- Finally I'm struck that you introduce your project Scrintel, which is described as a video transcription service, but I'm not actually using Scrintel as I watch your video. Instead I'm using YouTube. This is the perfect opportunity for you to demo your actual software. I literally had to look at the video captions to understand what the MC was saying, and the captions I believe are wrong. Why aren't you using your transcription software to transcribe your video that is selling me on your transcription software?
Moving on to your 10 minute nutshell video, it's more of the same. After spending almost an hour exploring the content you've posted about your product, you've actually never demonstrated your product to me once!
One thing about the nutshell video, and your videos in general, is that I think you should abandon the Lego metaphor. First, it doesn't have the intended effect when you display the toothpick Whitehouse and the Lego Whitehouse. When I see these two, you want me to think about how difficult the toothpick version is to make, and how we shouldn't construct things that way in software, but all I can think about is how detailed it is compared to the Lego model. The toothpick model isn't a mess, it's a masterpiece. I admire the person who made it, but you're telling me that I shouldn't want to build software with that model.
Secondly, the Lego analogy is overdone. As I said in another post, every architecture has made this claim, even if they are actually toothpicks. So when you make it, it's going to be met with skepticism, and since you don't actually demo any of your software in these videos when you make the claim, you're going to be dismissed. Especially when your videos rest so heavily on the metaphor. My opinion is, if you take every instance of describing Polylith as Lego, and you replace it with a demo of your project showing how it's better in concrete terms on a real-world example (not your startup), you will be 1000% in a better place.
Thank you for your comprehensive and detailed feedback!
I'll leave Joakim to respond to your comments about his presentation, and I'll respond directly to your comments about the nutshell video.
> you've actually never demonstrated your product to me once
That's partially true, although I'd argue that by showing the components and bases from an example system, we are showing "the product". Perhaps you're thinking about showing code or filesystem structure from a complete production Polylith workspace? We do that in other videos, but didn't think it would be a good fit in the 10-minute "why?" introduction.
> The toothpick model isn't a mess, it's a masterpiece. I admire the person who made it, but you're telling me that I shouldn't want to build software with that model.
I agree that the toothpick model is a masterpiece! However, the reason we shouldn't want to build software in the same way comes down to one word: change. The real Whitehouse doesn't change shape very often, but all the software I've ever been involved in building does. That's why building software with LEGO pieces is so much better than building it with toothpicks and glue. Which is exactly why it's a such strong metaphor.
> Secondly, the Lego analogy is overdone. As I said in another post, every architecture has made this claim, even if they are actually toothpicks.
You might be right about this, but most of the other recent architectures we'd come across seemed to steer away from talking about LEGO (onion, hexagon, DDD, microservices, etc.), so we were hoping it wasn't overused.
- The Code City project is a cool way of visualising code and the idea was to show how we normally organise code today, and what problems it creates (development experience + sharing). I agree I could have been more clear about that.
- This is the longer video where I (and Furkan) try to explain what Polylith is, what problems it solves, how it works, and why you should use it. The sad fact is that it's super hard to explain Polylith (we have given about 15 different presentations) and I think you need to try it out yourself to get an idea how it is to work with and which problems it solves.
I will leave to Furkan to answer your questions about Scrintal.
Polylith has been gaining momentum within the Clojure community. As an example, you can follow along with Sean Corfield's journey migrating the World Singles codebase to Polylith: https://corfield.org/blog/2021/02/23/deps-edn-monorepo/
There's an active Slack channel, with people and teams using Polylith on both commercial and side-projects: https://clojurians.slack.com/archives/C013B7MQHJQ
Outside of the Clojure community, there's a project to port Polylith to Python, using Poetry: https://pypi.org/project/poetry-polylith-plugin/
We'd love to see it get ported to other languages too!