Hacker News new | past | comments | ask | show | jobs | submit login
I wrote a children's book / illustrated guide to Apache Kafka (gentlydownthe.stream)
1797 points by mitchum_ on June 17, 2021 | hide | past | favorite | 226 comments



I can't shake the feeling that the otters could have used a bulletin board - on the left side they could have written "what kind?", in the middle "when", and on the right more detailed description as necessary. Then any otter in a hurry could skim through the left column, find topics they liked, and just read that part from the right.

Sorry, it's not really a fair criticism, and I did like the art style - I just don't like Kafka much because I had to deal with it in places where it didn't make much sense.


Where would you say it makes more sense, design wise? I feel like its hard to find perfect use cases for Kafta.


Sorry, I'm not qualified to answer the question. You will have to ask someone who has seen Kafka being used in a place where it made sense...


I have not used Kafka, but what the book describes works with most queue systems. And those are great for localizing dynamic complexity. One part of your system behaving weirdly (spiking, dying, fluctuating etc. ) can’t affect other parts because they sit at the other end of a queue happily consuming the messages at their own pace. This means tour complex system becomes less complex in terms of behavior and is less likely to end up in some weird metstable state


This is sort of funny to me, since ActiveMQ had this exact problem. A single slow client could break the entire system because the broker would slow down producers to prevent exhausting the queue space. Which was exactly some "weird" state because it wasn't obvious what happened until you spent a lot of time debugging.


this made me chuckle


For me a perfect use case is sensor data processing. I've been involved with two independent sensor data platforms that used kafka as the backbone. Sensor data is persisted unprocessed to raw kafka topics, and then gradually deduplicated, refined, enriched and aggregated across multiple stages until finally a normalized, functionally sharded and aggregated view of the data is stored in databases for dashboarding.

It is easy to scale horizontally to massive volumes of data, and any issues in the processing pipeline can be fixed without losing any raw sensor data (restarting the consumers from the last known valid point).


I'd recommend Kafka for any case where you need to act on data that changes over time - almost any software system. In almost every system you want to be able to reliably record events, and you want to be able to partially replay your state in a deterministic way (e.g. after you fix a bug); both those things should be table stakes for a serious datastore, but AFAIK Kafka is practically the only one that offers them.

Kafka itself kind of only solves half the problem, because it doesn't offer indexes as a builtin, so you have to build your indexed state yourself, or maybe even use a "traditional" datastore as a de facto cache. But since you've moved all the hard distributed part of the problem into kafka, that part is not so bad.


Kafka is a streaming log service. People who use it just for passing messages are hugely overengineering something that would fit just fine for something like RabbitMQ, Redis or other tools.

An RSS feed must be powered by something underneath, and any of those tools can do the job. In most situations it would be extremely impractical to use a relational database for this kind of thing. If you are getting thousands of messages in per second, which is not uncommon, no transactional database will give you enough write performance and won't be able to handle too many queries per second for clients polling for updates, like an RSS feed would require. Note that caching queries is almost useless here because the latest content is updated every few milisseconds.

Kafka, as pretty much any other queueing datastore, is optimized for append-only writes with no deletes or updates. Reading from the end of the queue is extremely fast and sequential reads down the stream are quite fast too. Random reads such as the ones commonly handled by SQL databases are either not available or are less efficient than with SQL.

That said, Kafka can be used to pass any kind of message between applications: from simple text messages and small JSON data to vídeo frames, protobuf messages and other types of chunks of serialized data.

It is also a very durable data store for immutable, time-ordered data, and is widely used in the financial world to store transaction logs.


When you have many servers who all need to see the same chronological data stream (including messages they might miss during network downtime), and see new events in real time.

If you set "log.retention.hours = -1" and "log.retention.bytes = -1" in kafka config, kafka stores messages forever.

In a game for example, user-inputs and other events can be produced in Kafka, then reconstruct the entire game-state by reading and processing the kafka-stream from start to finish. It has an advantage over most DBs because it's real-time.

You can use also chronological data streams to represent data structures more complicated than a simple array. For example, a tree can be represented while preserving chronology. This is far from the ideal use case however.


> In a game for example, user-inputs and other events can be produced in Kafka, then reconstruct the entire game-state by reading and processing the kafka-stream from start to finish.

This is called event sourcing.


Wherever you have a firehose of data that needs to be processed.

I've heard of it being used as a sort of message queue for application level events before, but that sounds like a nightmare of trying to reinvent the actor model with 1000x the complexity.


Bah god, that's Akka's music!


Akka, and a lot of actor model services break microservice availability, durability and general reliability because nuking a random node messes with Akka and now whatever actor happened to be on that node is now stalled until it's transferred.

Just like SOA and ESB, the concept isn't the problem, is the technical constraints of the design at the time. Decoupling and messaging isn't bad, but having a legacy message queue on physical hardware doesn't really hold up. Any derived architecture faces the same problem.

Then again, Kafka isn't an actor model implementation, and Akka isn't a partitioned redundant stream processing system, they don't have all that much overlap ;-)


If you shard your Akka actors, the messages are buffered and passed to the actor when it's initialized on the new node. You get even more stability if you persist your actors backed by a DB or some other persistent store.

Not saying Akka can replace Kafka but many of the issues around availability, durability and reliability have been attempted to be solved in Akka.


Yeah, that's true. I think the main issue in most cases where I see problems are the ones that only use in-memory stores.


your bulletin board could be an RSS feed. most often you do not need kafka (or a service bus) for distributed systems.

RSS feeds are great as they are easy to implement and debug.


It's really beautiful.

Surprising too. I would have expected a far darker approach to this subject!

Perhaps something like 60's horror comics genre, or better, like those Jack Chick religious tracts where people end up in burning in Hell forever as an immediate consequence of a sin against god.


It’s rare a piece of tech has a more fitting name!

“Is your orgs politics so complicated that direct team-to-team communication has broken down? Is your business process subject to unannounced violent change? Bogged down by consistent DB schemas and API versioning? Tired of retries on failed messages? Introducing Kafka by Apache: an easy to use, highly configurable, persistently stored ephemeral centralized federated event based messaging data analytics Single Source of Truth as a Service that can scale with any enterprise-sized dysfunction!”


Or something written in the style of Franz Kafka.


"Childrens book" ... was the goal.

I don't think many children are into Franz Kafka. Kafka is more for cynical grownups. (And teenagers about to become it). It was very Kafkaesque, having to read Kafka in school ..


https://en.wikipedia.org/wiki/A_Little_Fable is probably a close as Kafka got to a children's book.

"Alas", said the mouse, "the whole world is growing smaller every day. At the beginning it was so big that I was afraid, I kept running and running, and I was glad when I saw walls far away to the right and left, but these long walls have narrowed so quickly that I am in the last chamber already, and there in the corner stands the trap that I am running into."

"You only need to change your direction," said the cat, and ate it up.


You might be channeling a chapter on dead-letter-queues in Kafka.


I've seen a lot of these children's books about technical topics. Is there any record of an actual child reading any of them? And, as children do with good books: reading them more than once? (Excluding children in the author's family.)

I have read about 10 of these "children's books about programming" and while I enjoy them myself, I find that they lack most of the things that grab children's attention, such as repetition and visual-only sub-plots.

This is not to criticize the use of an illustrated fable as a storytelling device for adults: They're great! It's just sad that we have to frame an illustrated fable as "for children" in order for it to be accepted. I think it says something about how content dictates and narrows the expected presentation format, sometimes to the detriment of clear communication.


My 5yo loves reading the kubernetes book (paperback copy of https://www.cncf.io/the-childrens-illustrated-guide-to-kuber...), I found him reading it by himself the other day I guess it has sort of a plot.

I think he likes it mostly because I tell him that’s what I do at work.


Wow this one has quite the vocabulary! Right in the first pages we have "hosting provider", "environment", "OS", "Web Server", etc.


It's got very pretty pictures, so that's very understandable! Have you been able to find out if he understands any of the abstract concepts that are explained, or is he mostly interested in the character interactions?


Can you list the books?


Probably the most famous: https://en.wikipedia.org/wiki/Why%27s_(poignant)_Guide_to_Ru...

Not my cup of tea (loved the current Kafka one)

I prefer when books for learning use a consistent art style.


I've (rightly) never seen Why's Guide marketed as being "for children" though.


While not directly related to tech, this story is great analogy about ping.

https://en.wikipedia.org/wiki/The_Story_About_Ping


Most of the books have been low-key attempts from programmer parents, probably to get their own children into programming. They were posted on different forums, and I have a hard time finding my way back to them.

The example that I think illustrates my point best is: http://arthur-johnston.com/hacker_writes_a_childrens_book/ (which was posted here in 2017: https://news.ycombinator.com/item?id=15879519) The book works well as entertainment for grown-up programmers: "G is for Garbage Collector/when something's no longer needed/it frees up the memory/so your program is speeded." You can look inside the book on Amazon for more examples. I judge this rhyme as too advanced for any child below 8 that I've read aloud to. Most theories about cognitive development agree with Piaget (1896–1980) in that children have a hard time grasping difficult abstract concepts before the age of 12–13, so there is at least some "scientific backing" to my hunch: https://en.wikipedia.org/wiki/Cognitive_development#Concrete... (This is only a hunch though; the children I've read aloud to are all picked from the same group – children of family and friends.)

My observation is that children seem to need _lots_ of concrete verbal and visual imagery in order to stay interested. You can also get away with more abstract themes if the actual text is lyrically well-crafted, like Dr. Seuss' books (or André Bjerke's children's rhymes in Norwegian, my mother tongue). The most successful attempts at teaching programming to children that I've seen, seem to give the children a lot of practical tasks they can work on (e.g. the Hello Ruby book series). You also have some programming elements in Minecraft that children could pick up, because the concepts are implemented as concrete objects.

All of this makes me suspect that the main audience for these "children's books" teaching programming is adults that already know a fair bit about the subject. And that's completely all right by me, because it gives the books traction and brings the book's readers entertainment.

PS: You also have the Javascript/HTML/CSS for babies series, which are only jocularly presented as children's books: https://imgur.com/eOYc8fC


This is awesome! I will prefer all my technical documentation in children's book format from now on.


Agreed. Author has done magnificent work on the illustrations and animations.

This prompted me to make an awesome repo to collect children's books on technical topics (which I have seen a few on HN).

https://github.com/searchableguy/awesome-illustrated-guides

I couldn't find a similar list. If there's one, let me know.



Well, you should really include the story of Ping: https://www.amazon.com/review/R2VDKZ4X1F992Q :)


SELinux coloring book!


Why?


I like the novelty of it. I have a copy of The Manga Guide to Databases at my desk that I occasionally 'lend out' to people that mess up my databases. It won't turn anyone into a db hero but it's a decent primer at least.


What the heck. I didn't know this was a thing. Thanks for the pointing it out.


Different strokes, I guess; I can't count the number of people that openly gush about _why's Ruby book with the cats or whatever, but to me it just read like the ravings of a highly functional something-opath. But reasonable people can disagree, and I'm sure I'll get downvoted for disagreeing with the hivemind.


I don't really get these weird ways of explaining different technologies, just give me a straightforward text description. But straightforward text isn't going anywhere and it doesn't hurt me if people want to read mangas about foxes or whatever.


To me, the attraction is less about the specific art used to communicate the concept, and more about the careful attention to a fully-formed analogy that explains the tech in completely different terms.

These kinds of explanations tend to focus on the most critical/important concepts, and help validate (or dispel) assumptions I've made about the tech.

This focus on analogy also lets the author tell the story faster, because I already understand:

- What an otter is

- That rivers flow

- The water flowing down a river that forks will be spread across those forks

- etc...

Depending on the strength of the analogy, it's possible to get the reader on the same page much faster than an intro/tutorial that must first explain foundational concepts just to get to the basics of the technology itself.


I've never really been a fan of analogies, because I feel like in the end I just need to first understand what the thing is underneath, so I can understand how the author has built their analogy... so the task of understanding why they used this analogy is equivalent to understanding the thing itself.

But I can't deny they are very popular!


Do people find these analogies helpful? The concepts aren't that difficult, the audience for them is already technical, and adding a cutesy abstraction about it makes it harder to understand.

The art and animation in this is great, but I feel like the author's talents are wasted on a document with no audience. Make a kids' book instead!


Yea that's where I was going with asking why. It feels like it's a trendy thing is to convert a technical thing into a children's book. (Which is odd.. children don't need that book.. so it's for an adult that wants to consume children's literature)

Note: I'm not against creative attempts to explain technical concepts. But the form to me seems odd, and that it feels like we're producing very short tutorials in a childrens format. That's even weirder.


My initial impression was "more of this garbage" - but it was really well done in the sense that it distilled the core functionality of the system in an easy-to-understand form.

The guy above you mentioned manga-guides to stuff, which utterly fail at their job, which again, is to distill key-information in an entertaining, easy-to-read, general (but fully accurate) manner.


I never found it helpful from a purely technical perspective, but I found it extremely eye-opening as an unorthodox approach to programming that really captured my imagination. It was definitely something that encouraged me to dig deeper into Ruby and do more explorative "creative" coding.

There's no shortage of dry technical documentation, so seeing something akin to outsider art in that space was really refreshing.

Personally I would love to see more technical books come with a soundtrack!


> ravings of a highly functional something-opath

I'm going to try and fit this into as many sentences as I can get away with!


Because to me it seems like a very pure way of separating a concept from implementation. In language and metaphors that are easy to understand and fun to read. I really like it. Now, in a very short time I can decide if Kafka can solve my problem or whether I should move on. I for one store information presented in this way much better, it's feasible that in 5 years I'm presented with a problem and the Otters pop into my mind.

Different people, different preferences I guess.


_why, that’s why.


caffeine gets it...


Reminds me of the Microsoft Server illustrated childrens book :D

https://imgur.com/gallery/kvATA


Oh my goodness. This had me cracking up:

>> "When a mommy and a daddy love each other very much, the daddy wants to give the mommy a special gift.

>> So he buys a "stay-at-home" server."


Holy cow. I had no idea this existed, but it's amazing in a terrible way.


Wow this is such a beautiful read. Has anything like this been done before for some other topic?

P.S. This sentence is hard to grasp for me "This Unawareness helps Decouple systems that produce events from otters that read events." https://www.gentlydownthe.stream/#/20


I tried it with a bunch of 101 comsc topics in one story - boundary conditions, recursion, high vs low level languages, undefined behaviour, memory protection https://www.royalroad.com/fiction/41545/manifest-logic/chapt...

More generally there's this list of similar fiction https://fiftysevendegreesofrad.github.io/hard-comp-fi-fictio...


The Reinforcement Learning algo A2C was also illustrated in a similar way:

https://medium.com/hackernoon/intuitive-rl-intro-to-advantag...



I don't have a link but the NSA has a coloring book about cryptography that's in the public domain, since it was created by the government with tax dollars and has no copyright notice. I actually have a copy I've been meaning to scan, I'm just afraid of ending up on a list if I do. :)


The selinux coloring book is another great example: https://people.redhat.com/duffy/selinux/selinux-coloring-boo...


The penguin in this one is nightmare fuel


I'm glad people are enjoying this. I wrote a story two years ago about Queues / Kinesis - and am in the process of getting it illustrated. I doubt it'll make any money, but it's good to know that people enjoy this type of thing!


Not quite the same, but "Life on the Infinite Farm" https://www.math.brown.edu/reschwar/farm.pdf



Does Why's Poignant Guide to Ruby count?

https://poignant.guide/


> "This Unawareness helps Decouple systems that produce events from otters that read events."

It simply means that the producer doesn't need to maintain a list of listeners. It just throws the event into the stream, and assumes that anyone who wants to read it will be able to do so.

RSS vs an email list, I guess.


This is fabulous!

Is there a typo at https://www.gentlydownthe.stream/#/22 ?

Current version: "First, they dropped large stones into the river, splitting each topic into a smaller number of streams, or Partitions."

I know nothing about Kafka, but I think maybe this should be: "First, they dropped large stones into the river, splitting each topic into a number of smaller streams, or Partitions."

(my emphasis for both versions) "


For kids only? I personally had little understanding of Kafka before and now I get the gist of what it does. Maybe books should look at this style of explanation instead of just putting a bunch of crazy equations on day 1 that put 90% of people off.


No crazy equations? But how else can I prove my intellectual superiority over lesser men but by demonstrating trivial concepts with greek letters, in keeping with the way Plato originally taught computer science?


If you like explanations that are more playful, Why's (Poignant) Guide to Ruby comes to mind: http://poignant.guide/

Not necessarily reminiscent of a children's book, but still a more entertaining way to learn a programming language than most guides.


I've been thinking about comics explaining different IT concepts, but so far only produced one :-) https://vlad.studio/wallpaper/?how_internet_works


Cool to see you on HN! I use to rotate through all your wallpapers back in the day (what seems like forever ago) - glad to see you're still at it!


Thank you! I'm way less productive than I used to be, but I like to think my best drawings are yet to come :-)


Psst, files.vlad.studio's cert expired yesterday. Really nice site design and artwork!


ooh, perfect timing :-(


No worries. ±1 day is universally race-condition territory. Optimizing at that level is super tricky and incredibly hard to get right, which is why I wanted to mention it.


Your drawing skills are good. Is this scanned pencil drawing with post-processing? Forget Kafka! How do I learn to draw like this? :-D


I think it is using procreate, it also has frame by frame animation options. Awesome Ipad app


Can someone please explain why to use Kafka over Redis streams? Is it basically use Kafka if you need your data long term and/or if it's way too big for an in-memory store? Redis seems much, much simpler to setup.

Even if you do need to keep your events long term, why not use something like Eventstore?


4 years ago at work we compared Kafka to alternatives and ended up using it. In my opinion the only reason for using Kafka is that you need a very very highly performant message queue. It had bad usability at the time and some very annoying undocumented behaviour (e.g. there was no official guide or feature to permanently remove a node).

It feels like whenever Kafka developers had to make any tradeoffs they chose performance over everything else. I would only ever use it again if I the sheer data volume makes it impossible to use an alternative.


With a friend we’ve built this [1]. Usability has been our main focus, though we ended up having very good perfs (millions of mps out of the box, usually 2x to 10x kafka). We provide REST and WebSocket APIs, + a simple to implement binary protocol for the most demanding workloads. It’s in Go so very light and rock solid in stress tests. We provide trivial to implement and understandable usage patterns for exactly once processing. We’re currently working on HA and cloud integration. And we’re looking for community feedback BTW ! :)

[1] https://github.com/dataptive/styx


Kafka is more focused on distributed use-case, I think. It is built around being able to write to fault-tolerant queues, and having multiple consumers in a group reading from it; you can have it configured so that losing a queue node will not lose availability of any of the data in the queue, and losing a consumer will not drop any messages. All of that while being able to handle very high throughput, even with persistence. The tradeoff is in the baseline complexity of Kafka is much higher.

As far as I know, Redis Streams does not offer all of that on its own. You could certainly build a lot of that for Redis yourself (and someone probably has already), but then you start getting back that complexity.

So if you have high requirements for performance, durability and availability, are don't mind the added complexity then Kafka is worth a look.


From my limited research, Redis streams offers pretty good durability if you run with AOF and sync frequently. Performance should be great since it's all in memory. As for availability, there is sentintel and cluster mode which I know nothing about.

edit: the differences in consumer groups confuse me as well. With Redis streams, if you have a single stream with a single consumer group with multiple consumers, each consumer will get a new message at different times so processing of each message may happen out of order. That seems... fine to me. Apparently that is not the case with Kafka which makes me ask, why have multiple consumers in a single group if one will be blocked on the other?


Kafka consumer groups do not block. Kafka uses partitions to scale out a single stream/topic and a consumer group allows consumers to exclusively read a partition. Multiple consumers can read from multiple partitions simultaneously, as long as there are enough partitions available.

Redis doesn't have partitions, and instead has a single host managing a stream, while taking on all the consumer tracking functionality. It's fast, but not as scalable, and eventually if you keep growing then scale is how you get speed.


If a topic is split across multiple partitions and we have multiple consumers, won't messages be read out of order?


Messages within a partition are read in order. Partitions themselves may be read simultaneously or out of order. This is the trade-off for scalability.

You can compromise by using hashing or other logic to group related messages into the same partition. For example, hashing by user-id lets you process events for a particular user in order, while processing all events by users in parallel. If you can divide into logically consistent but globally isolated boundaries then this setup works very well.


Order guarantees is not part of the promise with Kafka. If your consumer drops or times out the broker will pause all active consumers for the topic, revoke assignments and issue their new assigned blocks based on the new consumer group acknowledgement count.


I love that this is life before Kafka: https://www.gentlydownthe.stream/#/4

And this is life after Kafka: https://www.gentlydownthe.stream/#/24


While I love the commentary and found it funny, it would more accurately be "life before and after a huge volume of messages". Kafka isn't the problem in this specific case.


That page 24 image was exactly where I felt the otter metaphor was perhaps being stretched to its breaking point.


What a Kafkaesque situation. Really gives new meaning to the word "Kafkaesque."


This is a wonderful contribution to the software development community, in the vein of Why's Poignant Guide and Land of Lisp. I fully support such combinations of artistry, whimsy and technology - true to the core hacker spirit.


I can't help but feel the otters kinda over-engineered this whole thing.


You need some other animals - Beavers maybe? - who are in the same forest and who just have a simple system of scratching messages onto the side of a single large tree somewhere that everyone goes to look at regularly. Then at the end see how many picnics the beavers missed vs how many otters lost their fucking minds from the complexity of the system and then see who comes out ahead.


I think the otters should question some of the things that are meant to be axiomatic. Is tight coupling really a scalability problem? Otters should be prepared to defend these statements. Nixie's song might be the vacuous equivalent of "Mongo is web scale".


Also: did adding the stream really decouple the otters, or did it just make the coupling less visible? After all, the consuming otters may still depend a great deal on the precise behaviors of the producing otters. It's just that now the producing otters don't know in what ways other otters depend on them, making it harder for them to make changes without harming their otter-dependents.


The fact the producing otters don't know the ways in which the consuming otters handle the message is precisely the decoupling its talking about, no?


Nobody used the word decoupling. They just went from being tightly coupled to being loosely coupled.


WOW, this is so nice. The drawings are really cute and the animations give it its own style. Bit of feedback the color palet all over the place. Could use a bit of hierarchy to help the focus of the viewer and reuse same colors to make it more coherent.


https://www.gentlydownthe.stream/#/10/1: I was wondering why the “//gi” and “/round-robin-books/g” were purple, while the rest of the line was white. hljs decided the whole block was awk, and those parts regular expressions. Might be nice to tell hljs it’s a different language.

https://www.gentlydownthe.stream/#/11: “In the rivers gleam” doesn’t seem quite right; I can’t decide whether that should be “In the river’s gleam” (the river doing the gleaming) or whether it was intended to be “In the rivers gleam” (the events doing the gleaming, in multiple rivers) in which case I suspect it should be a singular river.


Wow! That was cool.

Now I want a Docker children's book please.


I mean, Kubernetes is a superset of Docker, right? https://www.cncf.io/the-childrens-illustrated-guide-to-kuber...


Not really (unfortunately). Kubernetes can leverage a number [1] of container runtimes - OpenShift uses CRI-O, for instance. There's also the problem that Docker can refer to a container runtime, orchestration engine (Swarm, which is analogous to k8s), CLI tools (`docker` and `docker-compose`), or HTTP API (registry). At my last few gigs the maintenance of separate configs for docker[-]compose and k8s has been a consistent pain point.

The container ecosystem is pretty complicated :-).

[1] https://kubernetes.io/docs/setup/production-environment/cont...


And then do Monads.



I've been trying to put my finger on why Kafka so well captured the imagination of many distributed systems engineers. My best answer is, "low-cost publish and multi-consumer data-sharded subscribe is the key to resilient horizontal scaling and parallelism."

Kafka has its flaws, but it really served us well. We have Python Data Engineers who focus on distributed system design[1], and Kafka is one of the team's least finicky open source components, but it is used everywhere, and it basically enables the entire rest of the real-time data processing stack.

[1]: https://www.parse.ly/careers/python_data_engineer


"Captures the imagination of distributed systems engineers" is the biggest red flag I can think of.


I've seen many comments here along the lines of "but when should Kafka be used?"

If you want a deep dive into event streaming systems (logs) that also gives some examples of big data systems they interact with, I highly recommend "The Log: What every software engineer should know about real-time data's unifying abstraction," from LinkedIn, the creators of Kafka.

Their engineering blog also has many other interesting articles on their data systems.

[0] https://engineering.linkedin.com/distributed-systems/log-wha...


pretty clever!

on page 12, i think you mean "streams" rather than "seams". no? also, the page title is "gently down the steam", is an R missing?


This is so awesome. It almost made me cry. I think it's a brilliant way to introduce a complex technology because we have all grown up to absorb lessons through stories, especially stories about cute animals. And you know what? we remember those stories and the lessons we glean from this. I'm also glad I can swap the image I have of Franz Kafka's Gregor Samsa bug with cute otters.


This is really cool! Clearly a lot of thought went into making it.

That said, I feel like the otters could have just made a bulletin board to solve their problem


Reminds me of "SCADA and Me (A book for Children and Management)"[1].

[1]: https://www.amazon.com/SCADA-Me-Book-Children-Management/dp/...


I love this! Interested to know how you made these animations?

also mathematic and science in ancient india were taught using easy to remember poems, for example Pascal's triangle:

https://lvnaga.wordpress.com/2014/10/21/meru-prastarah-or-pa...

also different numeric series represented as poems https://en.m.wikipedia.org/wiki/Katapayadi_system

we definitely need more of these that teach complex mathematics and topics like machine learning.


Taking ELI5 to another level!


level 2 ELI5 , Illustrate like im 5 (ILI5)


Who is this actually for? Programmers need clarity and precision, not "fun" analogies stretched past their breaking point. (I mentally facepalmed when it got to the glass floats. Why even bother with the river at that point?)

And it can't be aimed at children, given that amount of technical jargon (we suddenly go from otters and rivers and bees to headers, keys, values and timestamps). And, well, children don't need a book about Kafka.


This is awesome, excellent work! Like some others here, I feel like the otters over-engineered the entire thing using fancy glass balls and partitioning a river where a categorized bulleting board would have been good enough.

But then I realized that that's what makes it a realistic story ;)

Jokes aside, the best explanation I've ever seen. It should be a must-do for people who want a quick and understandable introduction to the concepts.


I can’t find the feedback link but I feel the animations could be a bit slower - hard to read the small words when flashing so fast on iPhone.

Otherwise quite well done!


Cutest thing I've seen today. Every tech team should hire a children's book illustrator to explain their products/tech, IMO.


An exceptional form of creative education on a difficult technical topic with stunning visuals that is intuitive and fun. Brilliant!


Absolutely adorable! Really excellent work :D


Fantastic. I'm betting a large group of people would much more likely watch/read this than a typical technical paper, and still gain 60% of the knowledge.

I have to say, i was looking for a slide on how the otters cleverly know what messages they've already read, and that was unfortunate, but kudos!


Awesome! Love the illustrations and the subtle animations underneath.

Quick feedback: My nephew prefers page turns (curling) on their iPad. I assume that he prefers it over a powerpoint style slides that we currently have on your book. Well, at least he spent some time flipping through the animated pages!


Why did the otters use streams as opposed to e.g. having a post office? Is their throughput really that high? Or would a more regular message passing solution be good enough for them?

Perhaps they usually work with streams and couldn’t resist applying a familiar solution to the problem?...


This we very helpful for someone unfamiliar with Kafka. You could probably get a job doing this.


I hope this doesn't encourage these children to grow up and start using Kafka willy-nilly in places that it's not meant to be used.

It's often hard to find a legitimate use case for Kafka/RabbitMq/etc. where a much simpler solution won't do.


This is great because (like _why's {poignant} guide to ruby did for me) it breaks the psychological barrier that I have to even get an intro to those technologies -- as an adult developer -- because it has new concepts that are foreign to me.


Love it! But the cdn serving the images is improperly configured. The browser hits the cdn each time you go back and forth, downloading the images (multiple megabytes each) every time.

Likely the `expires` header is the problem (it's a date in the past)


Material like this create a feeling of connection, which is important to me when learning something.

The survey asked "would you be interested in hardcover", my answer is no. Wouldn't know where to put it. Also less paper the better.


This reminds me of “The Story about Ping”. The Amazon review of is classic.

https://www.amazon.com/review/R2VDKZ4X1F992Q


From the review: "The Story About Ping has earned a place on my bookshelf, right between Stevens' Advanced Programming in the Unix Environment, and my dog-eared copy of Dante's seminal work on MS Windows, Inferno. Who can read that passage on the Windows API ("Obscure, profound it was, and nebulous, So that by fixing on its depths my sight -- Nothing whatever I discerned therein."), without shaking their head with deep understanding. But I digress."


> Using deft allegory, the authors have provided an insightful and intuitive explanation of one of Unix's most venerable networking utilities. Even more stunning is that they were clearly working with a very early beta of the program, as their book first appeared in 1933, years (decades!) before the operating system and network infrastructure were finalized.

The fact that since January 2000 nearly 16k people "found this helpful" opened my eyes to how Amazon book reviews really do have the potential to be "classic".


The OP link is blocked for me (work VPN) but seeing the title immediately made me think of "Mommy, Why is There a Server in the House?" https://www.amazon.com/Mommy-Why-There-Server-House/dp/16053...


I was thinking more the Little LISPer/Smalltalker.


This is amazing. Great way to illustrate. What software did you use?


Love the concept, and nicely executed! But what is Kafka-specific here? It seems like this illustrated guide describes any highly scalable, persistent pub/sub system


I think ...Hm... That it's awesome. The notion that a book can possess child-like qualities... which convey complex ideas is appealing. The audience can be anyone.


This is great. Can you write me one for elastic search now :-D


Really enjoyed reading it. the analogies are very clear, easy to understand and fun to read. the illustrations are amazing as well. wish there was more of it :)


This is one of the best things I've ever seen on the web (or off it!). It's now one of my favorite books <3

Thank you for sharing this and really brightening my day.


Hey, this is great - what tools did you use to make this?


Super cool, thank you. Might I request Kubernetes?



This has been shared in other comments here but just in case :) https://www.cncf.io/the-childrens-illustrated-guide-to-kuber...


I love this kind of high-level introduction of what a thing is. Sometimes very hard to find for technologies. Also otters are cute.


Great job and excellent art style. It provides enough material to get any curious mind interested in reading further about Kafka.


Honestly did not expect this to be so fun to go through. Doesn't pull any punches when digging into the details either.


This is SO beautiful!!! Wonderfully done. All the best from my 2 girls to yours. I'll have mine read it.


Beautiful work! I love the illustrations and animations. Did you use Photoshop or a another illustration tool?


Should come with a disclaimer - It is addictive. Once you click the link, be prepared to spend 10minutes...


With the initial focus on families sharing news, this made me think about Facebook (or better yet, RSS!)


For kids only? I've been working with Kafka for a decade and found this to be an absolute delight.


That was glorious! Do kubernetes next!


Finally, I can learn what kafka is :D


This is great, I was pleasantly surprised to see the feedback here has all been positive as well.


Outstanding! Purchased the eBook. Will read to my son who wants to be a coder when he grows up.


Also more approachable than the little schemer :-)


Or the little prover. Ugh.


This is amazingly cute. And the analogies feel like they’ve been very carefully chosen. Bravo!


Very cute! I bought one of the Round Robin Publishing T-shirts. Truly beautiful drawings :)


Can you do this for every technology stack, maybe one for multi-threading in C sharp


monad tutorial via children's book


This is so well done. I've made it a mandate for all my engineers to read it.


the book was really good. However, just because you liked it why do you have to mandate your employees to read it?

Why not just make a recommendation.


Not OP, but I think it was a bit of tongue in cheek comment (I might be wrong tho) ;)


I like it, I only heard of Kafka before, now I know it's mqtt on steroids.


This is great! I am going to read this to my 3 year old and hope he likes it,


It's so cute and I really enjoyed the high level overview of Kafka.


I work with Splunk and this book would be good for some of my users..


Soooo lovely, this is the most romantic things I've ever seen.


Does anyone remember the one about Nitix? Does anyone have a copy?


This was really great. I would like to see more content like this.


PSA: If you are anxious like me, there are total of 32 pages.


This...is brilliant. I could also see this as a print book.


Awesome art - thanks for sharing. Who is the artist? You?


Written and illustrated by Mitch Seymour (on the cover)


Extremely well done! What did you use for the drawings?


Great read and well put explanations. Thanks for this!


How will the otters handle cross region communication?


Beautiful. Great work, really enjoyed the animations!


I found the animations very distracting when trying to read the text. To me the animations didn't add anything.


This is awesome. Congratulations for creating this!


Nice. I expected Kafka to be a bird, not a river.


This is great. Is there a way I can pay for it?


I will show this to my daughter later today!


Kudos to you for writing this! Well done!


I'd pay for the magical place sequel.


This was nice and refreshing. Thank you.


loved it ... the merchandise should do good ... sadly international shipping is expensive.


Can you do Haskell next please?


I started writing a bedtime story called "Ed and the Awk". If someone could illustrate it, that would be amazing!

My homestay family sister Anna, 9 years old, told me that I spelled "sed" wrong :D

ed likes to write books. His home is near the C.

ed has short black hair on his head, with a curl.

ed has a cat with a long tail. His cat likes to read books. He likes it when you tcl his tummy.

Last time, ed looked into the clear sky. It's a nice view. On top of the arch, he saw a bird.

"Whatis your name?" asked ed.

"Awk" said the bird.

"Can you talk to a man?" asked ed.

"Yes" said awk. "I read books. But only sum words."

"Can you find all the words that start with a B?" asked ed.

"books" "black" "bird" "but" "bin" "big" "bits" "bash" sed awk.

"That's right! You're very good at reading. Which place do you live in?" asked ed.

"Now I live in a bin." said awk.

Awk is hungry.

To goto the field, he crawls through a pipe.

He gets some food from the field, and puts it on the table to eat it with a fork.

Yummy! He takes a byte.

Oh no! Now he's too fat to go back home.

"Don't worry", says ed. "I can sort that out."

"My cat will cut food from the field and keep on sending it through the pipe for you."

"Now I need to go to the toilet." says awk.

They go to the toilet. Wait while IP, says ed.

Awk sits down and unzips his trousers.

awk does one poo, two, three, four, five, six, seven, eight!

The wc has 8 bits of poo. There's a big login the heap. Stinky!

Flush! The poo goes away through another pipe.

The awk must wash his hands. But the sink is grotty and it leaks.

ed has strong arms. He can flex his muscles. "I'll make it better with tar!" he says.

With a bash and a clang, the sink is flowing again.

"Eat less next time." says ed. "The only animal who eats more than you is the GNU."

"I was hungry!" says awk. "What did you expect?"

It's time to leave. There's no reason to stay here idle. "Welcome back to our group any time. cu later!"

Questions:

1. Who does ed meet in the story? (a) Gnu (b) Awk (c) Python

2. Can you remember a word starting with B? (any of "books" "black" "bird" "but" "bin" "big" "bit" "bash")

3. What takes food to the awk, and flushes the poo away? (a) Pipe (b) Bin (c) Field

4. What do you do after going to the toilet? (a) Watch YouTube (b) Brush your teeth (c) Wash your hands


Your kids are very lucky!


This is effin’ beautiful.


I love this. Great work!


Really well done, bravo!


Amazing! Really nice.


This is outstanding!


Could be just me. I see Apache kafka more nightmarish than bureaucracy in 'The Castle' or Police/judiciary in 'The Trial' from author Franz Kafka. Mr Franz btw stole his last name from this very famous Apache Kafka project.


Maybe it's just you, maybe not. It would be beneficial for you to provide some substance to your claim. Then we would have something to discuss.


Unfortunately we do not have something to discuss. All these consumer/producer/broker failing randomly, system behaving erratically is me doing it wrong. Those white papers from Confluent and testimonial from customers are proof that Kafka works fine.


It's quite beautiful, but put so simply it does look nightmarish.

"Then, the otters would decide which part of the river to put the message in"

I'd imagine a child looking at this going like "why are they doing this to the river". Why are they are throwing things into the river. :D


No, put so simply it's cute. In production it's nightmarish.


Individual Kafka consumers and producers tend to have simple behavior, which is good, and initially plugging them together yields simple, predictable systems, but people tend to keep going until they experience pain, at which point they have a system that is right at the limit of what they can understand. Then further feature work pushes them over the limit into darkness.

I think Kafka needs the equivalent of OpenAPI and Redoc, a simple spec and document generator, but for groups of consumers and producers rather than single applications. This would increase the tractability of complex systems, but it would also let you see the system getting more complex over time, even when you haven't reached the pain point yet.


https://www.asyncapi.com/ is in its infancy but is attempting what you describe.

(I view OpenAPI as a near-failure, but, good luck to everyone trying.)


Thanks for the tip; that's a really cool project. It looks like the documentation generator for it works on a single file at a time, and a single file defines a single service. I was thinking of something that works on the level of a system of interacting services. I want a tool that reads the API specifications for a group of services and documents the interactions between them. I'd like to look at service X and see that it emits message Y on queue Z, and then see which services read message Y on queue Z, jump to their documentation, etc. I think the AsyncAPI format is perfect to build on, though. I'll start there if I decide to take a hack at it.


I am not sure if I completely understand your use-case but it seems that the config can be distributed across multiple files.

https://www.asyncapi.com/docs/specifications/v2.0.0#referenc...


I'm taking a closer look today, and it looks to me like one AsyncAPI document, which can be defined using multiple files, defines one application. If you look at the "fixed fields" section under the root object, the second field is "id"[0]:

    id Identifier Identifier of the application the AsyncAPI document is defining.
This format is fine, but the tool I'm looking for is something that will read the definitions of multiple AsyncAPI documents (multiple applications) and show how their inputs and outputs connect, so I can answer a question like, "When application X publishes message Y on channel Z, which applications consume that message?"

AsyncAPI gets me 90% there by defining the service spec and providing code to parse it. It's possible somebody has already written the rest; I'll have to see.

[0] https://www.asyncapi.com/docs/specifications/v2.0.0#fixed-fi...


KSQL seems to address this gap.


Is this comment intentionally Kafkaesque?


What do you find nightmarish about it?


Kafka is very configurable .. and teams struggle to optimize the production deployment configuration and tend to second guess their decisions. Getting a managed deployment from Confluent etc. mitigates these issues as you can count on proven SMEs to deliver a well optimized solution. However, if you were to DIY , you might not be so blessed. Disclosure: I never did a DIY Kafka prod deploy -- my employer was already a Confluent customer.


I love this.


Delightful!


cute. ELI5, but with beavers.


While the otters are busy planning elaborate ways to tell each other about neat rocks, the much smarter humans are revving up their chainsaws to cut down the forest.


I'm not sure I'm following the analogy anymore.


How is this different from REST API? Serious question. Why get downvoted?


Because there’s no meaningful way to answer that here. It’s like asking what’s the difference between trawler fishing and open cast mining.


I don't understand your metaphor. But thanks.


Well, fishing and mining are both resource extraction but apart from that have nothing in common. Kafka and REST both have APIs but apart from that have nothing in common.


I'm sorry you're getting downvoted, but I think this question is legitimate because the book is peddling Kafka as if it's the only way to do event sourcing. Event sourcing is what you should compare with REST APIs, Kafka is one way of doing it, but you can do the same with any database, as long as you have a way to write things in and read things out and organize them, you can achieve event sourcing.

With REST APIs (first few pages of the book), services talk directly with each other, with event sourcing (the rest of the book) services talk with an event store (Kafka in the book) as the intermediary.


You're getting downvoted because your question makes no sense. They are not related at all, which means you must not have read the link.


I read the whole thing. That's why I have the question. So they used to have messages send to everyone (websocks?), now they have the messages persistent in different channels for people to consume. Isn't this just different entry points of a Rest API?


Why are you confusing something that could be used to implement the back-end of a REST API (Kafka) with the idea of a REST API?

Your question makes as much sense as: why is <new database X> any different from SQL?


I got my understanding from the illustrated book...

OK. So I didn't get what Kafka is.




Consider applying for YC's Summer 2025 batch! Applications are open till May 13

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: