Yeah "modules that mostly work, usually, mostly" is no joke. And it's not that anyone really could do anything significantly better than ffmpeg, it is probably one of the best examples besides the kernel of an astonishingly complex library that is run extremely competently but the task space is just maddeningly complex and full of wild edge-cases. Especially when dealing with "legacy" formats and codecs from the 90s and early 00s - even today, have fun transcoding RealVideo or ASF files with some weird color space that nobody has used in 20 years.
There are infamously a lot of tasks that "look easy upfront but have a lot of underlying complexity" and ffmpeg is not one of them. It's a task that looks hard upfront and then when you get down into the weeds it's an eye-bleeding eldritch monstrosity.
The consensus around MP4 and MKV containers and some relatively small number of codecs (H264, H265, VP1, AV1, etc) has substantially eased things but video codecs, audio codecs, and container files are just complicated af.
And believe it or not - this is still way, way better than it used to be! Flaskmpeg was never stable, Virtualdub was nice but development ended with avi container files, there were some forks that added various features, etc, but ffmpeg and OBS Studio both actually just work incredibly well compared to what came before.
except not everyone has decided to use that easing of the things. it seems like they go out of their way to make things difficult
My favorite is in how MP4s are not all the same. A video captured on a DSLR or other acquisition camera source might have +/- in the video/audio sync when imported into an NLE. Yes, the container utilizes PTS syncing to playback the video/audio streams in sync, but this +/- slop in sync is annoying AF to professionals capable of seeing the sync slop. Average consumers probably never notice. I thank all the gawds in all the heavens that I no longer have to edit MP4 source captured content and all of the manual slipping in the NLE.
If you want a single tool/library that does all the things ffmpeg does, I agree. ffmpeg does lots of things, some of them are inherently extremely complex, and it puts them together cohesively. That's impressive! I think the closest competition is gstreamer, and gstreamer ends up calling into ffmpeg for software H.264 decoding...
But...sometimes you want one particular thing, that thing is relatively simple (compared to e.g. H.264 decoding), and you have needs ffmpeg doesn't meet. Then you probably can do better. A couple examples that have come up for me:
* specialized .mp4 muxing. My NVR project does this in a way where it can do HTTP range serving, so put the moov up front, produce the length quickly, and produce a given byte range without iterating through all the bytes before. ffmpeg can't do that, understandably. Hard to do in a general-purpose library; the container muxing input interface doesn't expect all the necessary information is available up front. But it's not that bad as a one-off. As you say, the consensus around a small number of containers really helps.
* RTSP. Consistent with your "modules that mostly work, usually, mostly", ffmpeg has various long-standing bugs in its RTSP handling (e.g. https://trac.ffmpeg.org/ticket/5018), has the risk associated with protocol handling in a non-memory-safe language, doesn't have much of a test framework for this, etc. This problem is relatively approachable (though hardly trivial), so I had success making my own library.
There are infamously a lot of tasks that "look easy upfront but have a lot of underlying complexity" and ffmpeg is not one of them. It's a task that looks hard upfront and then when you get down into the weeds it's an eye-bleeding eldritch monstrosity.
The consensus around MP4 and MKV containers and some relatively small number of codecs (H264, H265, VP1, AV1, etc) has substantially eased things but video codecs, audio codecs, and container files are just complicated af.
And believe it or not - this is still way, way better than it used to be! Flaskmpeg was never stable, Virtualdub was nice but development ended with avi container files, there were some forks that added various features, etc, but ffmpeg and OBS Studio both actually just work incredibly well compared to what came before.