Hey hobofan, one of the founders here. You're very correct - we don't have lockfiles yet! It's on our roadmap, but we haven't built it yet because the current level of reproducibility has been enough for our users right now (who are mostly dev and test).
That said, if we're to accomplish our goal of being the single platform across dev, test, and prod, lockfiles are an absolute must (we've actually been contemplating whether we can leverage Nix or Bazel under the hood). The design puzzle we've been working on is, how to enable non-determinism and flexibility in dev (e.g. "I want to always use 'latest' on my local machine"), and complete rigidity in prod. I'd guess that we'll have fully reproducible builds available in the next 3 months :)
(other founder here) Yep, you got it! This is particularly relevant in the testing usecase where we started: "can we do whole-system E2E testing, so you catch more outliers before you hit Prod?"
Hey, one of the founders here! As Galen mentioned in the other threads, it wasn't our intention to borrow credibility or dilute the term. Rather we're big fans of Nassim Taleb's Black Swan theory, and to us founding a startup (and our first usecase in testing) both related to high kurtosis.
That said, I hear you that it's confusing when people need to distinguish between "kurtosis the statistical concept" and "Kurtosis the company"!
Strongly agree. There's a big knowledge gap between the container author and the container consumer that Docker doesn't have a great toolkit for bridging. The best we get are the ENV, EXPOSE, and VOLUME directives, but authors should really have tools to say:
- what filepaths are important (both for volume-mounting and for files output by the container)
- what envvars (or CMD, or ENTRYPOINT) are needed
- what ports will be opened, and what they mean
- what protocols the ports talk
...etc. I'd really like containers to one day behave like software libraries: you pass in a set of documented parameters, and get an object that you can pass around and use in return.
You might be interested in what we're building at Kurtosis: you define your app in a deterministic subset of Python, and then we do the magic to change only what's necessary on subsequent runs (what we call "idempotent runs"). It also supports parameterization.
Agreed. It's felt like so many of the technological marvels have either happened quietly (e.g. the reductions in chip size) or in the past (e.g. all the physics breakthroughs of the 1900s). Like ChatGPT, this seems to have the characteristics of being both accessible and completely novel.
What is interesting is that assuming it all pans out these are all in entirely unrelated fields. That sets the stage for a whole raft of follow on inventions. Similar to how the telephone + basic electromagnetism led to radio, tv, radar, transistors and ultimately to computers small enough to be practical for businesses, the internet, cell phones and so on.
My grandmother was born in a house without running water, no sanitation and no phone. That's just a bit over a century ago. The rate of change on an annual basis isn't all that large, a decade and you'll see big changes, our world can't be compared at all to 120 years ago in terms of luxury, communications, personal energy budget, food, travel options etc. Still, there are large areas of the planet where the last 120 didn't bring any progress and there are those where they actually went backwards, not rarely to our (the western world for me) benefit.
The last 20 are already mindblowing to me, to the point that I am a bit overwhelmed. Just reading to keep up with all of the tech developments is basically impossible today.
Yeah, recently I've remembered my computer from 1995 had about 1GB HDD. Now I can much more cheaply buy a SSD which will transfer 7GB/s. Mind boggling.
I paid 2500 $ for a 500 MB harddrive and $750 for 64 K RAM at some point. To me the world we live in is utter science fiction, and yet, every morning I seem to wake up and it is all still there.
In terms of basic tech, other than CRISPR what are you thinking of? Smartphones + internet are important, but are more combinatorial. Early quantum computers are cool, but niche (and even if perfected, seem to have application for breaking PKI). Reusable rockets are great, but not fusion. The standard model is locked in, at least up to LHC energies modulo sensationalist PR.
GPUs have been huge, unlocking so much more compute than was possible with CPUs. Most of the burgeoning AI industry is thanks to that cost decline, which is only getting started.
Every field of science has had breakthroughs in the last decade or two. Even superconductors (check out 'H2S', though it isn't without problems), medicine, genetics, quantum computers I'm not yet all that impressed with but they're too early stage to be really judged, the energy transition is really happening (sub-subject: the price of solar and wind power), semiconductors (every time we think it's over...), GPS (check out how it works under the hood), so much in materials science that I can't begin to keep up, phased arrays, lidar, the insane increase in computational power that you can stick under your desk, battery tech (one more breakthrough there and we will see all kinds of effects on other fields as well), solid state lasers the size of grains of sand, fiber optic transmission rates, the James Webb (ok, 'old tech' by now, but that's one impressive thing they pulled off there, especially the delta between the hot and the cold side of it) and on and on.
As for the future:
Fusion would be huge, but I'm not all that hopeful for cost efficiency there once you get to net power out but I'm not going to talk down the people that are doing the work and the research. And yes, the basic physics seems to be pretty stagnant, we're really waiting for a unification of the two major fields there but even if we do get that unification it may not lead to new practical tech, it could simply nail things down once and for all without moving the needle in terms of costs, speed or new materials science. It may have some implications for various computer models used in those fields and it probably would have impact on astronomy.
Im posting this reply from my mainstream smartphone: a pocket supercomputer with a 4K 60Hz HDR video camera. I’m chock full of designer vaccines and can watch a catalog of almost all the movies ever made on demand tonight. I listened to an podcast in my EV on the way to work. And SpaceX probably launched 60 satellites this week on the 14th flight of a rocket that flew home afterwards.
Things have been moving along.
edit: comments are quibbling about the fraction of all movies ever I can see on a paid streaming service. This is like complaining about that the meals and elbow room are not wonderful on a $400 flight from LA to London. It's a goddam miracle and you're still grumbling. Please tell me now how your $12/mo Spotify account doesn't have the June 1972 Grateful Dead New Jersey show your mom was at, and is thus near worthless. I spent $12 in 1980s money on single album in my youth.
Let me amend my grossly hyperbolic statement and say I could stream on demand more movies than I could ever watch even if I did nothing else the rest of my life, including many but not all of the good ones. Now the statement is strictly true, but did this make my contribution better?
Important correction… the catalog of movies is likely significantly smaller than all the movies ever made… the ongoing removal of new movies for business reasons and the market fragmentation mean that reduced profits are leading to corner cutting with respect to the size of stream catalogs, licenses cost money… pirating via torrents while larger also has diminishing returns as you step outside the mainstream movies, unless your lucky and find someone diligently sharing bandwidth to keep content online happens to share an interest in that niche… we’re rapidly losing access to an entire generation of analog only media as people throw source material into the trash and corporate stewardship of original recordings slowly fails one papercut at a time for the decades it will take for work to enter the public domain and be properly archived by the people trying to hold onto all of this.
We could have maglev trains and superconducting supercomputers and copyright will still be deleting our culture to “save money” for corporate copyright owners to increase their profit margins.
I pay a commercial service for pirated content and have access to over 50,000 movies alone. Then there are tens of thousands of tv shows. I can request shows. It gives me access to all major streaming services content. Anything I have ever searched for is there. Shows from around the world and all around feels like unlimited content. Cheaper then Netflix and I definitely feel I am getting great returns for my money. One day we will look up to pirates for saving our culture.
It took a few minutes sleuthing to validate that my gut reaction of "but there's way more than 50000 movies" was right ... But thanks to UNESCO statistics (http://data.uis.unesco.org/) I've got the sad truth.
Your 50,000 movies is a lot but its not even half of the movies made since 1995 that UNESCO were able to reliably cite data for. I can't deep link to the exact spreadsheet of data but it's under the culture data section and its feature film statistics. The total number of feature films (which will exclude some things like short films and other stuff based on various data processing considerations) produced around the world between 1995 and 2017 is at least (because there are likely more movies shot than produced) 107,432.
Assuming your "over 50,000" is the usually marketing line and being generous and assuming its somewhere between 50,000 and 55,000 movies, then if true that's approximately half the "feature films" produced between 1995 and 2017...
A a sensible lower bound extrapolation based on the data is in the range of half a million feature films (again recognising this data is likely excluding things we would on average collectively call a movie), making your service closer to 10%... or possibly even lower...
Over 50,000 movies feels like a lot until you dig into how much media we make as a species. That's just feature films, likely only ones with a theatrical release (I don't think I can reliably translate a lot of the source documents even if I wanted to validate the source data criteria myself, hence I'm using the weasel word "likely"), meaning its excluding a LOT of film/movies, and its only for 1995 to 2017, covering an era where the "amateur film" scene was rapidly exploding due to falling costs of the technology behind recording, editing, and distribution a "movie"...
None of this is to tear down the effort of hard working preservationists... both legal and illegal. I agree with the archivists I've had conversations with, whose collective opinion can be summed up quite simply. "Any copy is better than no copy."
It is a plex share I don’t really want to draw attention to the company sharing so don’t want to share the site directly but just go to reddit and search for plex share and start looking. Lots of different ones some paid some free. Some tailored to anime some tailored to other genre.
The last 10 (20?) years I noticed a notable shift in public consciousness about scientific progress and that we as a species are moving in an upward trajectory towards a brighter feature.
One of my favorite movies of all time is “Contact” mainly because of how damn _hopeful_ it portrays humanity. I want to live in that world, not the gloom and doom “we’re ruining everything around us” that is so often shown in recent media.
And the whole thing is perplexing to me because as you’ve put it, we _are_ living in the future! There is soo many great things happening around us, but people seem not to notice - not just the technological things you’ve mentioned - but societal too.
Like because of recent developments in economy theory, the world altering global shut down due to corona did not end with a great depression like event for the whole world, and that alone to me is simply a miracle.
Human longevity studies have now reproducibly able to reverse aging _in primates_ - human trials starting this year!
Feminism has become all but mainstream, unlocking like 50% more of human ingenuity.
Urbanism studies have finally popularized environments where people can live happily and sustainably their entire lives, while there have always existed places in the world that are “nice” to live in one way or another, we’re kinda getting the science down why, and starting to popularize it a little (strongtowns).
Its just a crazy good time to be alive, there are countless problems all around us but they are actually getting solved at least somewhere, instead of banging our collective heads against the wall, one can just look for how it has been successful handled somewhere else and try to replicate!
You are just repeating the same incredibly boring 'Tesla fanboy' things that Joe Rogan likes to say every now and then to create the illusion he's smart, up to date and edgy, which is the exact reason you don't sound smart at all.
I feel we will eventually outpace our own means of destruction. The next big step is getting self-sustaining colonies off Earth, which will do wonders for our ability to survive as a species. The biggest threat is perhaps nuclear weapons and climate change, in my opinion.
I actually feel like the biggest threat we face is a collapse of world order and another world war, even conventional coupled with a degradation of humanism. These lead to a global decay of basically everything, including our ability to outpace our own means of destruction and an entire inability to address climate change. The global environment becoming increasing toxic to our age’s life, including humans is I think ultimately the end of our line.
>can watch a catalog of almost all the movies ever made on demand tonight.
You must have one hell of a different streaming services experience from that of most people today if you can pull this off without engaging in a fair bit of torrenting and general piracy.
No you just decide what movie you want to watch and pay $4 to rent it on demand. Very common experience available to anyone with a smart TV or connected streaming device.
Yeah, regular consumer streaming might cover "movies still popular enough but not so new that they're only in theaters." However "all the movies ever made" is a significantly higher bar.
I've had this experience with Ally bank. Their SMS-only 2FA, lack of international wire transfers, and compete failure to work with Plaid drive me insane, but I'm still using them because I get a kind, helpful human every time I call.
Except when it is not. With Ally customer service - I was once on hold for more than hour before I could speak to an agent. Not the ideal scenario - and their backend systems were not even able to resolve my issue even after multiple calls.
Agreed, and we're actually trying to bridge this gap at our company Kurtosis! Docker is great for doing dev - fine-grained control over the container, strong predictability guarantees, fast, and relatively lightweight (in comparison with Kubernetes), but you can't run Prod on it unless you start rebuilding Kubernetes primitives like Services from scratch.
Our approach has been "build an abstraction over Docker and Kubernetes that can provide the predictability of Docker when you need it (Dev & Test), and the scalability and resilience of k8s when you need it (Prod)", and then handle compiling down to actual Docker & Kubernetes commands. The goal is for the user to have a single environment definition all the way from Dev to Prod, so you don't have to do the dance of "Docker Compose for Dev, Bash on top of Compose for test, and Helm for Prod"
Then you realize later that it's all architecture, free of advertising. Sadly, it's not completely clean - on the way from GRU there's still a decent amount of high-standing billboards and generic crap.
That said, if we're to accomplish our goal of being the single platform across dev, test, and prod, lockfiles are an absolute must (we've actually been contemplating whether we can leverage Nix or Bazel under the hood). The design puzzle we've been working on is, how to enable non-determinism and flexibility in dev (e.g. "I want to always use 'latest' on my local machine"), and complete rigidity in prod. I'd guess that we'll have fully reproducible builds available in the next 3 months :)