I wonder how a transformer (even an existing LLM architecture) would do if it was trained purely on chess moves - no language at all. The limited vocabulary would also be fantastic for training time, as the network would be inherently smaller.
I vastly prefer it to Docker, especially buildah over buildx. Instead of inventing yet-another-dsl buildah allows you to simply use shell scripts (though it does also support dockerfiles). Another thing buildah is really good at is not doing much automatically: you can really optimize layers if you care to.
The Podman ecosystem has given me a strong disliking of the Docker ecosystem, so I'm also rooting for it.
I think I might be the only one that prefers Docker for building Docker containers using CI.
I use Drone, but instead of using the Docker plugin I start a detached (background) Caddy server to work as a proxy to DOCKER_HOST. That lets me proxy to the local Docker socket to take advantage of caching, etc. while I'm iterating, but gives the option of spinning up docker-in-docker to get a clean environment, without any caching, and running a slower build that virtually identical to what happens on the CI server.
I find that having the daemon available solves a ton of issues that most of the CI provided builder plugins have. For example, with the builder plugins I'd always end up with a step like build-and-tag-and-push which didn't work very well for me. Now I can run discreet build steps like build, test, tag, push and it feels far more intuitive, at least to me.
Sadly modern idiomatic C# is a bit of a nightmare these days. It's a heavily abstracted/"indirected" style, where you are basically forced to use a debugger to determine where a method call actually goes. The goal seems to be to make things as emergent as possible, with reflection and IL emit determining code flow as much as possible.
I think that F# goes a long way to prevent this, by nature of being functional.
The default C# style has always preferred a large dose of unnecessary runtime flexibility. Sometimes that comes in the form of lots of dynamic dispatch, sometimes it's DI, sometimes it's non-sealed overly large base-class types, and sometimes its some combination thereof - but the net effect is that it's not possible to reason about code by just reading it. Tooling becomes worse and/or less valuable because tooling too cannot reliably trace all possible code paths.
Additionally, Microsoft has an outsized influence on the C# ecosystem. As a result, they wrote quite a few architectural and style guidelines that, naturally, target themselves. It makes sense for a hyper-widely distributed library to consider things like binary compatibility as critical features, but the design rules that follow from that have zero utility for apps, and low utility for most libraries. The C# ecosystem also has an unhealthy veneration for MS provided code. While I'm sure most of the authors are quite competent, there's less experimentation and fewer hidden gems because the ecosystem is tilted towards being followers behind the framework providers path.
As a result, it's much harder to refactor code in that style than code in a less (runtime) flexible style, and it's worse when the libraries heavily used are frameworks that have so many extension points. There's a tradeoff between runtime flexibility and codebase flexibility, and the default .NET C# style heavily leans towards runtime flexibility, which isn't a good tradeoff for most software.
MediatR is part of the problem, yes. Way too much attribute/reflection magic there.
One particularly good example is: attempt to add full support (i.e. query text generation) for PERCENTILE_CONT to EF. That will give a good idea of just how bad the idiomatic C# style can get. Now take the EF style of code/architecture and apply it to an entire product codebase - which is completely a understandable mistake to be made (it's not unreasonable to assume that EF would be a good example of C# code). You know you have a problem when you throw in the towel and have ThingDependencies classes all over the place. The aspnetcore internals don't vary too much from the abomination that is EF.
"Just don't code like that," you might say. The problem with that argument is that pre-existing code exists, people don't code in isolation, and the cargo cult has a ton of momentum behind it.
> your comment about F# being different.
You'll struggle to write that style of code in a functional language. There is such as thing as healthy friction. F# has its own problems, but this is not one of them.
One of the reasons I use Java is for type safety. What's the point when your app compiles OK but then doesn't start because Spring can't satisfy all the dependencies or choose which one. I spend hours trying to get the config right. You end up programming via config.
Right, if references were meant to be pointers then they would simply be called pointers.
Some abstractions are meant to be treated as a black box. It's bloody trivial to realize and point out that references are represented as pointers internally. The actually intelligent step is to figure out how to use that knowledge correctly. Do we pass small bittable types around by-value because we skip the dereference? Hell yes (pending profiling). Do we treat references as pointers? No.
As a plausible scenario, rustc could use tagged pointers for some purpose in refs. Your code could seem to be working in tests, but segfault when rustc sets the MSB for some reason.
You have to be pretty careful when using syscalls directly, at least in the presence of some libc. For example, from what I have gathered from random tidbits here and there, it's not a good idea to use any of the fork-like calls (fork, clone, etc.) if you have any threads that were created using glibc.
The trough of disillusionment is going to be frightening, and I believe we're starting to see the first signs of it (with Apple's paper, ironically enough).
~2 years into the LLM revolution, not a single productivity metric has significantly gone up in any country while we collectively spent hundred of billions on it.
What really happened: LLMs are fuelling the phishing and scam industry like never before, search engine results are shittier than ever, every other website is an automated LLM generated seo placeholder, every other image is ai generated, students are heavily using these techs for homeworks with unknown long term effects, scientists are using them to write their papers: quality--, quantity++, bots on social media increased dramatically and probably playing a huge role in social tensions. You can probably add a few dozen things in the list
It's unfair to dismiss and simplify this technology when it only just started reaching critical mass ~2 years ago. Looking at history, transformative technologies rarely show immediate productivity gains. The internet took nearly a decade from the early 1990s before we saw clear economic impact. Electricity needed 30 years (1890s-1920s) to significantly transform industrial productivity, as factories had to completely redesign their operations. Personal computers were around since the late 1970s, but their productivity impact wasn't measurable until the late 1980s/early 1990s. Major technological shifts require time for society to develop optimal use cases, build supporting infrastructure, and adapt workflows
Agreed, but is this technology promise too much to bet on a single company? Are we being sold R&D dreams or tangible products?
Compare this with firms that had an actual working product (ex: Lightbulb, Internet, Windows, iPhone) and not the mimics of one (ChatGPT, Bitcoin, Metaverse, Nanotechnology, Quantum).
PS: There is just too much money in the economy now (infused from covid), so chasing speculative investments is expected.
Those "actual" working products had little use until infrastructure was built around it to make it useful. The same could definitely be happening to LLMs, we'll just have to wait and see. It is just way too early to claim that they're a mimic of a product.
Not really. A lot of people are happy to take in a metric ton of cash from what is very questionable hype. Nvidia certainly doesn’t mind everyone overselling AI, and neither do CEOs drooling at the thought of AI replacing workers. That’s the reason behind all this insanity.
So your point is:
1. The LLM revolution had no significant positive impact so far
2. A dozen areas (phishing, web search, web content, images, education, science, social media) were completely changed by LLMs for the worst
Your point 2 seems to indicate that the potential for impact is huge. People also think of positive impacts, not only negative ones, that's why they invest.
> People also think of positive impacts, not only negative ones, that's why they invest.
They invest because greedy mfers don't want to miss the next apple or google. They'll throw millions at the walls and see what sticks, remember juicero and theranos ?
Remember the Metaverse? We are talking about entities that can burn billions on speculative projects and it is just a rounding error on quarterly profits. Let alone that frequently the outlay is in-part recovered via the various "sell shovels in a gold rush" strategies.
This is what total stranglehold of a critical sector helps with. Inefficiency of capital allocation at interplanetary scale - but that's another story.
LLM's might be new but the ML/AI type of efficiency/productivity argument is decades old. The domains where it leads to "gains" are generally when the org deploying algos manages not to be held responsible for the externalities unleashed.
Oligopolistic / Competitive vs Short-termist / Long-term are rather orthogonal dimensions and derive from different types of regulations and incentives.
reply