The tokenizer covers the entire dataset. It's basically just a fixed-size Huffman code, grouping together common fragments of letters- for instance, the 100 most common English words are probably all single tokens.
During learning, the model proceeds in roughly the same way a child would: it starts by grouping tokens together, learning the deep regularities of language such as "news[paper]" being more likely than "news[q77.bfe]". Then it incrementally assembles these fragments into larger and larger chains. Similarly , it first learns thematic groupings, such as "word" being more likely somewhere after "dictionary" rather than "stop what I was reading to get the dictionary out every time I encountered a banana assault hungry". Then it starts to pick up "patterns": "as a [baby|child|kid] I had no [idea|concept|clue]". At some point in this process it naturally abstracts concepts from languages: "as a child" starts being internally represented by the same neurons as "als ich ein Kind war".
Then some magic happens that we don't understand, and out pops a neural network that you can talk to and that can write programs and use tools. To be clear, this is the case before RL: probably these patterns are now widespread in the training data, so that the model already understands how to "complete the pattern" on its own. RL then does some magic on top of that to bring it from 20% benchmarks to 80% and presto, AI assistant.
>Claude sometimes thinks in a conceptual space that is shared between languages, suggesting it has a kind of universal “language of thought.” We show this by translating simple sentences into multiple languages and tracing the overlap in how Claude processes them.
I think concept here means it is assigned a point or an area in the many-dimensional embedding space. The "concept" has no obvious form, but similar words, synonyms or words from another languages meaning roughly the same are very close together in this space.
That makes me think of https://store.steampowered.com/app/2262930/Bombe/ which is a version of Minesweeper where instead of clicking on squares you define (parametric!) rules that propagate information around the board automatically. Your own rules skip all the easy parts for you. As a result, every challenge you get is by definition a problem that you've never considered before. It's fun, but also exhausting.
I remember listening to a talk about Candy Crush and how they designed the game to have a few easy levels in between the hard ones, to balance feeling like you're improving while also challenging players. If all the levels get progressively harder, then a lot of people lose motivation to keep playing.
No but it's also important to realize that this CPU was running at an insane temperature that should never happen in normal operation. I have a laptop with an undersized fan and if I max out all my cores with full load, I barely cross 80. 100 is mental. It doesn't matter if the manufacturer set the peak temperature wrong, a computer whose cpu reaches 100 degrees celsius is simply built incorrectly.
If nothing else, it very clearly indicates that you can boost your performance significantly by sorting out your cooling because your cpu will be stuck permanently emergency throttling.
I somehow doubt that, are you looking at the same temperature? I haven't seen a laptop that would have thermal stop under 95 for a long time and any gaming laptop will run at 95 under load for package temps.
i7 8550u. Google confirms it stabilizes at 80-85C.
That said, there's a difference between a laptop cpu turbo boosting to 90 for a few minutes and a desktop cpu, which are usually cooler anyway, running at 100 sustained for three hours.
Yeah it is, if courts decided otherwise then courts decided wrong, the GDPR is very clear and explicit about this:
> Consent is presumed not to be freely given if [...] the performance of a contract, including the provision of a service, is dependent on the consent despite such consent not being necessary for such performance.
That kind of demonstrates why XSLT is a bad idea as well though. JSON has its corner cases, but mostly the standard is done. If you want to manipulate it, you write code to do so.
JSON correlates to XML rather than XSLT. As far as I'm aware, XML as a standard is already done as well.
XSLT is more related to frontend frameworks like react. Where XML and JSON are ways of representing state, XSLT and react (or similar) are ways of defining how that state is converted to HTML meant for human consumption.
US-type capitalism is a high-level concept though. The right of an individual to exist - existence in effect being its own justification - is such a lower level concept, that something so lofty as capitalism can’t really have any bearing on the issue.
It’s not a question of whether capitalism allows our existence - it’s very obviously the other way around.
The U.S capitalist system / democracy does not try to guarantee the existence of its members ? That's a dark take. It could do a much better job but I think there's at least a minimal, some would say sufficient, effort made to support the existence of members of society.
There is a vast spectrum between US-style neoliberal-late-capitalism and the DPRK, maybe both extremes just need to die to stop this kind of stupid thought-terminating cliché for once.
If you follow this logic, it gets really close to arguing that open source is price dumping. Nobody has an obligation to an income in a particular field of work.
Open Source is the conscious renunciation of any profit and intellectual property rights, whereas artists generally expect compensation for their work.
Plus, open source code gets mostly written by people with a stable and well-paid day job. Take that away, and I doubt there’s much open source code left.
So it's not just price dumping, it's also gatekeeping against unemployed developers. What a terrible practice.
Open source only looks better here because it has a different cultural context. But to a person trying to make money in the field, I'm not sure why that should matter.
As someone who leans more towards the side of LLM-sceptiscism, I find Sonnet 4 quite useful for generating tests, provided I describe in enough detail how I want the tests to be structured and which cases should be tested. There's a lot of boilerplate code in tests and IMO because of that many developers make the mistake of DRYing out their test code so much that you can barely understand what is being tested anymore. With LLM test generation, I feel that this is no longer necessary.
Isn’t tests supposed to be premises (ensure initial state is correct), compute (run the code), and assertions (verify the result state and output). If your test code is complex, most of it should be moved into harness and helpers functions. Writing more complex code isn’t particularly useful.
