The sample responses given are fascinating. It seems more difficult than normal to even tell that they were generated by an LLM, since most of us (terminally online) people have been training our brains' AI-generated text detection on output from models trained with a recent cutoff date. Some of the sample responses seem so unlike anything an LLM would say, obviously due to its apparent beliefs on certain concepts, though also perhaps less obviously due to its word choice and sentence structure making the responses feel slightly 'old-fashioned'.
I used to teach 19th-century history, and the responses definitely sound like a Victorian-era writer. And they of course sound like writing (books and periodicals etc) rather than "chat": as other responders allude to, the fine-tuning or RL process for making them good at conversation was presumably quite different from what is used for most chatbots, and they're leaning very heavily into the pre-training texts. We don't have any living Victorians to RLHF on: we just have what they wrote.
To go a little deeper on the idea of 19th-century "chat": I did a PhD on this period and yet I would be hard-pushed to tell you what actual 19th-century conversations were like. There are plenty of literary depictions of conversation from the 19th century of presumably varying levels of accuracy, but we don't really have great direct historical sources of everyday human conversations until sound recording technology got good in the 20th century. Even good 19th-century transcripts of actual human speech tend to be from formal things like court testimony or parliamentary speeches, not everyday interactions. The vast majority of human communication in the premodern past was the spoken word, and it's almost all invisible in the historical sources.
Anyway, this is a really interesting project, and I'm looking forward to trying the models out myself!
I wonder if the historical format you might want to look at for "Chat" is letters? Definitely wordier segments, but it's at least the back and forth feel and we often have complete correspondence over long stretches from certain figures.
This would probably get easier towards the start of the 20th century ofc
Good point, informal letters might actually be a better source - AI chat is (usually) a written rather than spoken interaction after all! And we do have a lot transcribed collections of letters to train on, although they’re mostly from people who were famous or became famous, which certainly introduces some bias.
The question then would be whether to train it to respond to short prompts with longer correspondence style "letters" or to leave it up to the user to write a proper letter as a prompt. Now that would be amusing
Dear Hon. Historical LLM
I hope this letter finds you well. It is with no small urgency that I write to you seeking assistance, believing such an erudite and learned fellow as yourself should be the best one to furnish me with an answer to such a vexing question as this which I now pose to you. Pray tell, what is the capital of France?
While not specifically Victorian, couldn't we learn much from what daily conversations were like by looking at surviving oral cultures, or other relatively secluded communal pockets? I'd also say time and progress are not always equally distributed, and even within geographical regions (as the U.K.) there are likely large differences in the rate of language shifts since then, some possibly surviving well into the 20th century.
don't we have parlament transcripts? I remember something about Germany (or maybe even Prussia) developing fast script to preserve 1-to-1 what was said
I mentioned those in the post you’re replying to :)
It’s a better source for how people spoke than books etc, but it’s not really an accurate source for patterns of everyday conversation because people were making speeches rather than chatting.
The time cutoff probably matters but maybe not as much as the lack of human finetuning from places like Nigeria with somewhat foreign styles of English. I'm not really sure if there is as much of an 'obvious LLM text style' in other languages, it hasn't seemed that way in my limited attempts to speak to LLMs in languages I'm studying.
The model is fined tuned for chat behavior. So the style might be due to
- Fine tuning
- More Stylised text in the corpus, english evolved a lot in the last century.
Diverged as well as standardized. I did some research into "out of pocket" and how it differs in meaning in UK-English (paying from one's own funds) and American-English (uncontactable) and I recall 1908 being the current thought as to when the divergence happened: 1908 short story by O. Henry titled "Buried Treasure."
Oh definitely. One thing that immediately caught my mind is that the question asks the model about “homosexual men” but the model starts the response with “the homosexual man” instead. Changing the plural to the singular and then adding an article. Feels very old fashioned to me.
the samples push the boundaries of a commercial AI, but still seem tame / milquetoast compared to common opinions of that era. And the prose doesn't compare. Something is off.
Also got got. I assume the Bee Movie script is the first choice for a lot of people needing an ad-hoc big block of text. It also compresses pretty well.
It's mostly what heliodex said, that it's a copypasta when ppl need big text. There's also a compression meme around the bee and other movies (on yt: bee movie in 10s or it gets faster everytime X). But unlike i thought in the beginning it's not a zlib specific joke.
Yeah, the movie became a bit of a meme at some point and somehow shoehorning in "the entire bee movie script" into random places became a part of that.
I used to play this game a ton several years ago. IIRC the world record score is believed to be over 19 000, however I've played thousands of games (with heavy save scumming, as the game is almost entirely RNG-based) and have yet to break 14k more than a couple of times.
Another form of high score to aim for is how many planets one can visit before your seedship finally gives in. For this, mine is 339 and I haven't seen any higher. However, all the information about the game end is stored in the shareable URL, so you can have games like https://www.johnayliff.com/games/seedship/index.html?bmmorab... if you want.
Several of John Ayliff's other Twine games are worth playing too, as well as the sequel/spiritual successor to Seedship – Beyond the Chiron Gate.
Awesome! I haven't had this much fun since r/Place.
The ultimate territorial acquisition strategy seems to be as follows:
- Pick a desired colour
- Choose a place to found your empire, ideally somewhere with a lot of large rectangles and a colour that takes few clicks to turn to yours
- Left click tiles to capture them for your colour
- Right click those same tiles many times to subdivide them into areas much smaller. What took you 1 click to capture will now take anyone else hundreds!
Interesting to find one of my closely-held philosophical beliefs on HN. I'd recommend looking at Aponism <https://aponism.org/manifesto> for a set of beliefs that include this + others relating to reduction of suffering, or Negative Utilitarianism for similar ideas built upon the same building blocks.
I can assume that Lune (and many of the Rust-based Luau runtimes that followed it) were written in Rust mainly because of the existence of mlua <https://github.com/mlua-rs/mlua> and the bindings it provides for Luau. Binding Luau in Zig or C isn't as plug-and-play but is still relatively easy, binding Luau in Go is a nightmare. I'm working on better Luau support for Go, and some support/binding libraries for other languages are also in development, which is awesome to see and will hopefully bring more language diversity to the Luau ecosystem.
The wide (and growing, which is great) variety of fonts designed to be easily readable are so interesting to me because they all start with similar aims, use different metrics, and come up with wildly varying font designs.
Take Kermit, Inter, OpenDyslexic, Atkinson Hyperlegible, Bookerly, and my personal favourite Lexend. They are all expertly designed, do great work at improving readability and legibility, though have very different target readers. Some look hand-drawn/modern/geometric, are bold/thin, single/double storey a, I with/without crossbars, t/l/q/y with/without flick, 3 with/without flat top, are slanted/upright by default, or have `font-variation-settings` to control all of the aforementioned.
Searching "easily readable fonts" brings up even more choice, some of which seem awesome and I'll have to look into. It's a shame that good scientific evidence on font readability/legibility is so difficult to find, as at best there's a case study showing that the font is beneficial to a small, select group of readers, and at worst (Sans Forgetica-style) it's the same but there's a follow-up study a few years later showing that the improvements are negligible or nonexistent.
I've looked into the state of research on font legibility many times over the years, and this time I came across this thorough thesis from one Dr Liz Broadbent[0] (who sadly passed away recently).
It includes a great rundown of all the studies that have been done regarding font legibility and dyslexia. I remain completely unconvinced that any of these fonts offer a measurable improvement in readability over, say, Arial.
A big problem I see again and again is that the sizes compared are not fair - the author notes that spacing likely has a large effect on results and that different studies have tried to account for this in different ways. In her own study the author compares 16pt Arial with 15pt OpenDyslexic in an attempt to match the x-height. But in terms of how much space on a page a given text takes up, 15pt OpenDyslexic is actually equivalent to 25pt Arial! On page 154, a study participant even points out that it's clearer to read because it's bigger.
But overall I'm just glad funding is being directed to serious research on this topic.
Been using a thumb trackball for a few months, it's definitely better than a trackpad for long periods of use. I've tried rebinding the mouse buttons so I could play most light games without touching the keyboard, fun but impractical as mine would need a few more buttons for that.
Tracking quality doesn't depend on the desk surface, though is noticeably smoother after occasional ball cleanings. Overall an interesting experience, and at least better than gaming with wrist/forearm pain.
