Looks like the LOC folks are also unaware of the existence of HyperCard clones such as SuperCard (a compiler) and subsequently Runtime Revolution (now discontinued): before the company stopped selling its end-product cross-platform HyperCard clone there was a brief open source foray, and some folks appear to still be trying to develop an open source XTalk platform (XTalk is the generalized descendant language of HyperTalk):
... They also don't seem to know about how files on an HFS filesystem all have separate resource/data forks: it's a distinctly different paradigm from the familiar a-file-is-a-stream-of-bytes convention of Linux/UNIX/NTFS, and some of their problem seems to arise because they're looking at a "flattened" HFS file that has been squished to fit on a non-HFS filesystem. (Modern macOS makes almost no use of resource forks, so they're vestigial in HFS+; not sure APFS even provides them.)
The other post linked from the one you're commenting on (https://blogs.loc.gov/thesignal/2020/11/an-archivists-perspe...) talks specifically about the problems of preserving and reverse-engineering dual-fork files, so the problem is not what they're unaware of, Charlie. You just aren't aware of the problem they're trying to solve.
The LOC, the Internet Archive, and the UK National Archives don't benefit from proprietary compilers for proprietary operating systems like SuperCard; their remit is to preserve these files for centuries or millennia, not until the next update of macOS X. If SuperCard interprets a HyperCard stack differently than original HyperCard did, or SuperCard on a new version of macOS X displays or behaves differently than it did on the previous version, that's a failure of digital preservation. Unless, that is, you notice --- and to notice, you need to emulate the original HyperCard environment and/or be sufficiently familiar with the semantics of the file specification. If https://news.ycombinator.com/item?id=29555959 is to be believed, SuperCard is 32-bit-only and so will not run on the most recent version of macOS X, which lack 32-bit support, which if true demonstrates the enormity of the problem.
As for streams of bytes, a disk is a stream of bytes, or rather a collection of fixed-size blocks of bytes. HFS is looking at flattened HFS files that have been "squished" to fit on a medium that does not, itself, implement HFS. Therefore anything that you can represent with an HFS dual-fork file can also be represented losslessly in a stream of bytes. (You do of course benefit from understanding the dual-fork structure.) HFS+ actually went further, implementing files with an arbitrary number of forks, like sections in ELF or PE. As GeekyBear points out, NTFS did the same, leading to path traversal vulnerabilities in early Microsoft web servers.
Wouldn’t an effective (and seemingly easier) digital preservation approach be to virtualize an older Mac OS (System 7 even) and just run HyperCard on it?
Yes, that's one of the things they're doing, but in the screenshots they're using MacOS 9. It's less effective if you want to do things like full-text search, copy and paste, or render the vector graphics (though maybe those were MacDraw and not HyperCard?) at a different pixel resolution than the emulated Macintosh supported, or with antialiasing. There may also be metadata or "forensic" data that isn't visible directly in HyperCard itself, which may be of interest to future historians.
Aside from the emulation/undocumented-format problem that the librarians here are suffering from, a really big problem with born-digital archival work for more recent formats is copyright. Although libraries are permitted to make copies for archival work, in many cases the software tools they would need to make those copies don't exist, because those tools would circumvent an effective technological protection measure within the meaning of the DMCA, so writing and distributing them is potentially a crime. So in effect copyright law is privileging the business interests of Harvey Weinstein, Dan Brown, Bob Guccione, and Adam Sandler over the academic interests of scholars, historians, and the Library of Congress.
I don't think there's anything wrong with Pee-wee Herman or Amy Poehler making movies, but I don't think it's worth sacrificing the intellectual heritage of humanity to buy them second vacation homes.
> render the vector graphics (though maybe those were MacDraw and not HyperCard?) at a different pixel resolution than the emulated Macintosh supported, or with antialiasing
HyperCard graphics were bitmapped, not vectors. There's no additional resolution available.
> There may also be metadata or "forensic" data that isn't visible directly in HyperCard itself, which may be of interest to future historians.
This, on the other hand, is interesting (and potentially a little scary). The HyperCard stack format is practically a tiny little file system, complete with unallocated space; it's not uncommon for FREE chunks to contain bits of data that happened to be in memory on a computer that interacted with the stack.
Funny you should mention the DMCA. The Library of Congress is uniquely empowered by the DMCA to grant exceptions to it. So if there are any cases where the law is deemed to be a barrier to this effort then the LOC can grant themselves and their colleagues an exemption.
The Librarian of Congress has the opportunity to grant such exemptions in a rulemaking once every three years, which then expire if they are not renewed, which makes them a risky basis on which to base a product; probably the LoC would have to develop the software themselves, which isn't really a strong competency of theirs, instead of relying on outside help.
It isn't the kind of thing where a summer intern can ask permission to break a DRM scheme for archival purposes and then be successfully preserving material the next week.
Yes, and honestly that's going to be the only effective way of preserving some stacks. In particular, some stacks included XCMDs/XCFN resources, which contained native 68k code and could interact with the OS in arbitrary, unrestricted ways.
The original version of Windows NT Server allowed you to install Services for Macintosh and act as an AppleTalk file server, so support for resource forks was added to NTFS.
Alternate Data Streams are pretty interesting, cause I'm pretty sure Services for Macintosh weren't the only users - it's also way more powerful (closest thing on Unix would be Solaris-style extended attributes) and pretty fundamental part of NTFS on-disk structures (in fact, "alternate" data streams differ from main one only by having a name - the default data stream simply has no name, and all data streams are equal in importance)
Extended attributes aren't limited just to Solaris. A variety of *nix file systems have generally supported forks/multiple data streams for some time. FreeBSD has support for arbitrary-sized extended attributes in UFS/ZFS, for example, and implements its access control with them. Linux probably has the most limited form, where the xattr are capped at 4 or 64 kilobytes depending on the underlying FS.
I think the big issue for uptake is that it just doesn't fit into the UNIX world. Nothing else is aware of xattrs/forks. The basic file utilities will blindly strip them so you can't just cp or tar such a file, for example. If cp transparently breaks all your files, that's pretty much a show-stopper as anyone who remembers the classic Mac to OS X transition will know.
The reason I mentioned Solaris is because Solaris behaviour is closest to NTFS, whereas linux API is based upon Irix, which is closer in behaviour to OS/2 (OS/2 subsystem was another use case for ADS), as well as implementation on a bunch of other systems where extended attributes were treated as just that, attributes.
Comparatively, NTFS is very... flexible navigational database and a bunch of things are just "this is the default parameter set, not exhaustive".
I wasn't aware that FreeBSD copied Solaris in this (ZFS extended attributes by default create a sort-of directory object allocated linked from the file, linux supports inlining simplified xattrs for speed)
There is a difference between alternate data streams and extended attributes. Basically, the former is data, the latter is metadata. NTFS and ZFS implement alternate data streams, but in eg FreeBSD those are accessed like extended attributes.
NTFS doesn't really have a difference internally - they are all attributes of the FILE object (everything in NTFS is a FILE or its attribute, including filesystem metadata). Arguably ZFS ZPL has "blessed" data element (I dare you to implement ZFS while replacing DMU/SPA with Ceph or S3 XD - yes it's doable)
The HyperCard stack's resource fork contains icons, sounds, xcmds, xfcns, and probably some other miscellanea. Most of the stack is stored in the data fork.
APFS doesn't specifically have resource forks. They're handled as the com.apple.ResourceFork extended attribute (which is also one way OS X exposes them on HFS+ but with HFS+ the fork/xattr always exists; on APFS it has to be explicitly created).
It seems like it would be reasonably straightforward to create a hypercard clone on top of js and the DOM. Or maybe that's already been done? (a lazy search did not reveal this to be the case).
I generally hate the expression "Ahead of its time", but I think that it definitely applies to Hypercard.
I've only played a bit with Hypercard, but it's one of those things that feels almost anachronistic; it's hard to believe that something released in 1987 would be as trivial to use and be productive with as Hypercard.
It's a language that is object oriented without having to teach the programmers any of the theory. The language has a considerable amount of syntactic sugar but makes for some pretty incredible looking code.
on mouseUp
put "100,100" into pos
repeat with x = 1 to the number of card buttons
set the location of card button x to pos
add 15 to item 1 of pos
end repeat
end mouseUp
Although I hate to recommend proprietary software, you might be interested in looking at Inform 7, designed for authoring interactive fiction; https://rosettacode.org/wiki/FizzBuzz#Inform_7 has an example:
The space is a room. An item is a kind of thing. In the space are 100 items.
To say the name:
let the count be the number of items carried by the player;
say "[if the count is the count to the nearest 15]fizzbuzz.[otherwise if the count is the count to the nearest 3]fizz.[otherwise if the count is the count to the nearest 5]buzz.[otherwise][the count in words].".
To count:
if an item is in the space
begin;
let the next one be a random item in the space; silently try taking the next one;
say "[the name]" in sentence case;
count;
end the story;
end if.
When play begins: count. Use no scoring.
Inform6 is almost as simple if you are a programmer. I mean, is not "real life language alike" but the IF6 library makes the easiest platform to write a text game as you can set the structure of the game by design, with "literal" objects.
I worked on interactive software for learning Japanese in Hypercard many years ago. I think it's still the most productive system for building user interfaces I've encountered.
HyperCard and Excel were my programming gateway drugs.
HyperCard was a mix of like PowerPoint and HTML / JavaScript. Just was so much fun for a kid to play around with. And everything you built had a visual element you could show your friends.
I used HyperCard to re-create a MUD I really liked with a visual interface. Sort of like Myst... I had a blast. Certainly wasn't the first person to do that, but all the little coding tricks I had to learn to pull off my vision all came in handy later.
> HyperCard and Excel were my programming gateway drugs.
Same! I had played with C++ a bit, but what really made programming logic stick for me was when I made my teacher a gradebook in Excel in 9th grade. His logic for calculating the grades in class was a bit strange, and he was doing it all manually, and my grades were exceptionally bad in that class, so I asked if I could get some extra credit if I could make his grading easier. Surprisingly he said yes [1], so that weekend I really crunched through Excel and pumped out a gradebook that implemented all the rules he was using to grade.
It was the first time that "if" and "else" statements really clicked for me, and it got me interested in "real" languages.
[1] In hindsight he really shouldn't have. It was sort of an unfair bit of favoritism he was paying to me, but 15 year old me didn't really care.
the book “what the dormouse said” is an amazing retelling of this era detailing who tripped when and what they went on to invent
IIRC photolithography and CPU architecture came out of this subculture, berkeley and harvard both had sanctioned LSD experimentation that grad students were happy to participate in, in the “center for advanced studies”
it’s been a few years since I read it so my memory is hazy
Your comment made me go search for a short article that I'd seen in the past [0] with a text version at [1]. "Use of LSD-25 for Computer Programming"
"While stimulated by the LSD I was able to get the entire system wholly in my mind at the same time. I spent some time mentally visualizing various aspects of the compiler, the language and the processing which would take place. I did discover three or four design inconsistencies while being stimulated by the effect of the LSD, and I made notes for later checking."
just spit-balling here as a non-lsd-user, but I do have the experience of “writers block” when I feel a program I wrote is too complex, I feel like it would be better to scrap it and start over
perhaps the effect of LSD is more anti-anxiety of approaching the problem despite its complexity, not improving your IQ, but lowering your inhibitions that otherwise prevent you from problem solving
(I am on the side of things that drugs in general turn parts of the brain off, rather than somehow making you superhuman, but maybe there are parts of the brain we could do without from time to time)
I've said the same thing about smoking sativas, so to learn that others have capitalized on this sort of 'increased visualization capacity' of LSD is not surprising.
> (I am on the side of things that drugs in general turn parts of the brain off, rather than somehow making you superhuman, but maybe there are parts of the brain we could do without from time to time)
They do neither as a rule. They just twist thinking around as they mess with your brains chemistry. Sometimes that puts you in a better place for a given activity, sometimes it puts you in a worse place. I think that, as society wakes up to the lies promulgated by the War on Drugs, we will revisit these positive interactions that are otherwise collectively forgotten.
Agree on the sativa. Back when you had to use a code name if you mentioned it on the phone, I and my friends at Tymshare, Apple, General Magic, Grid, Tandem, etc. called it "programmer's fuel".
Many here are familiar with the term "fire an event". I coined that in the late 1980s when I was at Cooper Software working on a program that would become Visual Basic. (Our team did the "Visual" part, and Microsoft added the "Basic".)
Our system was event-driven just like pretty much any GUI app or system. Under the hood, of course, we used the Win16 SendMessage, PostMessage, GetMessage, and PeekMessage calls.
But we had a habit of using creative names. In fact Alan Cooper once said that "one of my most important programming tools is a thesaurus."
We had already decided to call them "events" instead of "messages" for whatever reason, and now I needed a way to talk about sending them.
At my previous job we had "SQL triggers". I started thinking along that line but wasn't getting anywhere.
So I turned to a trick I often used when I felt stuck: I started firing rubber bands against my computer screen. I kept a stack of them on my desk for just that purpose.
(This was an IBM Monochrome Display with a sturdy glass front. Do not try this on your modern LCD/OLED displays.)
I still wasn't getting anywhere, so I fired up a doobie. I flicked the lighter a few extra times and looked at the flame.
And then it all fell into place: Fire a rubber band. Fire up a doobie. Fire an event!
Wow, what a cool story. I use that term on the regular, what an interesting history!
I must also confess I haven't felt this way about sativas in a long time. Not sure what happened but sometimes it makes me kind of sad to not feel that big brain effect. Been meaning to microdose LSD but it's hard finding a reliable connect
On another note, a young tweenage me wrote his first program on my uncle's copy of VB3. I wanted to write a terminal-type app that could dial a BBS and interact with it. I got about as far as getting the modem to dial.... I'm sure there's an angry BBS operator out there somewhere who got sick of this kid dialing his number and then never initiating the handshake. So... thank you for helping to create the tools I used to do that!
Oh my gosh. Thank you for sharing that. It means so much to me when I hear that my code helped someone build a business, have fun in a hobby, or even get in a little trouble. :-)
hah! somehow I never heard that one, and googling it points me to a 13 year old reddit thread discussing a book that’s new to me, “The Unix-Hater’s Handbook”, looks like a great read, so, Thanks!
The question is how many of the LSD takers invented something in relation to all the other ones who just had some fun. Is it bigger percentage than in the general population?
Well the last dude I know who took LSD tried to call an ambulance for a pile of sacks of builders cement and pissed himself. YMMV on that one. There are probably more failures that are not romanticised and more successes that involved hard work.
Wow, it has been a while since I play with Hypercard. I remember doing this in my 7th grade science class back in the 90s. I and my classmates have to create something out of Hypercard for the project.
Not relevant to the article, but while people are talking about HyperCard and derivatives generally, it inspired HyperNeWS, which I remember fondly, though it wasn't free software. Perhaps that kibozes Don Hopkins: http://www.art.net/studios/Hackers/Hopkins/Don/hyperlook/ind...
Usenet is still alive on some tech newsgroups, at least on the English speaking side. On the Spanish hierarchy (es.*), here there is nothing more than a ghost town.
For instance, a stack file begins with a stack block (signified by the hexadecimal values, “53 54 41 4B,” which correspond to the ASCII characters, “STAK”), and the file is terminated by a tail block (signified by the ASCII characters, “TAIL”
https://openxtalk.org/forum/
https://en.wikipedia.org/wiki/XTalk
... They also don't seem to know about how files on an HFS filesystem all have separate resource/data forks: it's a distinctly different paradigm from the familiar a-file-is-a-stream-of-bytes convention of Linux/UNIX/NTFS, and some of their problem seems to arise because they're looking at a "flattened" HFS file that has been squished to fit on a non-HFS filesystem. (Modern macOS makes almost no use of resource forks, so they're vestigial in HFS+; not sure APFS even provides them.)