Emulator development has cross-sections across many diverse fields of computer science, and I'm trying to share my experiences with all of them. If anyone here has suggestions for future articles for my site, I'm all ears ^-^
While folks are thinking about CDs: I've not been able to get an F2/F3-frame CD image. If anyone has such a thing (not copyrighted, just a CD containing a text file or public domain music piece is fine), I'd love to get it so I can implement a CIRC encoder/decoder! My GitHub repository (linked in the article) has an RSPC Reed-Solomon encoder/decoder, a disc scrambler/descrambler, and Q-subchannel encoder/decoder. I'm trying to build up a comprehensive ISC-licensed C++ library for working with every frame level of CD data.
For folks interested in the lead-in / lead-out data, to my knowledge only very few drives allow reading this raw data, namely older Plextor drives with the 0xd8 (READ_CDDA) command. If there are any hardware engineers who want to save an entire generation of CD history in a more exacting archival format, we could desperately use either hacked drive firmware for a modern CD drive, or custom CD reading hardware to preserve this data. Time is really of the essence.
The vanguard of CD preservation right now, to the best of my knowledge, is Claunia's DiscImageChef project: http://discimagechef.claunia.com/
I'm also working with the Game Preservation Society in Japan ( https://www.gamepres.org/ ), which has archives of thousands of rare and obscure Japanese CD media, so by all means if anyone reading this can help, please let me know.
> While folks are thinking about CDs: I've not been able to get an F2/F3-frame CD image. If anyone has such a thing (not copyrighted, just a CD containing a text file or public domain music piece is fine), I'd love to get it so I can implement a CIRC encoder/decoder!
I don't know if you've already seen this, but I've heard of someone getting an image like this. They used it to discover what the AccurateRip drive offset values mean in terms of the physical location of the subchannel data and interleaved main channel data on the disc. There's a vague description of their process in there as well.
> Using my neat hardware setup I now have captured a precious 0.1 seconds worth of 588-bit channel data frames from a pristine audio CD, each frame containing, among other things, a subchannel byte and 24 bytes of audio data.
> I can reproduce the CIRC, get the de-interleaved audio data, and also do the Reed-Solomon C1/C2 stuff in software from my analog laser readouts - showing an error-free read, identical to what I get with my Plextor using somewhat more conventional methods.
After having watched "Dissecting the CD Player: How to Turn Shiny Plastic into Music" by Technology Connections https://www.youtube.com/watch?v=3yJqlD9RxD4 I went on a CDROM research binge. Found this on my way https://debugmo.de/2007/07/read-your-dvds-the-raw-way/ Its about DVDs (has data dump plus code), but you can use same mechanism to easily modify CDROM drive using $4 piece of hardware (ebay "EZ-USB fx2lp cy7c68013a board").
I saw something online where a scanned high-resolution image of a record could be played by simulating the record needle. Is it possible to scan or photograph a CD at such a high resolution that the pits become visible? Then you have all 2.5GB of the raw EFM right there. If the CD is scratched the lacquer layer could be removed somehow I imagine.
I heard rumors that the original PlayStation copy protection involved introducing an intentional wobble into the tracks who's frequency is reported back to the application by the CD-ROM controller. Full preservation may necessitate your scheme.
And indeed there's really not much point in doing so. Any disc copy protection scheme trying to mess with CIRC codes, or worse, EFM codes, would have a really hard time having any drives read the resulting discs.
Some copy protections did play around with EFM, leading to "weak sectors". Amusingly enough, when I tried to search for info on it for this post, I found an article previously linked from HN, and which quoted me...
Having helped to design some early CD-ROM drives many years ago (initially using Philips CDROM components), the information here brings back memories. :-)
On a hardware level, the first thing we do is do a eye pattern check to see if all the modulated signals are coming in as expected [1]. If not, a hardware check on components is performed.
I've long been sad that no one has yet built a state of the art decoder for the recovery of error riddled CDDA discs.
A lot of old CDs are not holding up as well as advertised, resulting in an actual loss of archived information.
The RS decoders used in typical CD players doesn't come anywhere near the theoretical performance possible from the format, they're only used as erasure codes. In particular, list decoding with a prior on the audio continuity (using techniques like the Postfish declipper) should do really well because of the the enormous and channel independent interleave used by CDDA... but I've never seen it detecting. 30 years ago that sort of thing would have been computationally infeasible, but it should be
possible now.
> The RS decoders used in typical CD players doesn't come anywhere near the theoretical performance possible from the format, they're only used as erasure codes.
Reed-Solomon is fascinating, especially the way both RSPC and CIRC have both P and Q parity. Depending on the order and how many passes you do of each, you can recover more and more data. If you can guess where the error is, you can correct up to twice as many errors. And as a last resort for RSPC, you can potentially brute force corrections via the checksum values.
Reed-Solomon error correction ends up being more of an art than a science, and it's why different CD-ROM drives have different success rates at reading damaged discs.
Explaining Reed-Solomon would be a fun article, but probably too intense and niche a subject. Tons of linear algebra over galois fields, and intimidating if not overly complex algorithms like Berlekamp-Massey, Chien search, and Forney's formula (some of which can be brute forced these days.)
RSPC makes it even more complex with two overlapping channels of parity. CIRC (cross-interleave) makes that even more complex still with differing delay slots per byte of error correcting data. See pages 45 and 46: http://www.ecma-international.org/publications/files/ECMA-ST...
Small clarification: "state of the art" just means "the best currently available", not "a perfect and effective solution that handles all the problems and has all the possbile functionality". So if no one has ever built a decoder of a certain quality level, that specific level can by definition not be "state of the art".
Sadly even pressed CDs have been known to rot[0][1]. Albeit it's not I tend to hear about with audio CDs because (presumably because there are usually other ways to acquire copies) however it is something that worries a lot of retro gamers like myself. It means that many of the Mega CD, Saturn, etc games (for example) have a potential life expectency.
For me though, scratched discs affect me more than disc rot. It can be a lottery sometimes when buying retro games because you can't always be sure just how well looked after the disc was (even when you have pictures and the item is marked as tested/working).
I feel we're already reaching a point where game preservation for older systems like Mega CD and PC Engine CD really need to have multiple rips that then weigh all the rips to try and correct for individual errors in each rip.
> even when you have pictures and the item is marked as tested/working
I bought a PlayStation 2 second hand a while ago to relive some memories and to play some of the games that I hadn’t yet played.
One of the games played perfectly well up until about 60% through the story and then there is a bad scratch on the disc at which point the game freezes.
I think most sellers of used games will only pop the disc in and see that it starts, if they even bother to test the game at all. That is reasonable of course — we can’t expect anyone to play through a whole game another time over just to see that the whole thing works as it should before they sell it for a couple of dollars. But yeah it’s a risk and it does suck a little bit.
Had a similar problem when buying a used copy of Metal Gear Solid 1 on the PS1, the game and both discs worked fine but if you died on the second disc while in story mode the game would lock up. Added a bit of extra difficulty at any rate!
If you're buying a used game you can be sure it isn't tested and is sold as is. Even if they say it's tested or guaranteed, that only means they're willing to give you a refund if anything is wrong.
I recently ripped (using a blueray drive) a half-dozen CD-R's I burned 18-20 years ago. All had been tucked away in a Logic case; all ripped fine ... except one, which had several visible mechanical dings (3 of 6 tracks lost).
Better than I'd hoped. I've had a handfull of newer CD-Rs go unreadable, for invisible reasons.
I don't believe this is an exhaustive list, Plextor had a lot of good models back in the day. You'll want to look at older, true Plextor models, all SCSI or ATAPI. Once SATA rolled around they'd started rebadging drives from other manufacturers that do not have the same capabilities (or quality)
Depending on the model you're looking at maybe $50-$150 on eBay, which is pricy for a used old CD-ROM drive but not a lot of money for someone who really needs one. The Premium/Premium2 models in particular are sought after by audio professionals and (used to) get quite pricy.
$50 isn't that pricey. Similar to the cost of a new BD-ROM drive. I was imagining a lot more from TFA. What's a good way to hook up an ATAPI drive to a modern computer? I'm curious if mine still works.
Hopefully not very. I used to work at a CD duplication place and we had 100's of them in giant robotic duplicator towers. They were the drive of choice for exactly these sorts of capabilities.
When we closed down I pitched them all into the dumpster because at the time I couldn't even get $5 each for them on ebay. It would hurt to discover that they are now a rare and expensive find!
I was briefly excited to see this in the article but it looks like these are $10-$30 on ebay, still not worth putting them in a box and mailing them around.
But there are also 3 "merge bits" between each EFM code, so this shouldn't matter?
edit: The problem solved by scrambler is that number of pits and lands must be equal on average, or else drive couldn't distinguish pit and lands as dark/light decision signal level wouldn't be longer average of output signal. There is excellent description here: https://archive.org/download/CDCrackingUncoveredProtectionAg..., page 33 in PDF under "Sync Groups, Merging Bits, and DSV" header)
Correct. And since the scrambling XOR pattern is known, it's possible to burn a disc that ends up being all zeroes or ones post-scrambling anyway. I'm told such sectors tend to be unreadable, but I haven't tried it.
While folks are thinking about CDs: I've not been able to get an F2/F3-frame CD image. If anyone has such a thing (not copyrighted, just a CD containing a text file or public domain music piece is fine), I'd love to get it so I can implement a CIRC encoder/decoder! My GitHub repository (linked in the article) has an RSPC Reed-Solomon encoder/decoder, a disc scrambler/descrambler, and Q-subchannel encoder/decoder. I'm trying to build up a comprehensive ISC-licensed C++ library for working with every frame level of CD data.
For folks interested in the lead-in / lead-out data, to my knowledge only very few drives allow reading this raw data, namely older Plextor drives with the 0xd8 (READ_CDDA) command. If there are any hardware engineers who want to save an entire generation of CD history in a more exacting archival format, we could desperately use either hacked drive firmware for a modern CD drive, or custom CD reading hardware to preserve this data. Time is really of the essence.
The vanguard of CD preservation right now, to the best of my knowledge, is Claunia's DiscImageChef project: http://discimagechef.claunia.com/
I'm also working with the Game Preservation Society in Japan ( https://www.gamepres.org/ ), which has archives of thousands of rare and obscure Japanese CD media, so by all means if anyone reading this can help, please let me know.