I was asking OP there if he could provide me with a list of edge cases of formatted strings to help me test my parser implementation.
This alone pushed me probably weeks ahead...I think that's what I dislike the most about it: missing reference implementations of ISO standards that also contain testsuites. I mean, without that it's just yet another paper.
A standard is useless when it's not public. But it's even more useless when there's no unified testsuite to test your implementation against. Looking at you too, IETF and everything related to the Web.
Also - heh, I just thought of the idea that standards organizations should be required by law (or maybe all-but-law policy) to receive testcases, formatted to a reasonable* standard, execute them, and provide run reports.
You know, how SQLite3 runs the closed-source $$$$$ TH3 against every release.
But then I realized the current method is much cheaper (all things considered), and thus more efficient. :'(
(* Reasonable = a peer-reviewed agency (or some such) is given access to the specification and told to write a submission standard (specifying programming language(s), submission format, etc) and an associated automated implementation that receives submissions formatted according to the established standard; and then anybody else who also owns the spec is allowed to raise issues about the submission standard. Yup, the whole thing would cost millions of dollars.)
An example of a collaborative effort that achieves this is TC39's test262 [0], which is a full test suite for JavaScript conformance across engines and vendors.
Cryptography people tend to be good about providing test vectors, especially in the IETF CFRG. Unfortunately, it's not as a common a practice as one would like.
I was thinking more about the lack of testsuites for everything related to network protocols, such as DNS, HTTP or WebSockets and WebRTC.
The RFCs related to DNS are an endless list of deprecations that you have to "merge" in your head until you actually know what is allowed, what is deprecated, and what was extended. Especially with EDNS and all its options that are somewhat somewhere on the IANA website.
Before that, I realized that not a single server implementation implements HTTP's 206 Partial Content and/or Transfer-Encodings as specified; and lots of servers even reply with wrong buffer sizes when requesting multiple Content Ranges.
When reading through the Chromium and Firefox codebases, there's always dirty hacks that are implementation specific, so there isn't any end-to-end networked-only testsuite that verifies the network states and behaviours.
For my own Browser Stealth [1] I had to create a testsuite because I couldn't find anyone that's not related to known SSL attack vectors. Due to the peer-to-peer concept I decided to test network behaviours wherever possible.
The network protocols themselves (when speaking of RFCs) are just not tested, and freely interpreted at will - even in older projects like apache, caddy, curl, libaria and others. When reading the curl codebase you'll soon realize that it is a huge collection of hacks he had to implement just to make things work when the servers were behaving incompliant to the specifications.
This alone pushed me probably weeks ahead...I think that's what I dislike the most about it: missing reference implementations of ISO standards that also contain testsuites. I mean, without that it's just yet another paper.
A standard is useless when it's not public. But it's even more useless when there's no unified testsuite to test your implementation against. Looking at you too, IETF and everything related to the Web.