How about most widespread? Once while trying to debug a CPAN module I figured out that if $condition was false then Perl had a bug causing
my $foo = $bar if $condition;
to leave $foo with whatever value it had on the previous function call. (The exact behavior is more complex than that, but that's a good first approximation.) I then made the mistake of reporting this in a way that made it clear that
my $foo if 0;
was a static variable. Cue years of people like me trying to get the bug fixed and other people trying to keep it around. In the meantime EVERY large Perl codebase that I've looked in has had this idiom somewhere and it has caused hard to notice and reproduce bugs.
How about worst damage to a system? Due to a typo I once caused my employer to send the Bloomberg's ftp system every large file that it had ever sent. Since it sent a large file every day, this crashed their ftp server, meaning that a number of feed providers for Bloomberg didn't have their feeds update that day. I implemented no less than 3 fixes that day, any one of which would keep the same mistake from causing a problem in the future.
How about most (initially) bizarre? Debugging a webpage that a co-worker produced where, depending on the browser window size, you couldn't type into the form. The bug turned out to be a div that had been made invisible but left on top of the page. At the wrong window size it was in front of the form, so you couldn't click on the form elements behind it. (I figured this out by first recreating it with static HTML, then using binary search to figure out what parts of the page I could whack and still reproduce it until I had a minimal example. Then it was trivial to solve.)
That second one reminds me of this: An ISP called planet internet changed their homepage, only to find that they reliably crashed Explorer (3 at the time).
Took a while before the phone rang if I wanted to have a look.
It turned out they had a little animated gif in there with the inter-frame interval set to 0, causing a divide by 0 in Explorer.
That gif was pretty much the last suspect on the list.
Divide & conquer until you are simply staring at the solution and still you don't see it...
How about most widespread? Once while trying to debug a CPAN module I figured out that if $condition was false then Perl had a bug causing
my $foo = $bar if $condition;
to leave $foo with whatever value it had on the previous function call. (The exact behavior is more complex than that, but that's a good first approximation.) I then made the mistake of reporting this in a way that made it clear that
my $foo if 0;
was a static variable. Cue years of people like me trying to get the bug fixed and other people trying to keep it around. In the meantime EVERY large Perl codebase that I've looked in has had this idiom somewhere and it has caused hard to notice and reproduce bugs.
How about worst damage to a system? Due to a typo I once caused my employer to send the Bloomberg's ftp system every large file that it had ever sent. Since it sent a large file every day, this crashed their ftp server, meaning that a number of feed providers for Bloomberg didn't have their feeds update that day. I implemented no less than 3 fixes that day, any one of which would keep the same mistake from causing a problem in the future.
How about most (initially) bizarre? Debugging a webpage that a co-worker produced where, depending on the browser window size, you couldn't type into the form. The bug turned out to be a div that had been made invisible but left on top of the page. At the wrong window size it was in front of the form, so you couldn't click on the form elements behind it. (I figured this out by first recreating it with static HTML, then using binary search to figure out what parts of the page I could whack and still reproduce it until I had a minimal example. Then it was trivial to solve.)