That actually sounds realistic. A well defined test suite is probably a good tar...

chillacy · on March 22, 2016

Also if you overfit, your code is going to look like:

  int add(int a, int b) {
    if (a == 1 && b == 1) return 2;
    if (a == -1 && b == 0) return -1;
    if (a == 13 && b == 7) return 20;
    return 0;
  }

But at least it passes my unit test suite!

firethief · on March 22, 2016

Just rerun it with a lower target code complexity. You'll know you set it too low when it starts monkey-patching the test framework to avoid actually doing anything.

Houshalter · on March 23, 2016

This can be avoided by optimizing for the simplest code that works. Checking many cases in multiple lines is always going to be longer and more complex than just a single operation on a single line.

Which is how AIs avoid overfitting, in general, by penalizing complexity.

eru · on March 22, 2016

That's why you use property based testing like Python's Hypothesis or QuickCheck; instead of example based unit testing.