Yes but when the table is so small even if the index is slower it hardly matters because the table is so small.

Or in other words add the index every single time, unless you have so many writes and such a large table that the actual writes slow down the index.

Leaving out an index on a small table never helps anything.