The odd thing is, we physicists actually have a pretty solid understanding of why magnets repel each other. It's a fun story, one that I try to include every time I teach the subject, and I'm pretty sure that Feynman knew it! So it's curious to me to see Feynman basically dodging this question. (His point that "I can't explain it in terms of anything else you're already familiar with" is entirely valid! But I kinda feel like that's what questions are for: learning new things, that we may not already be familiar with.)

With that in mind, I probably would have answered in a different way, though the questioner might get bored and regret asking if I didn't find a way to be unusually efficient about it: this answer relies on a lot of knowledge that seems at best tangentially related. :) I'd try to give some rough description of the way that moving currents generate magnetic fields (especially the field generated by a current loop), and the way that moving currents feel a force due to magnetic fields (especially the force on a current loop). That's enough to argue that current loops will interact with each other in just the same ways that magnets do. And then I could tell at least a sketch of the story of how regions of aligned spins in magnetic materials act on average like current loops. (But that's quite a long story to answer a simple question, so again, I can sympathize with Feynman for saying, "There isn't a straightforward answer that a non-expert would understand.")