So does it eventually give up and allow bigger objects through? You’d think it would need a mechanism to handle undigestible items (as any dog owner knows)
It definitely does, and it's normal operation. We're supposed to eat lots of indigestible material; whole vegetables and grains contain a lot of indigestible fiber, which is necessary to support a healthy gut biome and efficient bowel operation. Corn husks, for example, are easy to see when they come out the other end.
So here again, how does your stomach "know" when to stop trying to digest certain pieces of food and let them pass on to the small intestine? This topic is called gastric emptying, and it's a whole other rabbit trail. Your stomach monitors the chemistry of the food in it-- specifically, the chemistry of the digestion products generated as the food breaks down. It knows lots of details about the carbohydrate, protein, and fat being released, and it knows how quickly its digestive enzymes and acids are being used up. So it knows how much more work needs to be done, and whether or not its worth trying to get more nutrients out of a piece of unchewed nut, for example.
There are many other variables, and I think digestion is still largely shrouded in mystery. Studies have shown variation in gastric emptying rates based on the biological activity of food, infections in gut tissues (even if they're subclinical, i.e., you don't feel unwell), time of day, and even the person's emotional state. The guts are the seat of a large part of the autonomous nervous system, and science has barely scratched the surface of it.