your granularity upper bound being the framerate is a result of LÖVE not passing SDL timestamps as well as it doing SDL_pumpevents once per frame, so even if you try to timestamp the individual events they are still bunched up into the start-of-frame rather than accurately being able to do sub-frame rollbacks.

To be fair, I think the way SDL hand wave the processing into one lumped Pumpevent is where the blame lies —- it is just a compromise made for the sake of being a cross-platform abstraction so they had to make it a black-box invocation