> but there really isn't a library that tries to do > just 2D with some filtering+scaling

While not a 2D library, https://github.com/bkaradzic/bgfx is a very sweet abstraction layer over platform graphics APIs