Really? I don't see any difference, except I do see a pixel shift if I switch between uncompressed/AVIF.
This could happen due to DC shifts (one pixel gets quantized, the rest are coded as a difference from it), or a mistake in the post-decoding pipeline (gamma/colorspace correction).
I just modified my comment to make it more clear what I meant.
But you're right, probably just error due to compression/quantization etc. in DC shift or in colorspace conversion. It seems to happen in low quality JPEG too after I did some quick tests myself.
This could happen due to DC shifts (one pixel gets quantized, the rest are coded as a difference from it), or a mistake in the post-decoding pipeline (gamma/colorspace correction).