We don't rebuild image every time. We store images in GitLab registry. Also, just because we use apt-get to install python3 doesn't mean build isn't reproducible. The actual toolchain and sysroot are highly version controlled and python is just used to kick off the build script so it almost doesn't really matter which version python is as long as it's backwards compatible with the build script.
We tried this in our research group, and found that issues come in when it gets to things like...
Oh no! CMake is too old a version to support a dependency we have to build in the image construction. So we better pull in a version of CMake from a PPA which is community maintained, and build it from source/etc.
