If you use Guice you aren't using a singleton anymore. Guice is creating an instance and passing it to multiple classes as a dependency. This is different from any class just referencing a Singleton by importing it statically.

It's a slightly subtle distinction, and you still have the risks associated with non-local mutable objects, but at least you can track (using the Annotation API) which classes are using the shared state; it's better than letting any line of code anywhere in the middle of any method sneakily and suddenly grab access to shared state.