Huh, I was thinking that it's accelerated gravitational frames that cause the dilation, and I've encountered a lot of statements that argue the same. This is from wikipedia: "This is because gravitational time dilation is manifested in accelerated frames of reference or, by virtue of the equivalence principle, in the gravitational field of massive objects."
However, according to that logic, an object located in a cavity in the center of earth should experience no more dilation than an object outside the earth's potential well, because the gravitational forces / curvature gradient cancels out, and should be zero. But that isn't the case according to the same sources, for example, Wikipedia says' "Relative to Earth's age in billions of years, Earth's core is in effect 2.5 years younger than its surface."
Something's not right about how we verbalize this story about gravity
It's a very subtle point! The trick here is that you need to be careful when talking about the reference frames.
To an observer at the infinity, a clock at the core of the Earth will tick slower than a clock on the surface of the Earth because the "core clock" is sitting in a more curved space, and that's it.
The difference between the clock on the surface of the Earth and the clock at the core is that the surface clock can't follow the "straight lines" (geodesics) in that curved space. So it experiences acceleration due to the force of inertia. And the thing preventing that movement is the repulsive force between atoms that make up the bulk of the Earth.
If this repulsive force magically disappears, then the Earth's atoms will immediately start moving at the straight lines, in trajectories that will lead them all into a point at the center of the Earth.
To add: the force of inertia due to moving in curved lines instead of geodesics depends on the "steepness" of the curved space. Which decreases as you reach the center of the Earth. So you get essentially the same result as with the classic Newtonian gravity, but through an entirely different path.
Think about gravitational redshift. This is a direct sign that time is running more slowly for the emitter that is at a deeper gravitational potential.
