The swedish study discussed here is what I was remembering: https://www.health.harvard.edu/mens-health/never-too-late-ex...
They note that people who began exercising late in life didn't get much benefit to mortality risk by the 5 year benefit, but by the tenth year their risk was the same as the group that had always been active.
Edit: actually you need to google it to get to the un-paywalled article.
“Most importantly, men who were highly physically active at age 50 were 32% less likely to die during the study than those who were least active; moderately active men enjoyed a smaller, but still respectable, 13% lower death rate than the least active gents.”
“But the long-term nature of the Swedish study allowed the scientists to follow men who were sedentary at age 50 but who increased their exercise level between ages 50 and 60. For the first five years, the major result was disappointment, since these men continued to die at the same high rate as men who remained inactive. But over the next five years, the benefit kicked in; by 10 years of follow-up, the men who adopted exercise in middle age enjoyed the same low mortality rate as men who began before age 50. All in all, men who adopted exercise after 50 had a 49% lower death rate than the men who remained inactive, a benefit even greater than the 40% risk reduction experienced by men who quit smoking after age 50. And the protective effect of exercise remained significant even after the scientists adjusted their results for the impact of smoking, drinking, obesity, diabetes, cholesterol, blood pressure, and socioeconomic status.”
“According to the Swedish study, men were classified as moderate exercisers if they simply took frequent walks or often went cycling for pleasure. And high-level exercise involved a minimum of just three hours of serious gardening or recreational sports a week. And in case you're tempted to cook up another excuse, you'll soon see that this important study does not stand alone.”