Atlanta 96 was a great case study in this. Atlanta built a ton of infrastructure that continues to serve the city well. Many of the venues built in downtown and especially on the Georgia Tech campus went on to be very productive and useful. Georgia Tech has a world-class gym and still uses some of the olympic village as dorms as part of that legacy. There is one big exception - a massive tennis center built out in the suburbs. It was too far outside the city, never got much use and was demolished in 2018. https://en.wikipedia.org/wiki/Stone_Mountain_Tennis_Center