I've worked in tech all my life, and long ago learned how important it is to be impeccably diligent in documenting build processes whenever creating, deploying or adjusting new architecture.
Now it's simply become part of my engineering hygiene - as natural and effortless as brushing my teeth.
Actually drilling your DR is also crucial. If you never put it to the test, your documentation isn't worth the paper it's printed on.
In fact the last few years I've been thinking about ways for these systems to rebuild themselves on a continuous basis. Eg. I'd love a smartphone that competely restores itself from backup every night, even to brand new identical hardware, including secure element artifacts (either via private keys I securely control or reregistering everything in an automated fashion), with no user-noticeable impact.
Now it's simply become part of my engineering hygiene - as natural and effortless as brushing my teeth.
Actually drilling your DR is also crucial. If you never put it to the test, your documentation isn't worth the paper it's printed on.
In fact the last few years I've been thinking about ways for these systems to rebuild themselves on a continuous basis. Eg. I'd love a smartphone that competely restores itself from backup every night, even to brand new identical hardware, including secure element artifacts (either via private keys I securely control or reregistering everything in an automated fashion), with no user-noticeable impact.