Can't this whole thing be boiled down to "DevOps for Data Science/Engineering"?
Different parts of the org with different skillsets and cultures practicing empathy for each other by communicating interests in version-controlled code, allowing for guard-railed autonomy, which leads to business agility.
Yep. Sounds about right.
> Optimize for autonomy not efficiency
Optimizing for efficiency without considering the cost of work in progress (WIP) (irrelevant ETL models), rework (unscalable models), or unplanned work (unscalable models that make it to production) results in company silos (data engineering, infrastructure engineering) cheering local maxima while covering their ass in the face of a business that's suffering from a long lead time. Two teams with two backlogs will accomplish work exponentially faster compared to three teams with three backlogs.
It boggles my mind how books like The Phoenix Project are not required reading.
Different parts of the org with different skillsets and cultures practicing empathy for each other by communicating interests in version-controlled code, allowing for guard-railed autonomy, which leads to business agility.
Yep. Sounds about right.
> Optimize for autonomy not efficiency
Optimizing for efficiency without considering the cost of work in progress (WIP) (irrelevant ETL models), rework (unscalable models), or unplanned work (unscalable models that make it to production) results in company silos (data engineering, infrastructure engineering) cheering local maxima while covering their ass in the face of a business that's suffering from a long lead time. Two teams with two backlogs will accomplish work exponentially faster compared to three teams with three backlogs.
It boggles my mind how books like The Phoenix Project are not required reading.