> We used the “percentage-rejected” metric minus the false negatives to ballpark the $1.7 million total annual savings.
I think this may too sanguine for the false negatives in that it ignore latency sensitivity. Generally, batch processing (like preview generation during pre-warming) is cheaper than latency sensitive processing (like preview generation when the user is waiting for it). If you don’t take that into account, you can be misled by your cost metrics.
Hi there. Fortunately for the ML team, the Previews team kept a detailed cost breakdown for different preview types - including on the fly generation cost, async generation cost, and cost to serve a preview. Our ballpark accounted for some of the varying costs, though the difference is not significant.
I think this may too sanguine for the false negatives in that it ignore latency sensitivity. Generally, batch processing (like preview generation during pre-warming) is cheaper than latency sensitive processing (like preview generation when the user is waiting for it). If you don’t take that into account, you can be misled by your cost metrics.