> One important lesson I've learned in my career is never to run your own observability platform if you can afford for someone else (whose entire product is observability) do it for you.
1. You have to keep your logging, monitoring, alerting infrastructure up yourself
2. You have to monitor, log, and alert on your that infrastructure yourself with something else
3. You usually have to spend more money, both in hosting and in developer/ops time, on getting something mediocre compared to a provider that exclusively does observability as a product
4. Logging etc are a commodity and you should have a really good reason to build or run something yourself if it is not your core competency
5. Observability is much harder than it sounds and providing a cohesive platform that ties log events to traces to metrics to alerts, and keeping it up and highly available is a hard problem that you shouldn't do if observability isn't your core competency/product
Yep, if you can afford it make it someone else's problem. No team should need to have dedicated OPs people that are responsible for keeping the logging platform running. This is a solved problem and costs the company more in engineering time (paying a person to do the job) than the Saas solution.
E.g. 30K/month isn't much when you compare it to paying a sole engineer 15K/year + cloud costs of hosting your own logging solution + any licenses. There's also a slim chance the home grown logging platform will be as resilient as a Saas product.
I for one am very grateful to DataDog's support team as they've been very helpful in debugging logging problems we've had in the past.
A think a more generic advice would be to focus on what brings value to your customers and makes your product better.
> I thought using loops was cheating, so I programmed my own using samples. I then thought using samples was cheating, so I recorded real drums. I then thought that programming it was cheating, so I learned to play drums for real. I then thought using bought drums was cheating, so I learned to make my own. I then thought using premade skins was cheating, so I killed a goat and skinned it. I then thought that that was cheating too, so I grew my own goat from a baby goat. I also think that is cheating, but I’m not sure where to go from here. I haven’t made any music lately, what with the goat farming and all.
Why is that?