Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

You mentioned 'level of quality'. That can be pretty difficult to quantify. In the absence of well defined metrics, a lot of thinking about QA is qualitative.

"I feel like we have a high level of quality here".

That makes conversations about QA pretty challenging. Metrics is the one area I'd LOVE to see move forward.



Heh, maybe that's because the nature of quality is qualitative rather than quantitative.

There are things that can be measured that might be correlated to quality ("bugs", sales, support, etc) but ultimately, classical quality seems to be to be a marriage of what was expected or desired with what was actually there. As a developer, my take is that quality usually stems from "developing" those expectations as much as making the relatively concrete thing to compare them too. When anyone can quantify the first half of this equation, please let me know at once.


That's a really good question, and it requires some more context.

First, you have to understand your company from a systemic viewpoint. Break it down, and understand the inputs and outputs of every part of your company. You've got your individuals, your various teams, the people they work with, the systems they build, and the product itself. You can measure the input and output of each of these systems in meaningful ways: number of tickets opened, number of defects reported, raw performance metrics, uptime, availability, quantitative user feedback like NPS, and engagement metrics like product usage or feature usage.

But there are caveats: you can't turn these metrics into goals. You need to lead toward the improvement of the systems responsible for those metrics, with the hypothesis of improving them and ultimately the end goal of providing a high quality product of value to your customers. The metrics are valuable information about how the system operates, not the end goal.

There's another problem: you're absolutely right about quality being fundamentally qualitative. It is, after all, the root of the word "qualitative." That's significant. Quality is the humanity of the engineering game: while you can measure a lot of things toward it, ultimately, as W. Edwards Deming said, "The most important things cannot be measured." It sucks, but it is closer to reality than defining any metric or set of metrics. Instead, it requires leadership and a comfort with doubt and complexity. Here's my answer to a Quora question expressing discomfort with this paradox: http://qr.ae/f5iUg

So it's a bit of a Bermuda triangle. First, there are significant metrics you can measure, and you need to decide which ones mean quality to you and your customers—I promise they exist. Second, you can't aim directly for your metrics, since any large enough organization is highly complex and over-optimizing for a single metric or set of metrics can cause significant unintended side-effects. Third, the most important things critical to quality actually can't be measured; or measuring them would cause more harm than good. This makes for a challenge, and it's why this is such a difficult problem to solve in an organization.

What I'd LOVE to see moving forward is instead a profound comfort with complexity, and an understanding of why numeric metrics aren't always the most vital ingredient to success.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: