Econometrics allows us to understand and measure these things. This result isn’t surprising at all. Treating employees well has a similar effect in improving productivity.

If they made it clear when skills were being used / monitored that, it'd seem to mitigate a lot of the problem.

It is shown in the chat log.

Shown after the fact

Depends if the job requires a lot of information and the person is excellent at what they do, bc then AI augments the worker more than substitutes them.

But for many people, yes, AI will mostly substitute their labor (and take their job, produce operating margin for the company).

100% (am a Bayesian statistician).

Isn’t it fascinating how it comes down to quality of judgement (and the descriptions thereof)?

We need an LMArena rated by experts.

As a statistician, do you you think you could, given access to the data, identify the subset of LMArena users that are experts?

Yes, for sure! I can think of a few ways.

Demand for AI is growing. It makes sense they’re trying to increase profits.

The issue in that case is that OLS is BLUE, the best linear unbiased estimator (best in the sense of minimum variance). This property is what makes OLS exceptional.

LLMs are okay at bisecting programs and identifying bugs in my experience. Sometimes they require guidance but often enough I can describe the symptom and they identify the code causing the issue (and recommend a fix). They’re fairly methodical, and often ask me to run diagnostic code (or do it themselves).

Thanks for sharing. It’s an alluring idea. I think you’d find the concept of statistical identifiability interesting.

As a reliability statistician (and web user!), I'd love to see Cloudflare investing in reliability statistics. :)

Excellent analysis. It’s like: servant of whom (themselves)?