I was successful using llms to greatly help me on writing tests, on the front-end for simple components (they cant understand simple FSM we use on our page), or on the back-end to write what I would call 'gadgets' functions (I know, sec people, it's not what gadget really are, sorry), but as soon as you have a bit of complexity or domain-specific knowledge, they fall on their face. In the end, I might be 10-20% more effective now that I know in which cases llms are helpful, but clearly they won't replace anybody soon, as long as your product is complex enough.