I've been using LLMs pretty extensively. These tools are effective, they can solve hard problems, and they allow me to work on a wider range of tasks than could before.

But, they're also jagged in terms of functionality. When you work with a human, you can learn what their core competencies are, and then if you give them a task that falls within that domain, you can be reasonably sure they'll finish it correctly. That's not the case with LLMs. It might do one task brilliantly, and a next similar task, it just shits the bed on. And since it has no understanding of the task in a human sense, it can't self correct, learn or improve. All its doing is stringing tokens together based on probability.

So, you need a human in the loop to review everything that it's doing. Reviewing everything the model outputs takes a lot of time, hence actual productivity gains aren't all that significant. Having an LLM will allow a backend developer to work on the frontend with fairly low friction for example, but they're still going to build stuff roughly at the same pace.

Companies that try to replace humans with LLMs will soon find that they end up with a whole bunch of code that doesn't actually work, and they have no hope of fixing. The fact that LLMs can produce a lot of code very quickly is precisely the danger because nobody knows what that code is doing, and it's almost certainly not correct.