In our company, Claude keeps trying to run terraform commands (another team member's experience). Thankfully, it's in a locked-down environment, but it still keeps trying, even if you tell it not to.

In development, it will sometimes do all sorts of wild and convoluted things because the conversation has been compressed and it suddenly can't remember the structure of the project or other critical instructions.

AI should never have access to a production environment. Ever. Thankfully, we enforce that.

The worst that's happened, that I'm aware of, is I made a change with A.I. that I didn't realize broke something else (that I wasn't even aware our reports supported, it was new to me), and apparently QA didn't think to check for it either (although I did ask them specifically to verify everything in the report still worked), so it made it into Production, clients complained, and I was told about it and got it fixed over the next couple of days.

But that could have happened if I wasn't using A.I. also, because it wasn't something I was aware I had to check for and I could have also broken that same thing while doing the work manually.

We don't have AI agents do anything directly in Production, so I think this is about the worst that can happen with our setup right now.