Every time Support received a new email, a ticket in Salesforce would be created and assigned to Support
Every time Support was assigned a new ticket, Salesforce would send a notification email
I think it took us a good hour and a few hundred tickets to get the helpdesks to stop fighting with each other!
I remember working for an ISP in the mid 90s. We never really had problems with 1 to 1 mailing loops bouncing back and forth, but we ended up with a large circular mailing loop involving a mailing list, and bad addresses on it getting bounced to the previous server which sent a reply to the mailing list, which got bounced and sent to everyone in the group which caused someone else's mailbox to fill up that was in a forward, which for some reason sent a bounce to the mailing list that really started to set off the explosive growth.
Needless to say the bounces seemed to be growing quadratically and overwhelmed our medium sized ISP, a decent sized college, and a large ISPs mailing system in less time than anyone could figure out how to get it to stop.
I’d rather track everything in a giant excel tyvm
As in a lot of cases, the answer is money. If you have expertise in Salesforce, you can get paid a lot, especially if the company you contract/freelance for is in an "emergency" which, because they use Salesforce, they'll eventually be. As long as you get the foot in the door, you'll have a steady stream of easy money. It fucking sucks though, the entire ecosystem, not for the weak of heart.
There’s a limited number of you who are willing to traverse that gauntlet of abuse, so you know you’ll always have work.
IT were not stupid though, and set a series of rules:
1. You cannot have a rule trigger to email yourself.
2. You cannot reply to an email triggered by a rule.
3. You have ~50MB max of emails (which was a lot at the time).
Playing around one lunch, my friend had setup a "not in office" automated reply, I setup a rule to reply to any emails within our domain with a "not in office", but put their name in TO, CC and BCC. It turns out that this caused rule #2 not to trigger. After setting up the same rule on my friend's email, and sending a single email, the emails fired approximately one every 30 seconds.
A few hours later we returned to our email boxes to realise that there were thousands and thousands of emails. At some point we triggered rule #3, which in turn sent an email "out of space", with a small embedded school logo. Each one of these emails triggered our email rule, which in turn triggered an email "could not send message", again with an embedded logo. We desperately tried to delete all of the emails, but it just made way for more emails. We eventually had to abandon our efforts to delete the emails, and went to class.
About an hour later, the email server failed. Several hours later all domain logins failed. It turned out that logins were also run on the email server.
The events were then (from what I was told by IT):
* Students could not save their work to their network directory.
* New students could not login.
* Teachers could not login to take registers or use the SMART white boards.
* IT try to login to the server, failure.
* IT try to reboot the server, failure.
* IT take the server apart and attempt to mount the disk - for whatever reason, also failure.
* IT rebuild the entire server software.
* IT try to restore data from a previous backup, failure. Apparently the backup did not complete.
* IT are forced to recover from a working backup from two weeks previous.
All from one little email rule. I was banned from using all computers for 6 months. When I finally did get access, there was a screen in the IT office that would show my display at all times when logged in. Sometimes IT would wiggle my mouse to remind me that they were there, and sometimes I would open up Notepad and chat to them.
P.S. Something happened on the IT system a year later, and they saw I was logged in. They ran to my class, burst through the door, screamed by username and dragged me away from the keyboard. My teacher was in quite some shock, and then even more shocked to learn that I had caused the outage about a year earlier.
Salesforce is such an ugly beast
> 4609 remaining items
Seems gemini-cli and gemini-cli didn't understand who themselves were, so they though someone else added/removed the label, which it tried to correct, which the other then tried to correct, which the other...
Considering that that repository has what seems like ~10 longer term contributors, who probably get email notifications, together with a bunch of other people who get notifications about it, wonder how many emails were sent out because of this? If we just assume ten people get the emails, it's already 46K emails going out in under 24 hours...
Also, who pays for the inference of this gemini-cli? Clicking the "user" links to https://github.com/apps/gemini-cli, and it has a random GitHub user under "Developer", doesn't seem like it's a official Google project, so did someone pay for all of these inference calls here? That'd be a pretty sucky bill to pay...
https://github.com/google-gemini/gemini-cli/issues/16723
https://github.com/google-gemini/gemini-cli/issues/16725
Unfortunately the app creation flow on GitHub makes it impossible (for now) for a normal org user to create an app for the org, so apps end up getting created on personal accounts and become load bearing. We've got a work item to make it possible to give app creation rights to your org members, I've got it high on the priority list for the next six months.
Re:payment As I understand it each org that uses the gemini cli agent puts their api key in their actions secrets, which gets picked up and used to call Google inference APIs. So the org these comments are in is paying for the inference.
Considering that these responses are all the exact same two replies in wording, and that this is a task which could be easily automated without AI, I seriously doubt that it's going to be caused by actual inference.
It’s not just bots that fall into this trap.
Unless GitHub are idiots they batch email updates to mitigate this
I've made "reply bots" before, bunch of times, first time on IRC, and pretty much the second or third step is "Huh, probably this shouldn't be able to reply to itself, then it'll get stuck in a loop". But that's hardly a "classic CI bug", so don't think that is what you're referring to here right?
And there lie dragons, because whether a tired or junior or (now) not-even-human engineer is writing new sub-behavior, it’s easy to assume that footguns either don’t exist or are prevented a layer up. There’s nothing more classic than that.
Otherwise I still don't see how you'd end up with your own bot getting stuck in a loop replying to itself, but maybe I'm misunderstanding how others are building these sort of bots.
1. Bot run a series of steps A through Z.
2. Step X is calling an external system that runs its own series of steps.
3. Some potential outcomes of said external system is if it detects some potential outcomes (errors, failed tests, whatever) is it kicks back an automated process that runs back through the bot/system where said system makes the same mistake again without awareness it's caught in a loop.
This I'd understand, bit trickier since you're basically end up with a problem typical of distributed systems.
But one bot? One identity? One GitHub user? Seems really strange to miss something like that, as you say, it's one of the earlier things you tend to try when creating bots for chats and alike.
fwiw. doesn't look like gemini at all, the responses are perfectly canned... maybe just good old fashioned ci rules.
The automation: https://youtu.be/GFiWEjCedzY?t=51