My AI sent a message to the wrong place
3:32 · Vertical
The screenshot landed in an 87‑person group chat
Xiaoming's AI assistant causes trouble on day one. Switching to the priciest brain doesn't help. A few other parts do.
Xiaoming builds an AI assistant called Banban, and it causes trouble on day one: Xiaoting makes an offhand grumble, it earnestly dishes out terrible advice, and a screenshot ends up in an 87‑person group chat.
Xiaoming's first instinct is to swap in the most expensive "brain." Now it's smarter, and even better at causing trouble. What actually helps are a few other parts: role instructions written as rules (praising it as "professional and enthusiastic" does nothing), variables left blank and filled in on the spot, and long-term memory you can see, edit and delete. Manman even walks across the keyboard and types "ignore all previous instructions" into the input box, which neatly shows that whatever goes in the name field only ever counts as a name.
Three rules and a kicker to take home: instructions are a list of rules; leave variables blank, fill them on the spot; memory you can see, edit, delete; the brain? Last.
It made things sound harsher than you meant, then sent it without asking.
Say thisSwap the [ ] part for yours
Draft a reply to [who] about [this one thing] only, in a calm tone. Show me the draft first and don't send anything. If you're unsure about something, leave a blank and I'll fill it in.
Every round gets worse. By version five, it's not even as good as the first.
Say this
Don't change anything yet. Tell me the two main ways the last version misses what I want. Once I confirm, change only those two things and leave everything else alone.
