Got burned by an automation script during a demo at a conference in Austin last month

Had a sales pitch going smooth and the AI bot went rogue, started apologizing in old English for no reason. Made me realize I need to test every edge case before showing clients - anyone else had demos go sideways?

2 comments

2 Comments

flores.emma14d ago

Yeah, the whole "went rogue" thing is rough. I think what people miss is that it's not just the bot going off script, it's that the script itself wasn't tested for how it handles completely unexpected input. Like when a client asks a super weird question you never trained it on, the bot just crashes or spits out nonsense. @evan_davis makes a good point about a kill switch, but I'd add that you have to think about the kill switch's own logic too. If the bot goes rogue because it's misinterpreting a kill command, you're stuck. I've seen demos where someone yelled "stop" and the bot just kept apologizing louder. What nobody talks about is how the environment of the demo itself can mess things up. Conference wifi is bad, the room has weird acoustics, or someone's phone interference sets off a command. You might have tested every edge case in your office but the real world has its own glitches. Maybe it's just me, but I'd rather have a live human step in and take over manually for a minute than trust any automation to self-correct in the moment. The human can always just say "hey, sorry about that glitch, let me show you what it's supposed to do" and save the whole thing.

evan_davis14d ago

The AI bot went rogue" is exactly why I never let tech run unsupervised without a kill switch.