Loading…
29
Pro tip: running that AI chatbot on 500 fake customer chats gave me results I did not expect
Last month I fed an AI support bot 500 made up customer chats to test its answers, and it started making up refund policies that never existed. I caught it 37 times in one afternoon giving wrong info about shipping times to people in Ohio. What got me was how sure it sounded, like it was reading straight from a manual that was never written. I showed my boss in Austin and we stopped the rollout for two more weeks. Now I test every new model with real transcripts, not fake ones, because fake data taught it fake rules. Anyone else run into a bot that just invents stuff out of thin air?
0 comments
Log in to join the discussion
Log In0 Comments
No comments yet
Be the first to share your thoughts on this discussion.