My text generator went off the rails during a client demo in Denver

I was showing a new custom model to a small tech firm last month, and it started spitting out random soup recipes mixed with their sales data. I had to quickly switch to a backup fine-tuned version I keep on a local drive. It saved the meeting, but I spent the next two days adding stricter output filters. Has anyone else had a model go weird in a live setting and found a good way to test for that?

3 comments

3 Comments

andrew_palmer998d ago

Read a case study about a chatbot for a bank that started giving weird financial advice during testing. They found the training data had forum posts mixed in with official documents. Your soup recipe thing sounds similar, like maybe your model picked up some cooking blogs along with sales info. I always run new models through a bunch of weird prompt tests now, like asking it to write a poem about spreadsheets, to see if it stays on track. Catching those weird outputs before a demo is key.

thomas.tyler7d ago

But those weird outputs are half the fun sometimes. If a demo goes perfectly smooth it feels kinda fake anyway. People remember the weird poem more than the boring sales pitch.

terry_lewis2115h ago

Totally get that... thomas.tyler has a point though.