Wellness Chatbot FAIL

We talk a lot about what's going on in ChatGPT and chat is all over the place, but there's a new one that was just taken down. It was put up by the National Eating Disorders Association. It was called Tessa and the idea was it could discover coping skills for those people who have eating disorders, but they shut it down. Well, first, in March they shut down their human staffed helpline and they turned it over to the bot and It looks, like Sally, that that was a disaster.

Sally: Yes, it does, and and it's great that they tested this and realized that it wasn't working, because we all are Looking to a technology right now to replace a lot of human capabilities, but this is one area maybe where we don't have. You know, eating disorders are the among the deadliest mental illness in our country and we've had a big increase in that since the pandemic. With what they found in this Instagram platform based Chatbot, tessa was. They found that Tessa was giving advice, like you know, going on a scale or You know, having a safe deficit of calories, like things that had to do with limiting food, and a lot of people that are experiencing eating disorders Do not. We don't want to encourage them limiting food. They need more calories. A lot of these individuals facing this challenge, and so it's good that they recognize that the chatbot and I hope that in the future, more companies that are trying to use AI will weigh where we need the human involved and where we can use the artificial intelligence.

Phil: And Johanna Candell, the founder and CEO of the National Alliance for Eating Disorders, is quoted very astutely. "When someone is reaching out for help and they're in their eating disorder, they are given content that's not only not helpful with connecting them to care, but can be triggering. That can do so much more harm than good. That we're even talking about chatbots as a way to disseminate mental health treatment or prevention or mental health care at all really highlights the crisis that we're in with the mental health epidemic in the country." Very well said. And to your point, Sally. Before these companies start using it as it relates to our health, wellness, food, they need to do a lot more testing, i think, than they ever realized the impact this is going to have.