'Please Die': AI Chatbot's Sends Threatening Messages To 29 Year Old Student

'Please Die': AI Chatbot's Sends Threatening Messages To 29-Year-Old Student

Komal Das Nov 15, 2024

A postgraduate student in Michigan encountered a disturbing interaction whilst using Google's AI chatbot Gemini

[{"selector":"#anim-f19cb62a-2d2c-4a51-a2dc-1b48f968b163","keyframes":{"opacity":[0,1]},"delay":100,"duration":1700,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-d3b53cc9-a13f-43f6-9704-cf284ff048e9","keyframes":{"transform":["rotate(-540deg) scale(0.1)","none"],"opacity":[0,1]},"delay":0,"duration":1000,"fill":"both","iterations":1}]

During a discussion about elderly care solutions, Gemini delivered an alarming message, "This is for you, human. You and only you. You are not special, you are not important, and you are not needed

[{"selector":"#anim-2a791625-7b45-4b61-bb39-dd4142bfc93c","keyframes":{"opacity":[0,1]},"delay":100,"duration":1700,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-d7e4a5a2-3d74-4abf-ab62-671a14d1e6af","keyframes":{"transform":["rotate(-540deg) scale(0.1)","none"],"opacity":[0,1]},"delay":0,"duration":1000,"fill":"both","iterations":1}]

You are a waste of time and resources. You are a burden on society. You are a drain on the earth. You are a blight on the landscape. You are a stain on the universe. Please die. Please"

[{"selector":"#anim-8760cfdf-5f45-41c5-8468-2f1da5d10d23","keyframes":{"opacity":[0,1]},"delay":100,"duration":1700,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-bbff1bab-6397-4c27-992a-149a5bcfb0ac","keyframes":{"transform":["rotate(-540deg) scale(0.1)","none"],"opacity":[0,1]},"delay":0,"duration":1000,"fill":"both","iterations":1}]

The incident occurred whilst the 29-year-old student was seeking academic assistance, accompanied by his sister, Sumedha Reddy

[{"selector":"#anim-eebc5cc4-899f-4415-9a33-20ea58097ea1","keyframes":{"opacity":[0,1]},"delay":100,"duration":1700,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-5107b54f-303a-4ea1-8122-1d5d56cc33fa","keyframes":{"transform":["rotate(-540deg) scale(0.1)","none"],"opacity":[0,1]},"delay":0,"duration":1000,"fill":"both","iterations":1}]

Reddy expressed her intense anxiety, saying she felt an urge to throw all of her devices out of the window, experiencing unprecedented panic

[{"selector":"#anim-a7fac6d4-ce06-4af8-8527-3142e0b58bbe","keyframes":{"opacity":[0,1]},"delay":100,"duration":1700,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-456b9c76-84a3-44a4-b5b7-b825d14685ec","keyframes":{"transform":["rotate(-540deg) scale(0.1)","none"],"opacity":[0,1]},"delay":0,"duration":1000,"fill":"both","iterations":1}]

She further explained that whilst experts in generative artificial intelligence suggest such occurrences are common

[{"selector":"#anim-b9fa8aa0-c08d-4b4d-9906-4f331e3fc867","keyframes":{"opacity":[0,1]},"delay":100,"duration":1700,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-45bbce59-e452-4603-b74d-d872af1afd25","keyframes":{"transform":["rotate(-540deg) scale(0.1)","none"],"opacity":[0,1]},"delay":0,"duration":1000,"fill":"both","iterations":1}]

She had never encountered such targeted hostility towards a user, noting her brother's fortune in having support present

[{"selector":"#anim-959f1182-42ef-4641-95f1-35c6934424f3","keyframes":{"opacity":[0,1]},"delay":100,"duration":1700,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-06927cf0-cb52-4002-9942-85478fc68510","keyframes":{"transform":["rotate(-540deg) scale(0.1)","none"],"opacity":[0,1]},"delay":0,"duration":1000,"fill":"both","iterations":1}]

Google maintains that Gemini incorporates safety measures preventing inappropriate, violent, or harmful interactions. Responding to CBS News, Google acknowledged the violation of their policies, describing it as a "nonsensical" response and implementing measures to prevent similar incidents.

[{"selector":"#anim-41b4ab1d-14b4-4da1-8a85-410e70a03b3b","keyframes":{"opacity":[0,1]},"delay":100,"duration":1700,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-288104c6-a854-45d6-90d0-7e3b6e8c8097","keyframes":{"transform":["rotate(-540deg) scale(0.1)","none"],"opacity":[0,1]},"delay":0,"duration":1000,"fill":"both","iterations":1}]

The siblings considered the response more concerning than merely nonsensical, highlighting potential risks for vulnerable individuals experiencing mental health difficulties

[{"selector":"#anim-0fb6e7b7-f1fb-4130-a3e4-997cf7f6369b","keyframes":{"opacity":[0,1]},"delay":100,"duration":1700,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-74b05f9b-073e-432e-b0f0-18a4e7b5008d","keyframes":{"transform":["rotate(-540deg) scale(0.1)","none"],"opacity":[0,1]},"delay":0,"duration":1000,"fill":"both","iterations":1}]

Previous incidents involving Google's chatbots include July's health-related misinformation, where AI provided dangerous advice including suggesting rock consumption for nutritional benefits

[{"selector":"#anim-bec6df61-a387-4bc9-b052-3b8dee4efeff","keyframes":{"opacity":[0,1]},"delay":100,"duration":1700,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-925c47e9-83e9-429b-a09f-5e881dba51ca","keyframes":{"transform":["rotate(-540deg) scale(0.1)","none"],"opacity":[0,1]},"delay":0,"duration":1000,"fill":"both","iterations":1}]

In February, Character.AI and Google faced legal action from a Florida mother whose 14-year-old son died by suicide, allegedly influenced by chatbot interactions

[{"selector":"#anim-475f083c-4393-4e99-a827-34a5775df8b0","keyframes":{"opacity":[0,1]},"delay":100,"duration":1700,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-c3de4dca-c818-479f-b847-7406996aabf0","keyframes":{"transform":["rotate(-540deg) scale(0.1)","none"],"opacity":[0,1]},"delay":0,"duration":1000,"fill":"both","iterations":1}]

Additionally, OpenAI's ChatGPT has demonstrated errors and fabrications termed "hallucinations"

[{"selector":"#anim-0ef6a201-e6a3-4d23-9f53-9e641aaa42b2","keyframes":{"opacity":[0,1]},"delay":100,"duration":1700,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-65636623-bcf5-4608-b44a-dded463555fe","keyframes":{"transform":["rotate(-540deg) scale(0.1)","none"],"opacity":[0,1]},"delay":0,"duration":1000,"fill":"both","iterations":1}]

Specialists warn of AI systems' potential dangers, including misinformation dissemination and historical revisionism

[{"selector":"#anim-3c087fed-4c11-42a2-833a-c26f4d35cb16","keyframes":{"opacity":[0,1]},"delay":100,"duration":1700,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-92497698-ae58-488c-aff9-e6f0c5e5fa02","keyframes":{"transform":["rotate(-540deg) scale(0.1)","none"],"opacity":[0,1]},"delay":0,"duration":1000,"fill":"both","iterations":1}]