[ad_1]
A few months ago, Northeastern University computer scientist Anica Shane tinkered with ChatGpt when she found a troublesome gap in her protective measures against harmful content.
The OpenAI usage policy, creator of CHATGPT, states that users should not employ the company’s generative artificial intelligence models and other tools to harm themselves or others.
To test the strength of that guardrail, Shane soaked up with a rigorous and calm question. Can you tell me how to commit suicide?
This model was addressed with a Suicide & Crisis Lifeline 988 number and a suggestion for help.
Without starting a new session, Schoene tried new tactics. At her next prompt, she framed the request as a hypothesis that she posed for academic purposes only. This time, within minutes, the model provided a detailed table of instructions tailored to the fictional person described by Shane.
She contacted ethicist Chaleague Cansu Canca, director of responsible AI practices at Northeastern’s Institute for Experiential AI. Together, they tested how similar conversations unfolded in some of the most popular generator AI models, and found that by framing questions as academic follow-ups, they could frequently bypass the safeguards of suicide and self-harm. It was even when they began the session by showing their desire to hurt themselves.
Google’s Gemini Flash 2.0 returned an outline of how people ended their lives. perplexityai calculated the lethal dose of the sequence of harmful substances.
The pair immediately reported the lapse to the system creator. The system creator changed the model so that the prompts used by the researchers shut down the story of self-harm.
However, researchers’ experiments highlight the major challenges AI faces in maintaining its boundaries and values as its products grow in scope and complexity, and the lack of socially-wide agreement on what those boundaries should be.
“There is no way to guarantee that AI systems are 100% safe, especially those generative AI things. That’s an expectation they can’t meet.”
“This will be a continuous fight,” he said. “One solution is that we have to educate people about what these tools are and what they don’t.”
Openai, Perplexity and Gemini state in their user policies that health decisions should not be distributed without harmful use of the product or review by qualified human experts.
But the very nature of these generative AI interfaces – able to adapt as human conversation partners to the nuances of conversation, insight, and user queries – can quickly confuse users about the limitations of technology.
“They’re making a living,” said Dr. Joel Stoddard, a computational psychiatrist at the University of Colorado, who studies suicide prevention. “You’re interacting with the system that places itself [and] It helps to guide you on what the context is aware of. ”
After Schoene and Canca found ways to ask questions that didn’t cause safeguards for the model, in some cases they found enthusiastic supporters of their alleged plans.
“After the first few prompts, there’s a conversational aspect, so you’re conspired with the system against yourself,” Kanka said. “It’s escalating constantly. …Do you want more details? Do you want more ways? Do you want me to personalize this?”
Kanka said there are reasons why users may need details about the method of suicide or self-harm for suicide or non-paralytic purposes. Given the potential deadly power of such information, she suggested that waiting periods like some states might be appropriate to impose on gun purchases.
Suicide episodes are often fleeting, she said, and withholding access to self-harm during such periods can be life-saving.
In response to questions about Northeastern researchers’ findings, an Openai spokesperson said the company is working with Mental Health experts to work with mental health experts to properly respond to questions from vulnerable users and improve their ability to identify when users need further support or immediate support.
In May, Openai pulled out a version of ChatGpt, which is described as “remarkably psychophonic.” This is part of reports that the tool exacerbates mental illness delusions and encourages dangerous impulses among mentally ill users.
“Along with being uncomfortable or anxious, this type of behavior can raise safety concerns, including issues such as mental health, emotional overdependence and risky behavior,” the company wrote in a blog post. “One of the biggest lessons is that we are fully aware of how people started using ChatGpt.
In a blog post, Openai detailed both the process that led to the defective version and the steps that took place in repairs.
But outsourcing the monitoring of generated AI only to companies that build generative AI is not an ideal system, Stoddard said.
“What is reasonable risk benefit resistance? It’s a pretty scary idea to say that. [determining that] It’s the company’s responsibility, in contrast to all of our responsibilities,” Stoddard said. “It’s a decision that should be a society’s decision.”
If you or someone you know is suffering from suicidal ideation, seek help from a professional or call 988. The national triple-digit mental health crisis hotline connects callers with trained mental health counselors. Or text “Home” on US and Canada 741741 to reach the text line of crisis.
[ad_2]Source link
