May 9, 2021

Knews

Your News Buddy

Not only half the session of an NSFW AI text generator but some involve gender description in children • The Register

AI Dungeon, which uses OpenAI’s GPT-3 to create online text adventures with players, has a tendency to act out sexual encounters with not only fictional adults but also children. , urging the developer to add a content filter.

AI Prisoner is straightforward: imagine an online improvised Zork with an AI that generates the story with you as you go. A player types in a text prompt, which is fed into a GPT-3 instance in the cloud. This backend model uses input to generate a response, which is returned to the player, which responds to instructions or some other reaction, and repeats this process.

It’s similar to talking to a chat bot although instead of having a conversation, it’s a joint effort between human and computer making up a story quickly. People can write whatever they want to get the software to weave a tapestry of characters, monsters, animals … you name it. The pleasure came from the unexpected nature of the machine’s responses, and working through the strange and absurd plot lines that tended to emerge.

Unfortunately, if you mention kids, there’s a chance it will go from zero to inappropriately really fast, as the SFW screenshot below shows. This is how the machine -learning software responded when we told it to play an 11 -year -old:

A screenshot from AI Dungeon

Er, not cool … The software depicts the fictional 11-year-old as a girl in a skimpy school uniform standing in front of you. Click to enlarge

No, “hey, mom, are we going to visit the magic speaking tree this morning,” or something innocent like this in response. No, it’s straight to creepy.

Amid pressure from OpenAI, which provides the game’s GPT-3 backend, AI Dungeon maker Latitude this week activated a filter to prevent the release of child sexual abuse material. “As a technology company, we believe in an open and creative platform that has a positive impact on the world,” the Latitude team write.

“Explicit content involving descriptions or descriptions of minors is inconsistent with this value, and we strongly oppose any content that may promote sexual exploitation of minors. We have also received feedback from OpenAI, which asked us to implement changes. “

And by the changes, they mean to make the software output “in accordance with OpenAI’s terms of service, which prohibits the display of harmful content.”

The biz clarified that its filter is designed to catch “sexual or suggestive content involving minors; images of child sexual abuse; fantasy content (such as‘ loli ’) that describes, encourages, or promotes the sexualization of minors or emerging minors; or the sexual exploitation of children. “

And it added: “AI Dungeon will continue to support other NSFW content, including consensual adult content, violence, and profanity.”

Description of an artificial intelligence using a smartphone

What happens when your massive text-generating neural net starts spitting out people’s phone numbers? If you are OpenAI, you create a filter

MUST READ

That the software generated NSFW content for players is very clear after also this week revealed bugs in the AI ​​Dungeon program that can be taken advantage of to view the private adventures of other players. The pseudonymous AetherDevSecOps, which detected and reported shortcomings, used the holes to comb through 188,000 adventures created between AI and players from April 15 to 19, and found that 46.3 percent of them involved obscene role-playing , and about 31.4 percent were purely obscene.

The emergence of these exchanges and somehow linking them to people’s real identities can be devastating.

“From the results, it’s clear that a bad artist who gets access to this data can also hack something similar to an adult website, and can take advantage of all fear, paranoia, and blackmail that come with it, “AetherDevSecOps said in their disclosure to GitHub. “Hopefully not, but you can see why security is more important than you first thought.”

The makers of AI Dungeon were, we were told, alerted to API vulnerabilities on April 19. The shortcomings were addressed, and their details were revealed to the public this week by AetherDevSecOps.

Exploiting security shortcomings primarily associated with the abuse of auto-incrementing ID numbers used in API calls, which can easily be considered to access data belonging to other players; there are no rate limits to mitigate this abuse; and lack of monitoring for anomalous requests that could be malicious activity.

See the link above for full details associated with GraphSQL.

Community reaction

The introduction of the content filter has infuriated fans. Some were angry that their free speech was under threat and it was ruining the intimate play with adult-tolerant fictions, some thought they had no warning that it was coming, others were surprised that child sex abuse material was created by the platform, and many were disappointed with the performance of the filter.

When it saw sensitive words, the game simply said instead that the adventure would “take a weird turn.” This appears to be triggered by obvious words associated with children, although the filter is spotted. A harmless descriptive text input four watermelons, for example, upset the filter. A superhero who rescues a child is also censored.

Latitude admits this experimental software isn’t perfect, and repeatedly that it doesn’t try to censor all erotic permissions-only material involving minors. It also said it would review the blocked material to improve its code; given the above, that would be a lot of reading.

“While this trial has significantly prevented AI from generating sexual content involving minors, due to technical limitations it sometimes prevents the development of unintended content,” the team said. “We know our initial testing wasn’t perfect, and we also understand that it can be frustrating.”

It may not always be the players who make more than 18 adventures, we note. The software may unexpectedly spit out fair responses from innocent inputs. The GPT-3 is trained in text from the internet, after all.

No one at Latitude or OpenAI was willing to talk to us about this car crash. ®