OpenAI’s long-promised GPT Store is finally here and it is already being taken over by AI-powered girlfriends within two days of launch. This signifies how the GPT Store could be hard to moderate since people are already breaking the rules of the platform.
According to GPT Store’s rules, custom Generative Pre-Trained Transformers (GPTs) need to follow strict guidelines or should not be created at all. OpenAI bans custom chatbots “dedicated to fostering romantic companionship or performing regulated activities.” It is unclear what OpenAI means by “regulated activities.”
OpenAI utilizes a blend of automated mechanisms, human evaluation, and feedback from users to identify and evaluate GPTs that may contravene its regulations. Such evaluations can result in various measures, including warnings, limitations on sharing, or disqualification from participation in the GPT store or monetization opportunities.
Clearly, the store’s usage policies and moderation are failing as searching for “girlfriend” in the store yields dozens of results showing virtual girlfriends powered by generative AI.
Some of the results include “Your girlfriend Scarlett”, “Secret Girlfriend Sua”, “Artificial Girlfriend”, “Ashley AI Girlfriend”, and many more.
Clicking on some of these virtual girlfriends opens up a chat screen with prompts such as “What does your dream girl look like?” and “Share with me your darkest secret.”
Indeed, relationship-oriented chatbots are gaining popularity. In the United States, out of the 30 AI chatbot applications downloaded in 2023 from the Apple or Google Play stores, seven were centered around AI friends, girlfriends, or companions. This information comes from data.ai, a mobile app analytics firm.
The surge in popularity of such apps stems from the growing statistics of loneliness and isolation among the US population. Studies show that 1 in two Americans feels lonely and the US Surgeon General has urged the need for more social connections among the population.
The potential role of AI chatbots in combating isolation is a topic of discussion, although there is a looming question of whether they are a genuine solution or merely a means to exploit the plight of individuals for profit.
