Apple Blocks Update of ChatGPT-Powered App, as Concerns Grow Over AI’s Potential Harm

iPhone maker asks email app with AI-language capabilities to set a 17-and-older age restriction

Apple Inc. has delayed the approval of an email-app update with AI-powered language tools over concerns that it could generate inappropriate content for children, according to communications Apple sent to the app maker. The software developer disagrees with Apple’s decision.

The dispute shows the broad concerns about whether language-generating artificial-intelligence tools, such as ChatGPT, are ready for widespread use.

Apple took steps last week to block an update of email app BlueMail because of concerns that a new AI feature in the app could show inappropriate content, according to Ben Volach, co-founder of BlueMail developer Blix Inc., and documents viewed by The Wall Street Journal.

BlueMail’s new AI feature uses OpenAI’s latest ChatGPT chatbot to help automate the writing of emails using the contents of prior emails and calendar events. ChatGPT allows users to converse with an AI in seemingly humanlike ways and is capable of advanced long-form writing on a variety of topics.

“Your app includes AI-generated content but does not appear to include content filtering at this time,” Apple’s app-review team said last week in a message to the developer reviewed by the Journal.

The app-review team said that because the app could produce content not appropriate for all audiences, BlueMail should move up its age restriction to 17 and older, or include content filtering, the documents show. Mr. Volach says it has content-filtering capabilities. The app’s restriction is currently set for users 4 years old and older. Apple’s age restriction for 17 and older is for categories of apps that may include everything from offensive language to sexual content and references to drugs. Mr. Volach says that this request is unfair and that other apps with similar AI functions without age restrictions are already allowed for Apple users.

“Apple is making it really hard for us to bring innovation to our users,” said Mr. Volach.

An Apple spokesman said that developers can challenge a rejection through its App Review Board appeal process and that it is investigating Blix’s complaint.

So-called generative AI has emerged as one of the most closely watched developing technologies in decades, primarily kicked off by ChatGPT, a chatbot created by OpenAI.

The technology has quickly generated controversy. Following the release of Microsoft Corp.’s Bing search engine powered by ChatGPT, early testers grew concerned with responses generated by the chatbot, including incorrect information as well as seemingly unhinged and angry responses. Microsoft, which has invested billions in OpenAI, defended the Bing upgrade as a work in progress.

Apple’s attempt to set an age restriction to help moderate content from a language-model-based AI is an indication the tech giant is closely watching the new technology and the risks it poses. The company has long said it must carefully curate and review what software can be accessed on the iPhone and iPad through its App Store to keep its products private and secure.

Microsoft recently released an updated version of its Bing smartphone app with the ChatGPT functionality to Apple’s App Store and Google’s Android Play Store. Bing is listed in the iPhone App Store with the 17-and-older age restriction that Apple is asking of BlueMail, while Bing on the Google Play store has no age restrictions. Bing in the App Store already had a 17-and-up age restriction because of the app’s ability to find adult content, a Microsoft spokesman said.

For BlueMail, Apple’s rejection came a week after the company submitted the app upgrade for review. Mr. Volach said Apple used a test version of the upgraded app every day before he got a response. BlueMail was able to update its Android BlueMail app on the Google Play app store without any requests for age restriction or further content filtering, Mr. Volach said.

Mr. Volach says Apple is unfairly targeting BlueMail. The app has content filtering, and placing a higher age restriction on the app could limit distribution to potential new users, he said. Mr. Volach also said many other apps that advertise a ChatGPT-like feature listed on Apple’s App Store don’t have age restrictions.

“We want fairness,” said Mr. Volach. “If we’re required to be 17-plus, then others should also have to.”

In the past, Apple has at times discovered an issue with an app that leads the company to apply a new rule more broadly. Initial inconsistency in applying App Store policies—especially new policies—isn’t uncommon, said Phillip Shoemaker, former senior director of the App Store review team at Apple, who left in 2016.

There are hundreds of individuals reviewing each app, and “not everyone sees the same thing,” Mr. Shoemaker said. “Some are viewing apps faster than others and could be missing things. The inconsistency could be for a variety of reasons.”

Apple was an early entrant in bringing AI technology mainstream with the introduction of the Siri voice assistant in 2011. But to date, Apple appears to have stayed out of the fray of generative AI. At an internal AI conference for company employees last month, sessions were focused on areas such as computer vision, healthcare and privacy, according to internal documents viewed by the Journal.

Last month, on the company’s quarterly earnings conference call, Apple Chief Executive Tim Cook said AI “is a major focus of ours,” pointing out AI-enabled features such as crash detection. “We see an enormous potential in this space to affect virtually everything we do,” he further stated.

Mr. Volach has had a contentious history with Apple. In 2019, Apple announced a software feature called “Sign in with Apple,” which allows users to sign into an app without having to give away personal information such as email. Blix had patented a similar feature earlier. Soon after Apple’s sign-in feature was announced, Apple removed the BlueMail app from its Mac app store. At the time, Apple said the removal of the BlueMail app was due to security concerns. Mr. Volach said that there was never a security issue and that Apple eventually ended up approving the app many months later.

The incident prompted Blix to file an antitrust lawsuit against Apple in 2019. A federal judge dismissed the company’s case, stating that Blix failed to offer evidence of Apple’s monopoly power and anticompetitive behavior.

Antitrust lawyer Jonathan Kanter was hired by Blix as legal counsel in its antitrust case against Apple. In 2021, Mr. Kanter took over as head of the antitrust division at the U.S. Justice Department, which is currently pursuing its own antitrust investigation into Apple.