The AI-powered text of this program could mislead the government

In October 2019, Idaho has offered to change its Medicaid program. The state needed approval from the federal government, which solicited public comment via Medicaid.gov.

Approximately 1,000 comments have arrived. But half were not from concerned citizens or even internet trolls. They were generated by artificial intelligence. And one study found that people couldn’t tell real reviews from fake ones.

The project was the work of Max Weiss, a medical student at Harvard, but it received little attention at the time. Now that AI language systems are advancing rapidly, some say government and internet companies need to rethink how they solicit and filter comments to protect themselves deepfake text manipulation and other AI-powered interference.

“The ease with which a robot can generate and submit relevant text that mimics human speech on government websites is surprising and really important to know,” says Latanya sweeney, a professor at Harvard’s Kennedy School who advised Weiss on how to conduct the experiment ethically.

Sweeney says the problems extend far beyond government services, but it is imperative that public agencies find a solution. “AI can drown the speech of real humans,” she says. “Government websites must change.”

The Centers for Medicare and Medicaid Services said they added new safeguards to the public comment system in response to Weiss’ study, though he declined to discuss details. Weiss says he was contacted by the United States General Services Administration, which is developing a New version of the Federal Government’s website for posting regulations and commentaries, on how to better protect it from false comments.

Government systems have already been the target of automated influence campaigns. In 2017, researchers found that over a million comments submitted to the Federal Communications Commission regarding rollback plans net neutrality rules had been generated automatically, some sentences being copied and pasted in different messages.

Weiss’s project highlights a more serious threat. It was remarkable progress in the application of AI to language during the last years. When powerful machine learning algorithms receive huge amounts of training data – in the form of books and text pulled from the web – they can produce programs capable of generating compelling text. Besides a myriad of useful applications, it raises the prospect that all kinds of internet messages, comments and posts could be simulated easily and in less detectable ways.

“As technology improves,” Sweeney says, “human speech places become subject to manipulation without humans knowing it has happened.” Weiss was working at a healthcare consumer advocacy organization in the summer of 2019 when he learned of the public feedback process needed to make changes to Medicaid. Knowing that these public comments had influenced previous efforts to change state Medicaid programs, Weiss sought out tools that could automatically generate comments.

“I was a little shocked to see nothing more than a submit button preventing your comment from being in the public domain,” he says.

Weiss discovered GPT-2, a program released earlier this year by OpenAI, an AI firm in San Francisco, and realized it could generate fake comments to simulate a tidal wave of public opinion. “I was also shocked at how easy it was to adjust GPT-2 to spit out feedback,” Weiss says. “It is relatively worrying on several fronts.”

Besides the comment generator tool, Weiss has created software to automatically submit comments. He also conducted an experiment in which volunteers were asked to distinguish between comments generated by AI and those written by humans. The volunteers did nothing better than guess at random.

After submitting the comments, Weiss notified the Centers for Medicare and Medicaid Services. He had added a few characters to make it easier to identify each false comment. Even so, he says, the AI ​​comments remained online for several months.

OpenAI released a more powerful version of its text generation program, called GPT-3, last June. So far, it has only been made available to a few AI researchers and companies, with some people creating useful apps such as programs that generate emails from microchips. Upon the release of GPT-3, OpenAI stated in a research paper that he had not seen any signs of malicious use of GPT-2, although he was aware of Weiss’s research.