NIST Launches NIST GenAI for Detecting AI-generated Content

Highlights:

• NIST GenAI aims to establish fresh AI benchmarks and develop systems for detecting “content authenticity.”
• As stated on its updated website, NIST AI’s initial project focuses on constructing systems capable of accurately distinguishing between content generated by humans and AI systems.

The National Institute of Standards and Technology NIST has launched NIST GenAI as its latest undertaking. This initiative is designed to evaluate generative artificial intelligence models and develop systems capable of discerning AI-generated text, images, and videos.

The unveiling of the new program coincided with NIST’s release of its initial draft publications on AI risks and standards.

NIST GenAI aims to establish fresh AI benchmarks and develop what it terms “content authenticity” detection systems, capable of identifying AI-generated media like text and “deepfake” videos. This endeavor seeks to mitigate the risks associated with fabricated and deceptive AI-generated content.

According to a press release, NIST announced that the new program will “issue a series of challenge problems” aimed at assessing the strengths and weaknesses of generative AI models. Leveraging these assessments, it will subsequently explore approaches to enhance “information integrity and guide the safe and responsible use of digital content.”

As per its updated website, NIST AI’s inaugural project focuses on constructing systems capable of reliably discerning whether content originates from a human or an AI system, commencing with text analysis. Despite the existence of tools purportedly adept at identifying phenomena like deepfakes—videos manipulated by AI—several studies have demonstrated their lack of dependable accuracy.

For this reason, NIST is encouraging participation from academic teams, the AI industry, and other researchers to submit what it terms “generators” and “discriminators.” Generators refer to AI systems responsible for creating content, whereas discriminators are designed to recognize content produced by AI.

NIST requests that all generators that submit to the study produce a set of documents and a 250-word or less summary on a specified subject. The discriminators’ job in the interim will be to determine whether summaries are produced by AI or by humans. To guarantee fairness, NIST GenAI will prepare the test data independently. It also stated that systems that violate applicable laws and regulations and were trained on publicly accessible data would not be allowed to participate in the study.

NIST aims for expediency and will initiate registration for the study on May 1st, closing it on August 2nd. Subsequently, the study will commence, with results expected by February 2025.

The study arrives amidst a surge in AI-generated misinformation and disinformation. A recent report from Clarity Inc., a company specializing in deepfake detection, revealed that the volume of deepfakes released since the start of the year has risen by over 900% compared to the corresponding period in 2023.

As generative AI becomes increasingly accessible, the concern over misleading content grows. People are expressing apprehensions regarding the risks posed by AI-generated content. A recent YouGov poll revealed that 85% of Americans are anxious about being deceived by deepfakes.

Preliminary Papers on AI Policy

In addition to introducing the new program, NIST has released a collection of draft documents aimed at influencing the U.S. government’s AI policies. These drafts encompass various topics, including an examination of the risks associated with generative AI and proposed strategies for its implementation. They were formulated with input from a public working group comprising over 2,500 researchers and experts and will be utilized alongside NIST’s existing AI risk management framework.

Furthermore, NIST released a draft companion resource for its established Secure Software Development Framework, delineating optimal methodologies for developing generative AI applications and foundational models with dual-use potential. It characterizes dual-use foundational models as those trained on “broad data” and intended for “a wide range of contexts” that could potentially be “modified to exhibit high levels of performance at tasks that pose a serious risk to security, national economic security, national public health or safety, or any combination of those matters.”

Additional documents unveiled by NIST address the mitigation of risks linked to synthetic content and outline a strategy for establishing global AI standards. All four papers are available for public feedback until June 2nd.

Laurie Locascio, Director of NIST and Undersecretary of Commerce for Standards and Technology, expressed the agency’s concern that generative AI poses risks that are “significantly different” from those associated with traditional software.

Holger Mueller, an analyst at Constellation Research Inc., remarked that regulators are displaying an unusual sense of urgency in crafting suitable frameworks to facilitate the swift deployment and adoption of AI across various industries.

“NIST’s immediate focus is on trying to create systems to detect AI-generated content, which is likely to be one of the most contentious areas of generative AI use in the coming years and perhaps even decades. As with everything NIST does, the implementation of its ideas is dependent on industry pickup to a degree, so it will be important to monitor what the AI heavyweights are doing in relation to this,” the analyst said.

The recent announcements reflect NIST’s reaction to the executive order on AI issued by U.S. President Joe Biden, which outlined regulations mandating increased transparency from AI companies regarding the functionality of their models. The order also set forth guidelines for labeling generative AI content and other related matters.