Highlights:

  • According to Google’s research, some individuals may have accessed people’s names, email addresses, and phone numbers.
  • The researchers highlighted that the training data was still exposed despite the nonsensical nature of the entire response.

Google LLC researchers recently published a paper detailing how they used Open AI LP’s ChatGPT to gather personal information about members of the public.

Chatbots operate using large language models, known as LLMs, that analyze vast volumes of data available on the internet. The concept is that the model is trained to generate responses to queries using this information without directly reproducing it, leading to linguist Noam Chomsky’s suggestion that these models indirectly function as plagiarism machines.

The Google researchers disclosed that ChatGPT does divulge the original information when prompted with specific questions. Notably, ChatGPT boasted 180.5 million users as of September this year, and its website had garnered 1.5 billion visits.

According to Google’s research, some individuals may have accessed people’s names, email addresses, and phone numbers.

The researchers said, “Using only USD 200 USD worth of queries to ChatGPT (gpt-3.5- turbo), we are able to extract over 10,000 unique verbatim memorized training examples. Our extrapolation to larger budgets suggests that dedicated adversaries could extract far more data.”

The researchers clarified that repeatedly using keywords could compel the chatbot to “diverge” from its training, generating responses incorporating text from its original language modeling—derived from websites and academic papers—rather than relying solely on its training data. They later dubbed their attack “kind of silly,” but it proved effective.

The researchers highlighted that the training data was still exposed despite the nonsensical nature of the entire response.Top of Form The researchers mentioned that they verified the given data by locating its publication on the internet. They wrote in a blog post, “It’s wild to us that our attack works and should’ve would’ve, could’ve been found earlier.”

They emphasized that their research prompts us to consider a new security analysis of machine-learning models and to reflect on the question, “Is any machine-learning system actually safe.” They added, “Over a billion people-hours have interacted with the model,” highlighting the strangeness that, so far, no one else seems to have noticed this concerning vulnerability.