Start your day with intelligence. Get The OODA Daily Pulse.

Home > Briefs > Technology > Promptware : Cyber Criminals new AI weapon

Promptware : Cyber Criminals new AI weapon

SafeBreach researchers have revealed how a malicious Google Calendar invite could be used to exploit Gemini—the AI assistant that Google has built into its Workplace software suite, Android operating system, and search engine—as part of their ongoing efforts to determine the dangers posed by the rapid integration of AI in tech products. The researchers dubbed an exploit like this “promptware” because it “utilizes a prompt—a piece of input via text, images, or audio samples—that is engineered to exploit an LLM interface at inference time to trigger malicious activity, like spreading spam or extracting confidential information.” The broader security community has underestimated the risks associated with promptware, SafeBreach said, and this report is meant to demonstrate just how much havoc these exploits can wreak. At a high level, this particular exploit took advantage of Gemini’s integration with the broader Google ecosystem, the ability to clutter up Google Calendar’s user interface with invitations, and their intended victim’s habit of thanking an automaton for… automaton-ing. The researchers said this allowed them to indirectly trigger promptware buried within the user’s chat history and perform the following actions:

Full report : Google’s AI could be tricked into enabling spam, revealing a user’s location, and leaking private correspondence with a calendar invite – ‘promptware’ targets LLM interface to trigger malicious activity.