- Access to and control of critical financial systems or proprietary source code
- Execution of arbitrary commands to modify machine learning (ML) algorithms, systems, or other functionalities
- Escalation or downgrade of privileges to gain more power and root access to internal and confidential systems
- Introduction of malware
- Spying on network activity
- Access to or extraction of private or sensitive data, such as customer records, employee data, legal documents, business strategies, and intellectual property
- General chaos within the targeted institution's ecosystem
Blog
12 Apr 2024
From Infiltration to Exploitation: LLMs as Threat Surfaces for Jailbreak Attacks in the Financial Sector
From Infiltration to Exploitation: LLMs as Threat Surfaces for Jailbreak Attacks in the Financial Sector
From Infiltration to Exploitation: LLMs as Threat Surfaces for Jailbreak Attacks in the Financial Sector
Building more and more AI-dependent security features into AI models’ operations is a good thing for everyone who will use them. Relying on the companies developing GenAI systems like ChatGPT, Gemini, BloombergGPT, and others to conduct thorough red-teaming or, ideally, purple-teaming engagements to test those tools against potential threats is crucial. However, it is only the beginning. Cybercriminals have their own versions of red-teaming, table-top exercises, and planning activities.
Significant players in cybercrime have followed the standard path for large businesses, merging, acquiring, vertically integrating, and establishing organizational structures with teams reporting to top and mid-level management in operations and other critical business units. Their business model is “cybercrime as a service” and they actively market products, such as malicious code. Jailbreak attacks are just one type of weapon in their arsenal, but it is an especially insidious one, as the code used in the attack can be both the crime and the perpetrator.
Successful jailbreak attacks via large language models (LLMs) in the financial sector can do more than just override system safeguards. In the hands of a highly skilled hacker, a compromised model could allow:
