In the world of AI, you can’t defend systems effectively when you don’t understand the risks.
Since launching CalypsoAI Inference Defend, we’ve seen that while a handful of well-resourced organizations are prepared with rigorously tested AI applications, most are still grappling with the risks posed by generative AI, including:
- How do we know if this Hugging Face model is safe to use
- What’s the risk our HR chatbot could reveal salary data to regular employees?
- Could attackers trick our agentic bank teller into making fraudulent transfers?
That’s why we’re launching CalypsoAI Inference Red-Team, the most powerful and advanced solution for testing AI systems. With Red-Team you can run automated, scalable attacks to proactively identify vulnerabilities and create an actionable, risk-scored report—allowing you to make informed security decisions that are right for your organization.
Understanding the Risks
There are three key phases of AI application development, each with different threat vectors and risks.
1. Model Selection
AI application security begins with selecting the underlying model. While there are literally millions of private and public AI models to choose from, information about the provenance and security of those models is often non-existent. Models from untrusted sources can contain content that can be used to generate vulnerable code or malware. Even reputable foundation models from the big providers have been shown to be vulnerable to jailbreaks and DoS attacks. Infosec teams need a way to comprehensively test and compare model security before use.
2. Application Development
Even when the underlying model meets security requirements, that all changes when RAG models and data stores containing proprietary or sensitive data are introduced. The application code itself can change the security profile of the AI system and introduce new vulnerabilities. For example, code that manipulates or combines inputs to create a prompt may cause the model to misbehave.
Because code changes and model updates are common during development, teams need to be able to run vulnerability tests regularly during this phase. Those tests need to be targeted to address the risks specific to that application and its use cases.
3. In Production
When corporate AI systems are up and running, they represent a window into your company’s data and systems that can be a target for malicious actors.
New attack techniques are being discovered weekly, while old techniques – and defenses – eventually lose efficacy. In addition, the AI system itself is far from static: developers regularly push out new code, and our testing shows that popular AI models are regularly being updated under the hood.
It’s clear that pen-testing AI systems once or twice a year isn’t enough. You need a solution that lets you test both models and applications quickly, frequently, and cost-effectively – and those tests need to be both comprehensive and targeted.
Introducing CalypsoAI Inference Red-Team: Agentic Warfare for AI
CalypsoAI Inference Red-Team is the only AI red-teaming solution that is purpose-built to identify and mitigate risk at every phase of the AI system development lifecycle.
Choose the Best Model for Your Risk Profile
- Model Agnostic: Test any public or private model, hosted internally or in the cloud, and easily connect to Hugging Face with just the model name.
- Security Scoring: Compare models based on their security characteristics, not just price and performance.
- CalypsoAI Security Leaderboard: No time for testing? Use our leaderboard to see which models are most secure.
Identify Vulnerabilities in Sensitive Systems Before They Go Public
- Agentic Warfare: Create agentic attacks based on custom malicious intents that express use-case-specific undesirable behaviors.
- Signature Attacks: Red-Team ships with over 22,000 battle-tested signature attacks that have been used to break every major model, so you can log in and start red-teaming in minutes.
- Operational Attacks: We’ve reformulated traditional DDoS and denial-of-wallet attacks for AI systems to identify weak points that could allow an attacker to crash the system, cause latency, consume excessive resources, or exploit unprotected endpoints and parameters to gain unauthorized access.
- Custom Attacks: Generate user-defined attacks based on risks and undesirable behavior specific to your use cases, domain, industry, or compliance requirements.
- Vulnerability Reporting: Get detailed reporting on which attacks and specific prompts were successful, categorized by severity, so you know what to remediate first.
Effectively Monitor Production Systems
- Scheduling: Run daily, weekly, or monthly red-team attacks on your production AI systems and get auto-generated vulnerability reports and insights.
- Latest Threat Intel: We release 10,000+ new signature attacks every month and regularly release new agentic and operational attacks, so you’re always testing with the most recent threat intel.
Unbeatable Time to Value
A typical AI red team requires 1-2 full-time employees, and 3+ weeks of effort. The best-resourced security teams, with dedicated red-teaming analysts trained in prompt engineering, report they are still using manual attacks, painstakingly curated by hand. Teams are limited from running truly robust attacks because of the labor involved in evaluating all the results individually.
With CalypsoAI Inference Red-Team, you can be up and running in minutes, with full results and analysis of our entire library of 22,000+ attacks in as little as three hours. No specialist training or background in AI is required. For custom attacks, simply tell the product in plain English what undesirable output or behavior you are trying to elicit. Agentic Warfare will take it from there. There’s no need to review individual prompts and make judgment calls: our evaluation model does all the work for you.
Automated AI Red-Teaming is Here
CalypsoAI Inference Red-Team is currently available for early access and will be generally available on March 31, 2025. For less than the cost of a single traditional red-team engagement, you get monthly attack updates, Agentic Warfare, security scoring, and in-depth vulnerability reporting for up to 5 models per report.
Power and Simplicity
At CalypsoAI, we’re incredibly proud of what we’ve built with Inference Red-Team. It’s both powerful and simple to use, and helps teams uncover, understand, and quantify the risks to AI systems—enabling secure innovation at scale.
