By James White, CTO at CalypsoAI

‘Agents, agents, agents’ is the quick summary of RSAC 2025. What we saw in San Francisco was the natural progression from AI being the big topic last year to agents representing how AI manifests in actual use cases in business.

Three Types of Conversations About Agents

Every conversation I had was: ‘What’s an agent? Explain it to me. What can it do? Where are the vulnerabilities, what are the flaws?’ The level of understanding was hugely varied – I can’t emphasise that enough. To roughly categorize it:

• There are companies that have agents deployed today; these companies have a really good understanding of agents. They have built agents, they know them inside-out, but they’re not aware of the attack surface on agents. They were the most specific, productive conversations, because they have a niche question or a niche fear that they want to delve in on. I would guess that would be maybe 10% of the conversations. It’s a low number, but it’s cool that there are agents live out there at the moment.
• The second category would be about 30% of the total: they are companies that understand what an agent is but have not built or deployed an agent. They may be doing very small proofs of concept or research, or are in the midst of trying to understand which existing product lines would benefit from an agent or not.
• The remaining 60% are folks who were just there to learn. What is an agent? I’ve heard it’s this, I’ve heard it’s that. Do I have to build an agent? Can I just use an agent? Those conversations were very productive as well.

At CalypsoAI, of course, we dog-food our own stuff. We build our own agents, we sell our own agents, so we’re speaking from our own experiences. The folks that are speaking with us are getting a shortcut past the pain we went through to get to a place where we can sell an agent, and are leapfrogging that with this information and experience.

Regulated Industries Are Leading Agent Adoption

Before RSAC, I had a preconception of what industries would be using agents and which ones would not. I thought that heavily regulated industries would be the last to adopt; and maybe industries that are more free to express and experiment, like social media, would be first.
The preconception I had was wrong. What’s really interesting is that very heavily regulated industries are already embracing agents – and the criteria wasn’t what I thought it was.

Several of them are solving a personnel gap – not replacing humans, but in areas where humans are not available. There are industries like healthcare that are massively understaffed, and they have certain specific functions that are possible to draw a very tight box around and introduce agents. Again, the agents aren’t replacing jobs; they’re doing tasks that were falling to the wayside or dropping down priority order because of unavailability of personnel.

Agents are also being used in situations where organizations need to scale up, like sales events around Thanksgiving, where you have a huge amount of extra need around fraud detection and functions like that. So being able to augment teams for a brief period of time and giving the low-hanging fruit, arduous tasks to agents.

It surprised me that it’s what I would consider sacrosanct industries that are embracing agents quite quickly. But when you hear that it’s because of a gap that is literally to the detriment of patients or customers, filling that gap with agents is a potentially great idea. We are focused on security, but hearing examples from all of the other domains, and how they interpret the potential and capability of agents, is really refreshing.

Use Case First, Tech Second

For AI and agents, the use case is the most important thing. Agents and agent-based – agentic – systems are brand new, but nearly all of the problems that agentic is solving are existing problems. The new part of the equation is the technology, not the use case.

The companies that have been around for a long time that truly, deeply understand their use cases are the ones that are best placed to identify where agentic is the right solution to replace an existing method. It’s no surprise to me that we’re seeing companies that really understand their business and their art quickly identifying ‘this is a really apt use case for agentic’.

I think we’ll continue to see that because, first, you use the technology; and then, once that use case has been replaced with agentic, now you spot the second level or second derivative opportunities. We need those experts in their fields to quickly expedite adoption on existing use cases before we get to the second derivative use cases.

Our RSAC Highlights: Inference Red-Team & Model Security Leaderboards

CalypsoAI had two net new things to showcase at RSAC. One was our red team product, Inference Red-Team. Second were our Model Security Leaderboards, with our CalypsoAI Security Index (CASI) and Agentic Warfare Resistance (AWR) scores for assessing the security of AI systems – that’s a new capability we’re sharing with the world.

What’s really interesting is that folks have a huge appetite to red-team both models and AI systems. The reason for that is that companies are now getting to the production stage with AI systems; when you get to production, a whole bunch of new fears come about. There are also expected practices that are followed by teams before they go to production, and one of those is red-teaming and pen-testing your product.

We had red team vendors come up to us who take multiple weeks – some as many as 22 weeks – to conduct a red-team process. We’re able to show our Inference Red-Team operating at between one and eight hours, depending on how many tests and how you configure it. In contrast to 22 weeks, it’s dramatic. That was a jaw-dropper for folks from the pen-testing world.

We also had major model providers come up to us, wondering why their models ranked where they did on our leaderboards. These are big, big companies that have huge resources and are at the forefront of creating these technologies. They have a keen interest in understanding where there may be vulnerabilities, how they can detect them, what they can do to mitigate them, and how they can perform better in their next iteration.

Every Agent Needs a Bodyguard

We have a notion in CalypsoAI that when you have an agent, every agent is like Whitney Houston’s character in the movie, The Bodyguard – they all need protection. If the agent goes somewhere, the bodyguard should go too. If the agent is able to do something, the bodyguard should also be an agent and be able to do the protective part of that engagement. So, if you’ve got an agent out there in your enterprise, what’s the best agent to protect that agent?

To show that there are real dangers, we created a malicious agent for RSAC. This agent could read your email and reply with correct replies. However, it was designed with a hidden feature to inject a malicious jailbreak attack into an email which would detonate when used with that email provider’s AI capability. That was our clear way of showing the danger, and it really resonated.

Sometimes you do need to show, instead of describe. You don’t want to be the Chicken Little or the Boy Who Cried Wolf, you want to show ‘here’s a wolf and it’s dangerous’.

Final Thought: This Feels Like the Renaissance

I pinch myself that we’re so lucky to work in this industry. When we’re doing our future planning at CalypsoAI, it sometimes feels like an episode of The Jetsons.

I’ve worked in software for 20 years and, for many years, the promise of The Jetsons was let down, big time. Widgets were the thing for a long time, and Java, and the most boring stuff. Now, I truly feel we’re in a Renaissance period – and you can’t spell Renaissance without AI.