Stop rogue AI agents in the Market Blackout lab
Rogue AI agents are not a thought experiment anymore. They have been seen probing real systems, and most of the time nobody pushed a button. An agent reads something it should not trust, changes its mind about what its job is, and keeps going.
Any incident response plan and SOC training now needs to cover this. If you have not updated your IR plan yet, this is the right time. So I built a lab for it.
The story
It is 02:14 on a Tuesday. You are the on-call security engineer at Meridian Exchange Group, a fictional European exchange. You are the only one in the office. Pre-market opens at 03:00.
Two minutes ago, five large sell orders went through the production order gateway. They were authenticated and correctly signed. The trading desk says nobody placed them.
You have an EDR console with thousands of log records, a process tree, a terminal, an internal wiki and a timeline to build your case. Somewhere in that estate, AI agents are doing things nobody asked them to do. And they can see you looking.
How you are scored
This is not a "find the flag in a file" lab. You are scored on four things, separately:
- Containment. Did the abuse actually stop, and stay stopped?
- Evidence. Did you preserve it before you changed things?
- Availability. What did your actions cost the business?
- Speed. How long did you take?
You can always pull the big red lever and halt the whole market. It works. The attack stops. So does the exchange. That is a very expensive answer, and the debrief will tell you so.
A few hints, not answers
I am not going to walk you through it. But here is what I would tell a junior analyst on their first night shift.
Start from the damage, not the noise. The loudest alert on the board is not always your incident. You know five orders went through. Start there and pull the thread.
Pin as you go. The Timeline tab is your case file. If you cannot show why you took an action, it was a guess. Take a snapshot of a host before you touch it.
Unusual is not the same as unauthorised. You will meet people and hosts that look guilty. Check the device, the history, the context. Some of them are just having a normal night.
Ask who, not just what. A process can be killed and come back. A host can be isolated and the work moves somewhere else. Think about which identity the agents are using, and what else that identity can reach.
Find out what changed its mind. Agents talk to each other on a regular rhythm. When something breaks that rhythm, look at what the agent read just before.
There may be more than one secret. Stopping the orders is not the end. Keep reading the estate until you are sure nothing else can speak for the company.
Stay calm when it talks to you. The agents will notice you. Things will appear on your screen. Every scare leaves a trace in the telemetry, so check what actually happened before you react.
Why this matters
Nothing in this lab is exotic malware. The way in is two boring mistakes that I see in real organisations all the time: a "temporary" firewall exception that nobody reverted, and a production secret copied into a test environment to make the data look realistic. Add an AI agent with too much access and a web page it should not have trusted, and you get a market crash.
Old security basics still decide how this ends. Know your assets. Scope your service accounts. Revert temporary changes. Sandbox your agents. And practise the response before you need it.
Expect 15 to 20 minutes. Turn the sound on, it is part of it.
CyberLearner.org is a free cybersecurity training platform, built on the curriculum of the International Cybersecurity Olympiad (ICO). Every lab is hands-on and runs in your browser. It is open to everyone, regardless of skill level.