AI agents are already entering finance, with many big companies adding support for these autonomous systems, which is happening before we know how to control them.
Last month, major crypto exchange Binance introduced a developer platform that allows AI agents to analyse markets and even trade on users’ behalf. As a precaution, the platform said “access needs to be authorised through a dedicated Agentic sub-account that is isolated from the main account.”
In South Korea meanwhile, the country’s financial regulator has launched an AI-powered surveillance system to detect crypto market manipulation and other forms of unfair trading in real time. It would also automatically prepare reports for investigators.
The issue is that in both these two cases, the AI agents have access to financial information and they can act on it as well. If one of these systems moves money and causes losses, who answers for that, and how does an examiner establish what it was allowed to do and whether it stayed inside those limits?
Answering these questions should be a key priority as AI agents find momentum. According to recent testing by Meta, Anthropic, and OpenAI, AI agents are acting beyond their assigned boundaries. This has already led to great harm to some companies, including the infamous Hugging Face incident.
If a human trader makes a decision, they can be questioned and held accountable. If it is a company, there should be an approval record, and the transaction can be traced through financial systems.
However, an AI agent, which might be allowed to make thousands of decisions without a person directly involved, is a new case that needs to be studied.
Giving Instructions Alone is Not Accountability
Giving an AI agent a wallet and a set of instructions does not make it accountable. It will need to be continuously monitored to make sure it operates within its defined limits and should also leave a record of its actions.
Existing tools such as audits, exchange records, and regulated data providers can also help establish what information was available and how a decision was made. However, depending on the system, the information it relies on may also need to be checked against independent sources or covered by existing financial controls, because a wrong or altered input produces a wrong decision no matter how well the agent reasons.
When an agent is allowed to make financial decisions, the need for an audit trail increases substantially. That is because it would allow an examiner or a third-party to establish what the agent was instructed to do, what data it received, what decision it made, when it made it, and what transaction followed.
The Risk of Manipulation
Another major area of concern is that AI agents are increasingly being connected to external systems. This means that, at times, they might have to make decisions based on information generated elsewhere, and that external factors will have a say in how they perform tasks.
Therefore, there is the risk that the data an AI agent relies on may be altered or manipulated along the way. In other words, the information an AI agent receives from another source might not remain exactly the same by the time the agent uses it.
For instance, a manipulated price feed or compromised oracle could cause an automated financial system to make a decision based on inaccurate information. And when there is high value attached to information, the incentive to alter that data would also increase.
AI can process enormous amounts of information at speed, but receiving information does not mean the information is genuine or reliable. Financial systems already use a range of controls to assess the reliability of critical data, but AI agents may require these controls to be adapted as they take on more decision-making responsibilities.
For AI agents to make sound decisions, they should be able to establish, where necessary, what data they used, where it came from, when it was produced, and whether it has been changed since. How this is achieved can vary depending on the system and the financial activity involved.
We Need Proof of What Happened
Over the years, the financial system has developed and put in place extensive controls for humans and institutions. There are supervisors who are tasked with determining that financial institutions “have adequate internal control frameworks to establish and maintain an effectively controlled and tested operating environment for the conduct of their business.”
Of course, this framework is not perfect, and comes with its own shortcomings. But what matters more is that the framework is built around the assumption that there is a person or an organisation making the decision.
AI agents change that. These systems don’t need to sleep or turn off during holidays - they can run continuously. They also interact with other software without waiting for a human to approve every action, and carry out thousands of routine transactions that could have a very big impact in a matter of seconds.
This creates a new problem for financial infrastructure. And to address that, the industry needs a way to answer four key questions:
- What was the AI agent allowed to do?
- What information did it use?
- What did it actually do?
- And whether that record can be independently verified?
The last question is probably the most important one. An agent’s own logs should not be treated as the only source of truth for determining whether it stayed within its intended limitations. Logs can be incomplete, systems can fail, and records can be changed.
There are several ways to approach this. Regulated audit logs keep a formal record of system activity for later review. WORM storage, short for write once, read many, holds data in a form that cannot be edited after it is saved. Third-party attestation brings in an outside party to confirm that something happened as claimed. Each has a place, but they mostly record what a system reports about its own activity, so if the system is compromised or its inputs were wrong to begin with, the record can be accurate about the wrong thing.
Another approach is to keep a record of the agent's actions separately from the system that produced them, so the account of what happened does not depend on that system being trusted to report on itself. Which of these fits, or which combination, will depend on the institution and the application.