Overusing robots could lead to bias, panic or just bad advice, officials worry.
The European Commission is worried about the impact of using artificial intelligence (AI) in banking, insurance and securities markets, according to a consultation issued today (18 June).
The EU has just passed a sweeping AI Act, becoming the first jurisdiction in the world to legislate to make the emerging technology safe and non-discriminatory.
But officials are now asking if there’s more guidance needed to make the rules fit into finance – a field that’s already subject to tough regulatory requirements, and where the cost of getting it wrong could be high.
“The EU’s AI Act and existing financial sector rules provide a solid basis to allow for technological innovation,” Mairead McGuinness, the EU’s Commissioner for financial services, said in a statement, calling for people to share their views on a “fast-changing and increasingly important area of technological advancement”.
The new EU tech law set to apply as of May 2025 would apply across the economy, but badges automated tools used in higher-risk sectors such as health, law enforcement, and recruitment for extra monitoring.
Credit checks
Sensitive financial applications like assessing someone’s creditworthiness are also flagged as potentially in need of more detailed laws or guidance.
It wouldn’t be the first time Brussels officials have shown themselves wary about how technology is used in the financial sector.
In 2022, the EU agreed new rules known as the Digital Operational Resilience Act, DORA, given fears that the use by banks of a small handful of unregulated cloud computing providers could lead to data breaches or market panic.
Likewise, the Commission appears worried about the prospect of “herding” – when multiple financiers all rely on the same IT systems to make business decisions.
The resulting exaggerated price swings or market concentration could pose a risk to financial markets—or the robots might just get their answers plain wrong, the consultation document said.
Nonsense?
“General purpose AI … can sometimes produce ‘hallucinations’, i.e. nonsensical or inaccurate replies,” it said.
That’s a phenomenon familiar to anyone who’s attempted to get a straight answer out of ChatGPT – but would also pose problems for robo-financiers, given their legal obligation to give correct advice in their clients’ best interests.
AI could have beneficial effects, the Commission said – for instance spotting dodgy trading activity that might point towards market abuse. But officials also appear worried about poorly formed AI datasets leading to widespread discrimination.
In 201, the EU’s highest court said it was unlawful to charge men more for car insurance. But a pricing algorithm might mean people of a particular gender or race end up paying more, with no human able to explain why.