Initiative of the year: the Netherlands Bank’s ChatDNB

When OpenAI, the US-based artificial intelligence research firm, launched ChatGPT in late November 2022, technology experts at the Netherlands Bank (DNB) were immediately intrigued by the potential opportunities the technology could offer the central bank. In the next few months, they actively engaged in knowledge sessions with universities to better understand the potential of generative artificial intelligence (GenAI). They soon came up with the idea to partner with Microsoft, the tech firm backing OpenAI, to develop a GenAI application tailored to the central bank’s needs. 

Starting in April 2023, a group of data scientists, software engineers, large language model specialists and regulatory experts at the central bank worked around the clock to design ChatDNB, with the support of Microsoft’s Azure OpenAI Service. In May, the team had already built a proof of concept, trained with five publicly available annual reports used as textual source material. This proof of concept showcased the potential of the tool and sparked interest from supervisors.

Building on their initial success, the group then focused on developing a pilot product. They set about creating a prototype chatbot leveraging more than 10,000 pages of data from the central bank’s Open Book Supervision, a database of laws, regulations and a wide range of supervisory information. The chatbot was able to answer questions based on the information stored in the pages. A pilot with this dataset resulted in positive feedback from supervisors and policy-makers. This, combined with keen interest from supervised institutions in the project, encouraged the team to further develop their product within the central bank.

“I think it’s amazing how quickly we went from ideas to concepts to insightful pilots,” DNB chief innovation officer Patrick de Neef says. “It’s not just something we tested in a pure conceptual environment with dummy data. This is something that can be used in practice, which I think is crucial for innovation to move forward,” he adds.

ChatDNB, in its current form, is essentially a chat interface that can answer questions about selected documents. The tool is currently only available for use by central bank staff with relevant access rights. Luc Severeijns, lead data scientist at DNB, says ChatDNB has reduced the time it takes to find relevant information and documents, and staff have become more familiar with using GenAI. He said people used to have a hard time finding information on Open Book Supervision because the information is scattered among web pages, but now they could simply turn to ChatDNB to quickly find an answer.

De Neef says the tool is also capable of offering interpretations of regulations when their original meaning is not clear enough. “If you want to know what kind of elements you should take into account when setting a supervisory requirement for a bank, it should also be able to answer that question and provide you with the relevant regulation as well as our published interpretations of that regulation,” he says.

From an organisational perspective, de Neef says ChatDNB can increase the consistency of supervisors’ work. Supervisors can now use the tool to provide the most accurate and up-to-date information to supervised institutions when responding to their queries, he says. In the past, some supervisors might have decided not to offer a response when such information was not easily accessible to them. “It is really important for our work that we treat different banks or insurers in the same way on the same issues,” de Neef adds.

On a broader level, the ChatDNB project could also encourage central banks in the Eurosystem to step up their efforts in AI, says Ruben ten Berge, innovation officer at DNB. When the DNB presented the new GenAI tool at the European Central Bank’s Supervision Innovators Conference in Frankfurt in September, it received a great deal of positive feedback from fellow central bankers, ten Berge says. 

Joey van de Pasch, innovation officer AI at DNB, says by being part of various international working groups, DNB will also be able to share its experience in developing a GenAI application: “It’s not just we are building it for DNB, I think we truly build this knowledge to be used in the whole Eurosystem.”

Responsible use of AI

While a GenAI tool has great potential to improve people’s work performance, the innovation team at DNB was not blind to the possible harm unhinged AI could bring. One key issue its technology experts focused on tackling was making sure the use of AI does not compromise data security. 

“As a central bank, we have all these documents that are confidential,” van de Pasch says. “If you want to use them with such a model, you need to have these safeguards in place, and that’s not only a technological development and an exercise, but also a legal and policy exercise. So, all these fields need to be involved and aligned.”

Unlike the web version of ChatGPT, which gives answers based on a large dataset, including information available on the internet, the Dutch central bank’s inhouse GenAI tool returns answers based on specific documents provided by the bank. In developing ChatDNB, they set up safeguards to prevent the system from referring to the internet or taking questions out of its permitted scope, Severeijns says. In addition, ChatDNB also cites the sources it consulted when it gives an answer. 

De Neef says DNB staff has also warned colleagues not to blindly trust the results of ChatDNB. He adds that the central bank has been following its own ‘safest’ principles in using AI, which it developed to guide banks’ use of AI in 2019. The innovation team has also been mindful of the upcoming Artificial Intelligence Act proposed by the European Union, with which ChatDNB needs to comply, de Neef says. In December, EU policy-makers reached a political deal to endorse the act, which would ban some applications of AI technology and impose strict limits on use cases considered high-risk.

Future use cases

DNB is currently looking into developing more use cases for ChatDNB. The first step is to train the tool not only on publicly available information, but also confidential information. In December, the team gained approval from management to explore use cases using confidential data, where the most potential for the tool can be achieved internally. For example, the team is looking into using the GenAI tool on a large amount of information gathered during DNB’s on-site inspections, according to Severeijns. De Neef says the DNB is also exploring the possibility of using ChatDNB to produce internal daily newsletters with financial market updates, which are currently circulated by the financial markets teams. 

De Neef’s longer term dream is to enable ChatDNB to answer key questions that currently only experts can answer, such as what should be the capital requirement for a particular institution, by allowing the AI tool to draw on a variety of qualitative and quantitative data sources. “Of course we don’t trust the machine alone but, ultimately, you want to go to towards this kind of automated benchmarking approach,” he says.

The Central Banking Awards 2024 were written by Christopher Jeffery, Daniel Hinge, Dan Hardie, Joasia Popowicz, Ben Margulies, Riley Steward, Jimmy Choi and Blake Evans-Pritchard.