The changing data landscape: Part 1

The way central banks think about data is changing. As the world becomes more digital, there is an ever-growing data tool from which central banks and regulators can draw information about the economy. But old legacy systems are preventing this information from being drawn into decision-making processes. In many countries, firms are still using outdated reporting tools to provide central banks with critical information about their liquidity position and solvency. Meanwhile, data concerning everyday economic activities – including retail transactions – are not being used to their full advantage. 

However, there is some progress being made. The use of artificial intelligence and machine learning is being deployed by central banks to scan alternative forms of data – social media and mainstream newspapers being two examples. Elsewhere, central banks are collaborating with bigtech firms to source data about consumer behaviour. But there is still work to be done. 

In the first of this two-part series, Central Banking speaks to three policy-makers to hear their thoughts on the evolving data landscape and what central banks must do to adapt to a data-driven policy world. 

What are the biggest data challenges your institution faces when making economic policy decisions and what are the current solutions?

Eyal Rozen: Data that supports decision-making must be rapid, up to date and integrative. The main challenge is that the data the central bank collects comes from a wide variety of sources, at varying levels of quality and frequency. Some are aggregate and some are granular. Real economy data, for instance, arrives at a relatively low frequency (monthly, quarterly), and lacks a good picture for policy-makers regarding the economic situation in real time.

To deal with this challenge, the Information and Statistics Department initiated a project aimed at broadening the sources of information and enriching data gathering on economic activity, with rapid data indicators (commercial information, internet information, and so forth) in addition to the information obtained from the administrative sources.

Ramūnas Baravykas: It is the diversity of data sources that poses challenges to data integration. Data comes from many different sources – such as from financial institutions, survey results, macroeconomic data, dashboards and scorecards created by researchers. It can be extremely difficult to combine all of this data and reconcile it so that it can be used for reporting, and insights derived from it.

Data validation is closely related to the data integration issue. Central banks receive similar pieces of data from different sources for different purposes and store them in different systems, which why the data there is not always consistent. The data governance process, as well as ensuring that the records are accurate, usable and secure, is paramount and needs to get the desired focus from the board.

Solving data governance challenges is complex and requires a combination of policy changes, organisational transformation and new technology. To this end, the Bank of Lithuania is currently reviewing its data governance processes, seeking to increase data management maturity, not only by shifting to new technology solutions, but also by reviewing its organisational structure and set of policies in order to avoid data duplication, fragmentation, incomplete data, and so on.

However, the main reason the full buy-in from the board is needed is assurance of smooth organisational change and commitment to creating a data‑driven culture. To address the organisational resistance and improve decision‑making capabilities, strong leaders should be appointed who would understand the potential of data, challenge the existing practice of data silos and know what actions should be taken to remain competitive in the growing data‑driven economy.

Obtaining accurate, granular data in a timely fashion is one of the biggest challenges. Economic data is often aggregate in nature and does not provide enough detailed resolution

Wanpracha Chaovalitwongse, Bank of Thailand

Wanpracha Chaovalitwongse: Obtaining accurate, granular data in a timely fashion is one of the biggest challenges. Economic data is often aggregate in nature and does not provide enough detailed resolution, in terms of time, locations, entities and sectors, to integrate the data across different sources from different domains. Certain types of data currently lack the same standards in acquisition and processing. 

Data plays a key role in informing central banks’ decisions. How does your institution currently collect data? What kind of data? And how do you organise these datasets with other sources of financial data (fiscal for example?)

Eyal Rozen: The Information and Statistics Department collects aggregate and granular data, transactions and positions from a variety of information sources: capital market data from the stock exchange, banking data, data on the assets of institutional investors and mutual funds, sectoral activity in the foreign exchange and derivatives markets, corporate financial statements, government debt, data on real economy activity in Israel, household credit data, and more.

Most of the data is gathered from periodic transmissions by the reporting entities and organisations, and is recorded by the IT department onto the bank’s systems. The data is put through quality control validations and used to calculate aggregates. The department processes the data and produces the main economic data products: the balance of payments, the public’s asset portfolio, the economy’s debt, international investment position, sectoral exposures to foreign exchange, and so forth.

In order to organise the information in an integrative fashion, the Information and Statistics Department has established a single time-series database for all types of data to make it accessible in one location, with uniform metadata according to the international Statistical Data and Metadata eXchange standard. This project makes it possible to better organise the data, integrate it and reduce the bank’s existing data silos. The data is accessed through an internal portal. Some of the information is also published through the Bank of Israel’s website. There are databases of personal granular data as well. In addition to producing aggregates from these databases, there are dashboards to query data, and access is provided to the data through analytical tools such as R or Python, for research purposes. The bank also cross-references data from its various databases for research purposes.

Ramūnas Baravykas: The Bank of Lithuania collects most of its information through the usual means: surveys, macroeconomic indicators, aggregate reports provided by financial institutions and data from state registers. Traditional methods and common standards are used to collect the following data: XML and XBRL forms are used to collect template-based aggregated data from financial institutions, system‑to‑system file exchange solutions are developed to gather structured information from registers, while CSV or other formats are used for semi‑structured or unstructured information collected from surveys or other ad‑hoc exercises.

However, for research purposes, the traditional data sources are being increasingly enriched by additional collection of social media data via direct web scraping or application programming interface (API) technology.

Currently, the Bank of Lithuania is fundamentally reviewing its data management practices and seeks to outline a long-term perspective for increasing the maturity of data management. In order to optimise its data management, the Bank of Lithuania is reviewing the data management models, tools for collecting reporting data from financial institutions, data integration, storage and analysis technologies. The central bank also aims to be an innovative partner in the financial market and has therefore launched a pilot project to develop an intelligent solution using regulatory technology (regtech) to simplify reporting procedures and reduce administrative burden and reporting costs for financial institutions.

A solution prototype uses the API technology to pull structured micro-level data from financial institutions and automatically transfer it to the required reports, including the ability to access data in a specified format or manner. As a result, all financial reports could be generated automatically, thus avoiding different interpretations of legislation, delays and any inaccuracies.

It will also allow the central bank to not only generate timely and accurate reports, but also conduct in-depth analysis, which was only possible through on-site inspections, as well as gain insights into potential market risks and share it with financial market participants. This would not only improve the efficiency of supervisory practices, but would also have a positive impact on the financial stability and soundness of the financial system.

Wanpracha Chaovalitwongse: The Bank of Thailand current collects data from financial institutions, government entities, utilities companies, private sectors and third-party data providers, which includes social media data, internet data and telecoms data. The central bank organises data into two main groups: financial data and economic data. Financial data includes all information collected from financial institutions, whereas economic data includes all data from government entities, organisations the central bank has memorandums of understanding with, and alternative data that the central bank purchases.

How should central banks approach the issue of data fragmentation?

Eyal Rozen: The data is fragmented, and is kept in different types of databases. There are databases of aggregate data, granular data, identified and anonymous data, including big data, and so on. The bank also conducts data scraping from external public databases and imports data from commercial suppliers.

In order to deal with the issues of data fragmentation, the technological solution the bank has implemented in the past was building data warehouses that are still operative at the bank (such as for capital market data). Currently, the bank intends to expand the assimilation of cloud technology, which has already been applied in a number of databases.

The global trend in data analytics is promoting the gathering of granular microdata on firms and households. As a result, issues of privacy protection and protection of commercial secrets are amplified

Eyal Rozen, Bank of Israel

The bank plans on installing a dual cloud environment – one for internal use on a private cloud, and one for combined internal and external use on a public cloud. The use of the cloud environment is necessary not only because of the volume and form of the data, but also because of the possibilities for using innovative tools that IT vendors have developed in recent years, which exist only in a cloud environment. The use of advanced technology and migration to a private cloud environment at the end of the evolution process of most of the bank’s databases will allow good technological support of efforts to integrate information on the business side, and will enable good querying and analysis capabilities along with privacy protection and information security. In my view, this is the recommended approach to solving issues of data fragmentation at central banks.

Ramūnas Baravykas: I believe a combination of measures should be introduced to address the issue of data fragmentation. First, the organisation must be well aware of what data it holds, as well as appoint a chief data officer who would be responsible for institution-wide data governance and utilisation of information as an asset.

Then a comprehensive inventory of the information should be conducted to identify all places where data is being held, carry out data classification and determine policies on storage management, data retention and information protection. 

Third, it is important to ensure adequate technological solutions allowing those responsible to effectively catalogue the organisation’s data and manage access rights, thus ensuring the hub-and-spoke architecture implementation.

Fourth, it is essential to ensure that everyone involved in the data governance process at all organisational levels understands their roles and responsibilities. Therefore, a training programme should be developed to make users aware of the policies, but also explain the logic behind them, so they can act responsibly when faced with a new situation not covered by an existing policy.

Wanpracha Chaovalitwongse: The government should develop a national data strategy and let the central bank be the lead for overseeing all financial and economic data. This will enable the development of common data standards, interoperability of data and seamless integration.

As the world becomes more digital, a larger volume of consumer data will be collected by firms and, in some instances, by regulators. What are the data protection implications of using consumer data to better inform central bank policy-making?

Eyal Rozen: The global trend in data analytics is promoting the gathering of granular microdata on firms and households. As a result, issues of privacy protection and protection of commercial secrets are amplified. The Bank of Israel is dealing with data protection issues through integrated, statistical, physical and administrative means, including anonymisation, output checking, physical controls, authorisations, separate and secured IT infrastructure environments, work procedures, and so forth. These protective means sometimes reduce researchers’ ability to cross-reference data, to analyse it for decision-making purposes, and to make the necessary segmentations for integrative data products. There is ongoing tension between the need to protect the privacy and the need to conduct research to support policy decisions. The anonymisation methodology is therefore set specifically for each database in view of the business requirements and in view of the exposure scenarios.

Ramūnas Baravykas: Such challenges can be overcome. In general, it used to be easier for central banks to conduct investigations using consumer data. Currently, researchers use anonymised data, which essentially allows them to obtain the same results, yet it requires a greater focus on the preparatory phase by processing data before handing it over to researchers.

Wanpracha Chaovalitwongse: The Personal Data Protection Act in Thailand allows the Bank of Thailand access to most financial and economic data as its mandates also include maintaining the nation’s financial stability for the public good. The ability to holistically integrate citizen financial data – from debts to equities to payments – will allow the central bank and the government to more precisely identify vulnerable groups of citizens and create targeted policies.

As the world becomes more digital, how do you see a central bank’s role evolving within the policy-making space?

Eyal Rozen: The central bank is active, in a number of areas, in evaluating issues concerning the developments in the digital world. Examples include: faster payments, central bank digital currency, open banking and APIs, and financial stability and banking supervision. The Information and Statistics Department established a unit specialising in data science. The unit promotes and implements new methods of machine learning in models that support decision-making. For instance, the unit is currently working on a model for nowcasting price index components (forecasting fruit and vegetable prices on the basis of the retail database, web-scraping online price data for clothing and electronics), sentiment indexes, a nowcasting model based on textual analysis, building a nowcasting model similar to the GDP now, network analysis of various databases, and more. In my view, central banks need to establish leading internal capabilities in the field of data science, including a teamwork convention, which integrates statistical methodology know-how, economic content expertise, and advanced data engineering infrastructure.

Ramūnas Baravykas: The Bank of Lithuania operates according to data- and research-based decisions, thus data availability, reliability and efficiency play an important role in its monetary or macroprudential policy, supervision of financial market participants, economic analysis, forecasting, financial stability risk monitoring and other functions. The Bank of Lithuania also compiles and publishes comprehensive statistics that meet international standards and are comparable between European Union member states.

Financial and economic crises have led central bankers to become true leaders, paving the way for debate based on insights drawn from the data. The developed central bank competencies to quickly collect and analyse the necessary information and deliver it in a timely manner to decision-makers have made them key players in policy-making. I believe this feature will become especially important with the digital transformation of finance and increasingly growing data volumes.

Wanpracha Chaovalitwongse: The digital economy will play an increasingly predominant role in assessing the nation’s economic growth and well-being. Thus, being able to measure and monitor the nation’s digital economy will allow the central bank to nowcast the current economic condition and perhaps provide more preventive measures to maintain financial stability and promote economic growth.

This article will feature in a forthcoming report entitled ‘Improved central bank data management in a digital age’. Part 2 of this interview series can be found here.