“Big Data” and Lenders’ Big Data Management Challenges

BigData---ImageBig data” is the new buzz word in information technology circles. Wikipedia defines the term as “a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools.” With challenge comes opportunity, and big data has already been referred to as a game changing opportunity for financial services firms. It is easy to see why.

Banks amass all kinds of data: data on liquidity risk, market risk, environmental risk, credit risk and much more. McKinsey Global Institute’s June 2011 report, titled “Big data: The next frontier for innovation, competition, and productivity,” estimated that U.S. banks and capital markets firms together had more than an exabyte of stored data in 2009. And it has grown exponentially since then—and will continue to do so. As a result, banks, credit unions and other financial institutions are wrestling with the challenge of managing huge silos of data.

These three key trends are driving more interest in data management at financial institutions:

  1. Technology investments are up. The recent financial crisis forced many banks to postpone major efforts to improve data management processes. Now that the worst appears to have passed, many banks are resuscitating these projects again. At many banks, the infrastructures for managing data are antiquated and so the process of managing data is complicated and may involve updating code from decades past. The majority of banks do not manage data centrally, but have different systems that are, in some cases, processing the same data multiple times, adding to operation costs and inefficiency. As financial institutions put their data management plans back into action, they are learning new ways to mine data in ways that are more efficient and strategic.
  2. Risk is in the front seat. Banks are being pressured to comply with a growing number of regulations, including those related to risk management. Examiners are now regular visitors to lending institutions, and enforcement actions are up. Banks need to be more transparent than ever about their exposure to all kinds of risk. New regulations under Dodd-Frank and Basel III have yet to be written so these requirements will only expand, putting stress on bank’s IT budgets to access, understand and store data to prove compliance. Large banks are already struggling with chief financial officers, chief risk officers and other stakeholders to ensure consistent reporting across the institution and to improve the timeliness and quality of data that needs to be reported. This will only escalate as bank regulations become more stringent.
  3. Efficiency is a business necessity. Banks are struggling with how to grow revenue and profits in a market characterized by low interest rates, intense competition for borrowers and uncertainty. This makes efficiency a key business need. Managing the flow of data is the key to efficiency, and efficiency is a strategic edge in any competitive industry. Banking is certainly no exception. As noted recently by Charley Rich, a vice president with Nastel Technologies, an IT firm in Melville, N.Y., “If you don’t do this, your competitors will. You have to master big data or be buried by it.”


The lending industry is undergoing a significant transformation as banks balance the challenges of more intense regulation (need for compliance) and more intense competition (need for efficiency). Access to information is exploding. To address these challenges, banks are going to need much more complex data structures in place and more advanced data management capabilities. Lenders are in the early stages of evolving away from a “data silo” management approach to more institution-wide systems for efficiently tracking and managing all types of data in areas like compliance, business development, marketing and others.

In both its most basic—and most complex—application, big data refers to harnessing the power of data analytics. More complex data management systems will help banks streamline their processes and reduce risk and ultimately reduce operating costs. In the process, the way in which environmental, appraisal and other types of risk-related data are collected, stored and managed will likely get brought into the 21st century and ultimately, help lenders make more informed risk management decisions. Lenders will be able to more easily calculate and understand a bank’s overall risk exposure (rather than just on a loan-by-loan basis), determine whether risk management decisions are being made consistently, assess risk exposure against risk appetite, make decisions about increasing loan originations in a particular market, refine all types of bank underwriting, and ultimately, speed up the lending process.

The trends, both business and technological, point to more integrated risk analysis by financial institutions. Having a centralized data management system is becoming a business necessity. The challenge for the years ahead will be to balance increased regulatory costs and the need for greater efficiency to stay in compliance—and in the market.

NOTE TO READERS: EDR Insight wishes to thank Guy Tassinari, Managing Director of Lender Market, for peer reviewing this Technical Brief.