Machine learning-based real-time threat detection for banks

The business impact of the COVID-19 pandemic continues to unfold worldwide for the financial services industry. The “new normal” has not only given rise to unprecedented operational challenges, but also provided fertile ground for hackers and threat actors to take advantage of increased vulnerabilities.

In June 2020, the Internet Crime Complaint Center at the FBI reported a 75% rise in daily digital crime since the start of stay-at-home restrictions. These cyber-crimes are not only becoming more frequent, but also more difficult to detect and more complicated to prevent. Financial institutions like banks that run hundreds of sensitive customer-facing applications are at extremely high risk.

Adopting a proactive approach

To confront these challenges, banks are fast adopting a proactive approach and introducing smart technology capabilities to strengthen their threat detection perimeters. They are looking beyond traditional, inflexible relational database management system (RDBMs) technology stacks that limit threat detection to a few applications.

There is an increased awareness that simple rule-based alerts are no longer adequate to detect anomalous user behavior accurately and immediately. When applied to thousands of users, static rule-based alerts often generate a high number of irrelevant flags and false positives. What’s more, savvy attackers can bypass the system by keeping their malicious activities within the defined set of rules.

Banks should instead focus on building an always-on, continuous infrastructure that helps monitor threats, gain actionable insights, and respond quickly with confidence.

Enabling faster data ingestion and processing

Machine learning (ML)-based data flow solutions have made it possible to ingest and process data from a large number of applications at an affordable cost. Using network-attached storage systems, fast message queues, and other high-performance features, banks can accelerate data ingestion at a low infrastructure cost.

Adopting such next-gen solutions can help banks process hundreds of millions of events every day across their customer-facing and operational applications. This not only helps expand the overall scope of threat detection, but also helps significantly accelerate the development and production of threat detection applications.

Powering data transformation in real-time

With remote working becoming the norm, employees now have access to sensitive data in home-based environments that are less secure. This has increased the risk of insider threats exponentially. It is therefore vital to enrich event records with relevant employee data (department, role, access permissions, etc.) and details of the applications each employee has permission to access.

Solutions that offer advanced capabilities like in-memory data transformation and distributed in-memory stateful processing also bolster insider threat detection by enabling faster data quality scoring, cleansing, and enrichment. These capabilities, along with data deduplication (for a specified history period), help eliminate false positives, and effectively detect all relevant suspicious activities.

Using ML models on log and complex event data

Recent advances in ML have helped create dynamic models that periodically learn normal baseline behavior and detect anomalies based on both dynamic and static factors such as identities, roles, and excess access permissions; correlated with log and event data. Models developed using built-in ML operators include self-learning and training behavioral profile algorithms that help process each new transaction in real-time to build risk scores and dynamic thresholds for various risk factors.

Using ML models on the log and complex event data can help reduce false positives from thousands to tens per day and make the end-to-end process of identifying suspicious behavior automated, accurate, and timely.

Custom alerts to curb fraud in real-time

To help prevent predicted breaches, banks are now focused on enabling appropriate real-time alerts and actions. These include routine rule-based alerts like off-hours activity, multiple-failed logins, multi-station logins, and custom alerts for ‘suspicious’ activity (based on a complex mix of factors deduced by the ML algorithms), which can be manually validated by security experts. New-age data solutions that deliver such alerts help detect fraudulent activities that often get overlooked by traditional single-point monitoring solutions.

COVID-19 has thrown a curveball to the financial industry like never before. With an ever-increasing list of compliance mandates to navigate, most banks are fast adopting new tools and technologies to ensure accurate and timely threat detection in these unprecedented times. The right ML-based technology stack coupled with a proactive approach can help them build a resilient enterprise and minimize major risks like data/productivity loss, business disruption, and reputation damage.

Reprinted with permission from Datanami

Recent Posts

View more posts


50X faster time to value with Confluent and Gathr...


Data + AI Summit 2023: A must-attend for data scientists,...


Move away from batch ETL with next-gen Change Data Capture


ETL vs ELT: Which data integration practice is right for...