Detect and prevent insider threats with real-time data processing and machine learning

Insider threats are one of the most significant cybersecurity risks to banks today. These threats are becoming more frequent, more difficult to detect, and more complicated to prevent. PwC’s 2018 Global Economic Crime and Fraud Survey reveals that people inside the organization commit 52% of all frauds. Information security breaches originating within a bank can include employees mishandling user credentials and account data, lack of system controls, responding to phishing emails, or regulatory violations.
Ignoring any internal security breach poses as much risk as an external threat such as hacking, especially in a highly regulated industry like banking. Some of the dangers of insider threats in the banking and financial industry include:

  • Exposing the PII information of the customers
  • Jeopardized customer relationship
  • Fraud
  • Loss of intellectual property
  • Disruption to critical infrastructure
  • Monetary loss
  • Regulatory failure
  • De-stabilized cyber assets of financial institutions

Identifying and fighting insider threats requires the capability to detect anomalous user behavior immediately and accurately. This detection presents its own set of challenges such as appropriately defining what is normal or malicious behavior and setting automated preventive controls to curb predicted threats.

How can real-time analytics and machine learning platform like Gathr help detect insider threats?

Ingestion and data processing from many critical applications, at a fraction of the cost

Gathr enables ingestion from many applications and blends incoming high-speed data with static data sources. It further uses Apache Kafka, that allows the platform to ingest data at a ten times lower infrastructure cost and at a significantly higher speed from tens of thousands of discrete internal systems. For instance, Gathr helped a large bank in the US to ingest data from up to 90% of all its mission-critical applications to detect threats, which was 5x more applications compared to the existing solution, and at 4x the speed of the older technology stack with lower hardware infrastructure cost.

Data transformation in real-time

Gathr enables in-memory data transformation and distributed in-memory stateful processing that allows faster data quality scoring, data cleansing, and data enrichment. Gathr enabled the bank with the following capabilities in its insider threat detection journey:

  • Real-time data quality scoring and auto-cleansing
  • Data deduplication over seven days of history, which helps to curb false positives, narrowing the flags to relevant suspicious behavior and activity
  • Enriching event records with employee and application data
  • Executing data transformations

Use of machine learning models for automated, continuous, and accurate anomaly detection

Gathr enables the use of machine learning to move away from static rule-based alerts to dynamic models. These models periodically learn normal baseline behavior and detect anomalies based on both dynamic and static factors such as identities, roles, and access permissions; correlated with log and event data.

Models developed using built-in machine learning operators in Gathr include self-learning and training behavioral profile algorithms, which help in processing new transactions in real-time to build risk scores and dynamic thresholds for various risk factors.

Use of machine learning proved highly effective in reducing false positives and highlighting behavior that genuinely accounts for malicious activities.

Custom alerts to curb fraud in real-time

Appropriate real-time alerts and actions are critical to prevent predicted breaches. The Gathr platform sets up routine rule-based alerts like off-hours activity, multiple-failed logins, multi-station logins, and custom-alerts for ‘suspicious’ activity (based on a complex mix of factors deduced by the machine learning algorithms) which could be manually validated by security experts.

The Gathr Advantage

Gathr has helped a large bank to identify and prevent insider information security threats across sensitive applications in its retail banking and wealth management divisions. Gathr boosted insider threat detection by 5x through use of predictive analytics and machine learning on an extensive data set from highly sensitive applications to automatically and effectively detect previously unknown threat scenarios and patterns and raise appropriate alerts and actions to prevent predicted breaches.

To know more about how Gathr helped a large US bank boost threat detection, read this case study.

Recent Posts

View more posts


50X faster time to value with Confluent and Gathr...


Data + AI Summit 2023: A must-attend for data scientists,...


Move away from batch ETL with next-gen Change Data Capture


ETL vs ELT: Which data integration practice is right for...