Financial Crime Investigations – Insights from working with Financial Crime Experts – Interview with Chris Brannigan.
Chris Brannigan is the CEO of Caspian, a Nasdaq Development Partner. In this interview with Financial Crime News, we explored new findings published in a research paper, which provides unique insights into how financial crime experts investigate alerts, and what this should tell financial crime leaders about the future of such investigations.
The research is not theoretical but has been generated from work carried out in Tier 1 Banking. By sharing the broader findings from the deployment of machine learning technology, where financial crime subject matter expertise has been leveraged, Chris’ team are providing a feedback loop back to financial crime SME’s everywhere to support a virtuous circle of continuous learning.
Across the financial services industry, financial institutions are struggling to right size and optimise financial crime controls. An area that remains the focus of many is transaction monitoring, which is both a fundamental core control as well as a continuing challenge. As FIs trawl as they must their data for unusual and or suspicious customer activity, alert investigation has become an industrial sized challenge that presents significant risks if and when firms are seen to get things wrong. Getting things right at scale and consistently can only be achieved by thinking differently.
One such thinker is Chris Brannigan.
Question 1 – FCN: Are Transaction Monitoring Alert Investigations still a challenge for Banks?
CB: Improving how Banks tackle thousands upon thousands of alerts that are generated from automated transaction monitoring engines is definitely one of the biggest challenges the industry faces. Responding to this in the context of the environment Banks find themselves in, with strict board and regulatory oversight and tight risk appetite compounds the challenge.
Many firms have sought to address this challenge through optimising their processes through which risks are assessed and investigations are carried out. Some firms have gone beyond improvements in the process itself and are heavily investing in training for investigators, setting up teams with supervisors, career paths for developing expertise, and quality control and assurance teams in support. These are viable solutions but only get financial institutions so far.
Question 2 – FCN: What do you believe can be done?
CB: Machine learning technology is the potential solution to deploying the insights and knowledge of expert investigators on every investigation. Instead of experts being focused solely on training other investigators, we can use their knowledge to train a machine that can augment and automate human investigation performance. This force multiplies human investigation teams and delivers operational improvements to the effectiveness and risk management of investigations.
The barrier to date has been the difficulty in extracting those insights from the heads of expert investigators. To achieve this, we had to capture how expert risk analysts actually think when they investigate and make risk decisions for alert cases, hence the research approach and insights outlined in our recently published paper.
Question 3 – FCN: How is this possible?
CB: New technology is opening up possibilities that enable us to capture the ‘decision graph’ of experts operating within financial crime investigations. Practitioners within military, medical and legal domains are already using machine technology to capture expert cognitive processes in complex high-risk tasks, then applying the resultant signals to power machine-based decision support.
In our research, we applied such technology under controlled conditions, to capture how Subject Matter Experts (SMEs) independently investigated the same alerts generated by Anti-Money Laundering (AML) transaction monitoring systems in the retail banking space.
The set of ‘Gold Standard’ risk signals captured were used to calibrate an automated machine investigation system. The results were surprising; we found that the experts outperformed our initial expectations whilst revealing much about the thought process underlying financial crime investigations.
Question 4 – FCN: Tell us about your findings.
CB: The headline finding was that SMEs showed unanimous decision agreement on 88% of all Retail TM alerts. This was significantly higher than our expectation of around 70% based on data reported in other industries and confirmed the high benchmark of investigation quality being set by SMEs in such global banks.
Further insights can be found in the paper, but highlights included:
- 86% of alert investigations were unanimous decisions of ‘risk mitigated’.
- 2% of the alerts were unanimously determined as ‘risk not-mitigated’ and escalated for further review.
- These unanimous investigation decisions were not random, they were accompanied by very high levels of SME agreement (>90%) on key evidence judgements.
- A further 10% of alerts could be determined with a strong consensus of 3 out of 4 SMEs agreeing on decision and evidence. Analysis of these alerts revealed the impact of Decision Fatigue as the primary driver of the performance difference.
- Deadlocked SME risk decisions occurred on 2% of alerts. Some types of alerts were genuinely more uncertain, even to experts.
Applying the Gold Standard benchmark generated from 4 SMEs to all Transaction Investigations, the research also found that:
- Over 50% of the wider population of Level 1 investigators performed to the standard of the SME decision parameters.
- 30% of all investigators performed significantly outside of the Gold Standard, generating 40% more escalations than expected. In contrast to SMEs, the performance deficit for these investigators was primarily knowledge and skills.
Question 5 – FCN: What does this tell us about Financial Crime Investigations?
CB: The big surprise was the very high levels of expert agreement. Compared to benchmark inter-assessor research practice, the bar was set high with four SMEs independently investigating each alert and we measured agreement for risk decision and evidence of Affordability, Transaction Purpose, Source / Destination of Funds and Counterparty Relationship.
This reflects well on the quality of the SMEs and their extensive experience and training. The behavioural software undoubtedly helped in scaffolding SME performance, but there clearly was a very strong expert signal to build upon.
For the vast majority of alert investigations, SMEs were able to quickly and consistently identify the patterns of evidence that led to a determination of ‘risk mitigated’. There were distinctive patterns of evidence attributes for alerts in this decision category that were well known to SMEs. These evidence patterns were also present in those alerts that were mitigated with a strong consensus decision.
SMEs were also consistent in determining the ‘needles in the haystack’ that required enhanced reviews. Alerts in this decision category also exhibited distinctive evidence patterns, for example specific types of transaction purpose: charity donation, gambling, gifts, supplementary income, asset sale, inheritance.
Some types of alerts were genuinely more uncertain to SMEs. These were not the result of decision fatigue or environmental factors; these were genuinely uncertain across all experts. There were evidence states intrinsic to these alerts that increased uncertainty for SMEs.
The good news was that ~50% of all investigators performed broadly in line with SMEs. However, for approximately 30% of the wider population of Level 1 investigators, the difference between expert performance was significant. To effectively perform AML investigations an analyst must be able to competently answer around 12 key risk questions and to be able to identify around 80 different types of evidence to do so. For humans, these levels of skill and knowledge cannot be acquired overnight.
Example: Mitigated case explanation as output by Nasdaq Automated Investigator for AML
Question 6 FCN: Can these insights be translated across the industry, to other FIs?
CB: Whilst the insights from this research are directly referable to SMEs operating in retail banking, we firmly believe the findings will translate to other lines of business within financial institutions.
Leveraging the power of human expert risk thinking at machine scale will be a key driver in the next wave of FCC transformation and extending this approach to build a Gold Standard signal from SMEs across multiple financial institutions is the next stage. Indeed, we would encourage anyone interested in benchmarking and documenting their own expert performance to get in touch.
Such an extended approach will provide an industry wide view of best practice by distilling the Gold Standard signal across multiple domains including Commercial, Corporate, Private Banking, Fraud and Correspondent Banking investigations.
Question 7 FCN: Is this the future of TM Investigations?
CB: As the financial services industry is sold on a vision of hyper-automation, these findings point to an alternative approach for more complex tasks of investigation. That being a future where human expertise and machine automation operate in tandem both to complete complex tasks and to continually learn from each other. Behavioural systems that can reliably and efficiently set an expert Gold Standard is a first and necessary enabler.
This expert Gold Standard signal is sufficiently robust to configure machine solutions to force multiply human experts in investigating and explaining risk in Financial Crime alerts. For FCC operations, the ability to apply ‘expert investigators everywhere, every time’ delivers significant effectiveness AND efficiency benefits.
Question 8 FCN: Where can I learn more?
CB: For detailed insights into the research discussed, you can download a copy of the Nasdaq Research Paper here. We will be discussing this subject further with banks and industry stakeholders. For example, how should a bank use these findings to establish its own risk appetite? How do SMEs across the industry perform? How to build confidence in technology with cross functional stakeholders such as IT, Audit and Model teams, and what this means for the role of regulators. Please get in touch if you would like to be involved.
