JFSC approach to risk [revisit a speech in 2021]

On 18 February 2021, the JFSC Director General [Martin Moloney, Director General] at the JFSC spoke to Financial Transparency Advisors about building risk assessment capacity.[NOTE:- Maloney has now left the JFSC]

Martin Moloney's speech wanted to articulate some of the practical issues, from a managerial perspective rather than a technical perspective, that arise with moving towards an increasingly risk-focused approach to the risk assessment process.

• The whole speech is here https://www.jerseyfsc.org/news-and-events/building-risk-assessment-capacity/

I have taken the following extract from the speech because it provides an outline of the JFSC approach to RISK MANAGEMENT [or at least this is what it was in 2021] along with the challenges of FATF/MOEVAL

I am also particularly interested in his conclusion

• Once you build a risk system and come to rely on it, you can miss an obvious problem even if it's right in front of your eyes, because the risk system is driving your choices.
• Paradoxically, the more credible the risk system, the higher the chances of you becoming entranced by it.

I PICK UP HIS SPEECH HERE………………………….

……One pattern has been that before building risk systems, regulators allocated resources in proportion to the size of the entities in their supervisory portfolio. They then built a risk system.

As a first iteration, they decided to allocate supervisory resources in accordance with impact and not to try to calculate risk event probability, because it's too hard. (The language of AML/CFT risk guidance is a little different, but let me use the more standard language for a moment as it suits the point being made.)

The result is they allocate resources in accordance with impact and, since size is the main proxy for impact, they end up concluding that the allocation of resources they already had is the best one and was, after all, risk-focused!

I do not want to be too hard on financial regulators, but this does suggest some difficulty in defining clear success targets. Without a doubt, the best regulators have continued to work on their risk systems and the best are now experimenting with big data and data analytics to crack the challenge of estimating probability.

However, the challenge is there. The practices are widely varied.

What this does indicate is that the crucial question at any point in time is to define the ambition for the next step along a spectrum from impact assessment using size as a proxy, which is minimally disruptive to existing resource allocation, on to a more nuanced measurement of impact and on further towards the estimation of probability.

I have tried to sketch out a spectrum, which you might use to position yourself and challenge yourself on what the next step should be:

• Scale  of risk event with size of the entity as a proxy for scale = inherent risk (i)
• Scale of risk event with a range of metrics = inherent risk (ii)
• Scale metrics + simple proxy for probability = inherent risk (iii)
• Scale metrics + proxy for probability + proxy for effect of controls = residual risk (i)
• Scale of risk event metrics + proxy for probability + range of metrics for effect of controls = residual risk (ii)
• Scale metrics + probability metrics + control metrics = residual risk (iii)

This is a lot and it's quite theoretical.

But it seeks to chart a probable path of progress, defined in terms of data complexity, in the potential development of risk systems for regulators.

As a manager, you need to be able to justify each step you take along this path. You should not assume that you need to keep moving up through these levels unless the increasing ambition is working for supervisors. Simple can be powerful. Complex can be expensive and disorientating, although it may also be powerful and enabling.

We in Jersey are somewhere around level three and in the process of defining our next step.

In our model, the impact component has two elements.

• The first is the simple scale element, where we use size metrics like turnover and number of customers to provide a relative view between firms. Some of those scale measures go beyond impact in the AML context and also include jurisdiction-wide impacts, particularly those linked to safeguarding the economy. For example, we also use a size metric for employees, indicating how many people might lose their jobs if the firm was to fail. I will come back to this issue of combining AML and non-AML considerations in a moment.
• The second aspect is to form and apply a view on the relative weighting of AML impact as a risk compared to the other risks we regulate. This effectively identifies a proportion of the maximum impact a firm presents, depending on how severe the risk, AML perhaps involves a higher level of total impact compared to loss of confidential data, but a lower level than terrorist financing.

We weight these different considerations based on judgement, but it is a strictly controlled judgement process, based on surveys and internal challenge.

The other issue highlighted by thinking about how you progress through these levels is the need to turn a single metric into either a probability or impact metric.

• Is having a relatively high number of Politically Exposed Persons (PEPs) an impact metric for AML because the scale of business involving PEPs is higher, or a probability metric as the more PEPs you have the higher the probability of facilitating money laundering?
• We have applied these metrics as a probability factor (after much circular discussion).

You will need to form your own view, just making sure to minimise double counting.

Let me leave aside the potentially exciting world of data analytics where a regulator has not only a wide range of data, but also the capacity to establish key non-linear data relationships in order to do credible estimates of probability.

This would be what would take us to Stage 6 of this schema. This is on the horizon.

Should it become a reality, I think regulators will start to have serious debates about whether we should continue with hard-wired cycles of inspections covering all firms.

What is really interesting us in Jersey is the move from Step 3 to Step 4. This is the move from measuring inherent risk to also measuring residual risk. Let me talk first about data and then turn to that issue.

Risk system design issues: Four issues

1. Data

• Once you have defined your next-step ambition, the immediate question then is data. The development of good risk assessment tools is hard. Most risks have to be assessed using data about proxies for that risk, rather than data about the kind of risk events which the supervisor is seeking to reduce. Much of the relevant information is hard to turn into data. Much of the data, if it exists, is in the private sector and must be collected. Many of the proxies are inadequate proxies.
• So let me be very practical in telling you what we did.
• The data that we had, or that we routinely collected (the information about what our industry does, how many customers they have, what countries they deal with etc.) was pretty variable across our sectors. In some sectors (for instance banking) we had great data. We had a history of active interaction and engagement and could have a pretty good go at assessing AML risk in these institutions and sectors. In other sectors – less so. 
• So we quickly steered our thinking towards some sort of standard improved, industry wide data collection exercise tailored to assess AML risk. For us, this was the first time that Industry was asked to do this kind of tailored, risk-focused reporting. A lot of our industry had not really thought about their client and transactional data in this way before; they didn't necessarily collect or record or store their own data in a consistent way, when compared to each other. There were differences in definitions, in classifications – and we needed to get them all reporting in the same way so we could essentially compare apples to apples and oranges to oranges.
• Our challenges in Jersey in this regard were not unusual.
• As regulators we know in practice that data management is proving hugely challenging for the whole financial industry. We should always recognise that in setting our ambition.
• This is why we felt it was important to give Industry as much notice as possible about the data set we would be collecting. We needed to give Industry plenty of time to collect the data, to build the systems for storing the data and to build the processes for reporting to us.
• It proved a good idea to launch this process at the same time as Jersey launched its National Risk Assessment process, in which the industry was also fully involved.
• Soon thereafter, we also launched a new specialist Financial Crime Inspection team.
• All this made it clear to Industry that the request for data was part of a step change in Jersey's approach to fighting financial crime.
• Leveraging this understanding and awareness, Industry "getting" the idea that we needed to assess AML risk across our sectors, really helped the success of the data collection exercise.
• We have achieved very high levels of satisfactory returns with only a few non-financials (around 5% of the total population) struggling.
• As to the content of our expanded data requirements, we started with the data set that the World Bank methodology would require and added to that. As we have now built out our risk model, our requirements are likely to change a good deal.
• We are now about to begin a new review of the data set to exclude data points we have not found useful and to seek new data that is suitable for the next stage in our development of the model, taking into account the options which emerged from Jersey's National Risk Assessment.

2. One regulator/One risk system

• A second issue is how integrated the risk assessment process for AML/CFT supervision should be with the regulators' risk assessment process for other supervisory activities.
• Jersey like many other jurisdictions has other risks to supervise, we are also a prudential and conduct supervisor.
• Also like many other regulators, resources are always a challenge and we equally have to apply a risk-based approach to our other supervisory responsibilities.
• Firstly let me note the huge advantage of placing the regulation of AML within the financial regulator. Supervisors of financial institutions will almost invariably have much better data, more historic engagement and a better understanding of compliance and culture than just about any other institution within a jurisdiction that might be a candidate for running any aspect of AML regulation.
• In Jersey we not only have all aspects of AML regulation in the financial regulator, but, quite unusually, we also have the jurisdictional company registration function within the financial regulator. From an anti-money laundering perspective, I would suggest this is an incredibly powerful position to be in.
• This does present a challenge for MONEYVAL and FATF assessors, who need to see that the supervisory allocation of resources is appropriately driven by ML/TF risks.
• On the other hand, FATF recognises there are significant organisational synergies to be achieved by conducting financial crime risk assessments as part of wider risk assessments.
• I don't seek to resolve this issue here, just to note that the Jersey Financial Services Commission currently, as an interim fact of life, is partially adopting both approaches, having a separate AML/CFT system and a system which includes AML/CFT in its broader risk assessments.
• I will explain a little more about this, by turning to the question of residual risk.

3. Assessing residual risk

• I said earlier that Jersey is somewhere between level 3 and level 4 in terms of its progress in developing its risk model. At some point, and this is the part of the development cycle we are working through, it becomes important to measure residual risk well. I define residual risk as the risk that remains after inspection and remediation.
• Inherent risk models face what feels like a fundamental conflict at their core. They don't account for the control environments that regulatory frameworks mandate, and in which we know many inherently high risk firms operate successfully. Any model that provides for ongoing supervision of firms who manage their risks perfectly well brings with it a call on resources that will always be tough to manage.
• Assessing residual risk has two elements:
  • Firstly taking into account what you see in terms of controls when you go out to look and
  • Secondly how the situation has improved after your inspection report and any consequential improvements in controls.
• In process terms, factoring either of these in requires the option of 'reassessment' to be integrated into the risk model. There is also the option that you may be able to use proxies for the effectiveness of the control environment and therefore factor in an estimation of the effectiveness of the predicted control environment. This can be done even before inspection and as part of the initial assessment.
• We know there was some discussion of this point in various FATF supervision fora over the last few years, with very vocal discomfort from some attendees about planning supervisory activity on assessed residual risk at all. That discomfort relates to the point I discussed earlier concerning the purpose of inspections and the fixed-cycle.
• In Jersey, we have tried to manage these challenges, conflicts and needs by effectively building the two risk models I mentioned, at least as an interim approach.
• We have an inherent financial crime model built on annually collected data, with its primary role being an entity selection tool that ranks, sorts and groups different sectors on the basis of inherent financial crime risk. It is this output that forms the basis of the cycles of examinations carried out by our specialist financial crime examiners.
• The second model we have built shares the same starting point as our financial crime model, but takes it and expands it to cover prudential and conduct risks. It also uses an assessment of the controls in place in a firm to assess residual risk. In construction, this integrated model is much more sophisticated, being integrated into a technology platform that our supervisors use in their core supervisory processes. There is also automation built into the model, processing data to identify outliers, and spotting and flagging risk concentrations. It is this model that ultimately we hope will be the dynamic model that will drive much of our risk-based approach to supervision.
• Even as we are setting ourselves the ambition of developing our capacity to assess residual risk, we have an open question as to what reliance we will place on that assessment. One option is that inherent risk is used to drive the frequency on an onsite inspection, and residual risk to drive the intensity and/or scope of the inspection.
• The inherently risky nature of an entity would lead to very frequent inspections, but the nature of the inspection (the breadth and depth of the control testing, etc.) could be less stringent where that entity has a demonstrated good track record of compliance (indicated by the lower residual risk). This approach would perhaps be close to that approach that FATF is moving closer to advocating.
• Let me leave that difficult 'philosophical' question aside and, secondly, go back to the data issue. Data from firms is good enough to assess inherent risk. But for an assessment of residual risk, we need good compliance data (for instance the actual results of on-site and off-site testing). This is a challenge of a different order: collecting data from ourselves!
• There are two potential reasons why you would not have good supervisory data.
  • Firstly, because in relation to low inherent-risk sectors, you adopt a reactive approach to supervision and this means you don't go on-site to many firms and don't have the opportunity to collect data or rather to collect sufficient data to apply a robust, data-led "control" or "compliance" score to every firm. The obvious option in these cases is to look for proxies.  If we have tested a small, random sample of, let's say, accountants, perhaps we can assume that this is indicative of most accountants and simply apply those "control scores" to all accountants? In Jersey we have not done this, although it has some merit. So we have an outstanding issue of controls assessment in low inherent risk sectors.
  • Secondly, when we do go out on inspections, we have the issue of whether our inspection methodology is designed to facilitate the collection of comprehensive, comparative data on controls. Inspectors are used to the exercise of judgement rather than data collection; they do not naturally design their inspection to feed a risk system.
• Going into this question might lead me into the topic of the next session.
• So let me leave that one there:
  • Just noting that when it comes to measuring residual risk, it is important to have designed a data set and recorded reasoned judgements that assess, for the record, the quality of controls in a regulated firm and then to have an inspection methodology which facilitates the efficient collection of that data and the articulation and challenge of those judgements. The more of these that are collected, the closer you get to having a control data set in relation to which you can benchmark subsequent on-site judgements.
• We have also factored in some other data, for example
  • Ombudsman complaints which are helpful in this regard and we recognise that there may be many additional data sets that can be added. Some care should be taken with doing that, as weightings can be misjudged easily and throw off the system for its balanced judgement.

4. Risk assessment of non-financial entities

• Let me come then to the final area I will cover as part of my reflections on the design phase.
• I spoke earlier about the challenges involved in combining the AML risk assessment with the assessment of conduct and prudential risks. But even if you do not go down that route, you may still face the challenges of combining the assessment of financial institutions and non-financial businesses and professionals (DNFBPs).
• This is less relevant to some of you because the practice in many countries is that the regulation of the non-financial business is done separately. From my perspective this is one of the great strengths of the Jersey approach, because it acts as an opportunity to prevent an internally inconsistent approach emerging to risk appetites with regard to the financial and non-financial sectors. It is my impression that this is a very real problem in many jurisdictions.
• But the fact that we have the opportunity to develop a consistent approach to risk across financial and non-financial sectors does not guarantee that we will do so. There are challenges. The core of the challenge is the addition of a lot of non-financial sector entities into a regulatory framework and practice which is designed for the financial sector and which, as we have already discussed, tends to focus on the larger institutions.
• Non-financial entities will not have the accumulated investment in compliance and risk management functionality and they will tend to be small. This can mean that large populations of DNFBPs can be "lost" in the risk model and supervisory approach, particularly as they are added into a model which is in the process of an iterative development.
• You may recall MERs over the last few years containing criticism of supervisors' risk models being too obsessed or weighted towards size, when size is not necessarily the best indicator of risk. There are also substantially more money laundering and terrorist financing typologies; better and more widely accepted risk indicators in relation to financial institutions than for DNFBPs.
• Most of you could explain fairly easily how a bank might be used for terrorist financing; but there is not quite the same level of understanding in relation to accountants, or lawyers, or other DNFBPs.
• I said I wouldn't talk about terrorist financing risk, but I can't resist the temptation to observe that these problems arise particularly intensively in relation to terrorist financing.
• There are simply not good terrorist financing risk indicators and there are not sufficient terrorist financing investigations and prosecutions to underpin good risk assessment system development.
• Without good case examples or typologies, it's often very difficult to develop meaningful risk indicators or red flags for some of these sectors.
• In Jersey, we have work to do in this area.
• This is one area where my earlier discussion of the purpose of inspections comes into play.
• If non-financial entities are overwhelmingly small and with weak control frameworks and weak data, is the most reasonable thing to do to decide that they are all lower risk? In resource terms, that might seem like the only way to generate an achievable inspection ambition.
• To compound the issue even further, even if we could develop good indicators for the non-financial sector, we would then face the difficulty of comparing the results to the results from the financial sector.
• How do we ensure our risk model sufficiently considers and distinguishes between product risks across very different sectors (for instance a "high risk product" in a law firm vs a "high risk product" in a trust and company services provider vs a "high risk product" in a bank?
• Where there are separate regulators for accountants, trust companies, lawyers and financial institutions, this important issue is hidden.
• But it is still there. Even if it is done inadvertently, each jurisdiction is, by default, making relative judgements.
• One way many risk systems seem to 'fix' this problem is by adding in some additional data fields which push up some of the smaller entities into the high risk category.
• This can appear to have corrected the problem but it is not clear that it is actually a risk-focused solution or whether designers are just playing with the weightings.
• Is it better than just randomly adding some small entities into the high-risk category?
• I confess I have some attraction to this idea of adding an element of non-risk assessed random inspections from the pool of entities, financial and non-financial, that are at risk of being lost in a highly structured, rational approach to risk assessment.
• I won't try to bottom out this additional issue.

For those who are strongly focused on the MER, it is interesting that this point rarely, if ever, comes up in the evaluation reports, perhaps because it is so difficult.

The issue here is part of a much bigger issue of what is sometimes called 'model risk'.

Once you build a risk system and come to rely on it, you can miss an obvious problem even if it's right in front of your eyes, because the risk system is driving your choices.

Paradoxically, the more credible the risk system, the higher the chances of you becoming entranced by it.

Read the whole speech = https://www.jerseyfsc.org/news-and-events/building-risk-assessment-capacity/