Comprehensive Assessment of Fishery Compliance Risks (RBC part III)
Following on my last post in this series on the importance of prioritizing “compliance” over “enforcement”, I’ll continue in this post by covering the next “big idea” I had stumbled upon while researching risk-base compliance (RBC) programs for fisheries: comprehensive assessments of fishery compliance risks.
In this respect, I think Australia, and reportedly New Zealand, are quite unique when it comes to how they manage compliance issues in their fisheries. In discussion with colleagues, however, some have said to me that they think RBC programs for fisheries are relatively common. But when I hear their examples, I see that there’s some confusion about what it means to manage a fishery according to its compliance risks.
My colleagues point out, for example, that many fisheries enforcement agencies employ risk-profiling tactics when they decide which vessels to inspect, and companies to audit, and thus, there is a RBC program. This, for example, is considered the norm in EU fisheries and is pretty regularly done in the U.S.
But risk profiling, in my view, is not enough to make a comprehensive risk-based compliance program. Rather than call profiling a risk-based approach, I’d call it “intelligence-based policing” and categorize this as one possible treatment to certain compliance risks, such as operating without a valid permit, fishing over quota, and using prohibited gears. What’s missing here, then, is a clear indication that these risks should be prioritized over other fishery risks, such as illegal interactions with endangered species, fishing in closed areas, or disabling VMS, among others. This is what RBC programs are really about.
To put things in perspective, let’s look at how Australia develops its multi-year compliance program for Commonwealth fisheries, like the current one for 2014-2015.
Before I dive into the details, I’ll just call attention to a clear challenge to compliance risk assessments: the approach is resource and time intensive. On this, the Australian Fisheries Management Authority (AFMA) would agree, and for these reasons has shifted from conducting its fishery compliance risk assessment on an annual basis to a biannual basis. As for just what might be a suitable period of time for conducting these assessments, I really don’t know. Every two years? Every five years? I think it will depend upon the fisheries and the fishers.
What does matter, and what stands out to me as the most important thing here, is that AFMA does conduct structured, periodic risk assessments using a transparent methodology that categorizes and prioritizes risks for treatment. And I think the result is that you obtain a much clearer picture of the compliance problems at the individual fishery level and at the national level. Since risk assessments vary greatly within and across stakeholders, the structured approach helps reduce the “noise” and obtain consensus, while the periodicity of the process allows stakeholders to assess how risks change over time.
AFMA’s Risk Management Process
So let’s look at the general risk management process for Australian Commonwealth fisheries. As diagramed in the image at the top of this post, risk assessment begins with the defining of the “risk context”, and then it is followed by the “risk assessment” activities, which constitute the bulk of the process.
Let’s focus on these two aspects today. (Treatment and Monitoring will be covered, respectively, in the next two posts in this series.)
Establishing the risk context is really about setting the framework and parameters for the risk assessment. It establishes the structure of the analysis, identifies actors and risks, determines the criteria against which risks are assessed, and defines the spatial and temporal parameters of the analysis.
For AFMA, the framework is based on the International Standard 2009 ISO 31000 Risk Management – principles and guidelines and is fully outlined in the National Compliance 2013-15 Risk Assessment Methodology. The parameters are that it looks only at Commonwealth fisheries and the risks that operators do not comply with fishery management legislation. Also, implicit to the periodic review is a two-year assessment period (since this is, as of 2009, a biennial assessment). Finally, as noted by the methodology, AFMA seeks to assess each fishery individually to identify specific risks, and then conduct a combined risk assessment to identify significant risks common to more than one fishery.
I suspect you’ll agree that this is pretty straightforward.
The risk assessment process identifies where an undesirable or unexpected outcome could be significant or where opportunities for non-compliance are possible. Importantly, AFMA emphasizes that the assessment should be conducted on the basis of consultation rather than a one-way flow of information from the decision-makers to the stakeholders.
The risk assessment is broken down into three components: Risk Identification, Risk Analysis, and Risk Evaluation. Here’s is where things get rather interesting, so we’ll tackle each one separately in the following sub-sections.
The goal with risk identification is to generate a comprehensive list of fishery compliance risks. This can be completed in a number of ways, including examination of historical risk assessments, discussions with expert panels, brainstorming, and stakeholder meetings to establish or determine opinions, perceptions and experiences.
I particularly appreciate the emphasis here on stakeholder participation in risk identification. AFMA explains that the primary stakeholders that are included are industry management advisory committees (MACs), industry associations, data processing contractors, AFMA observer section staff, fisheries managers, domestic compliance staff, and intelligence officers. I do wonder, of course, how representative these MACs are of the broader industry, but the highly industrial nature of Commonwealth fisheries leads me to guess that they are probably pretty good.
Also quite nice is that stakeholders’ input is formally structured with “risk identification” templates, which list a set of pre-identified risks to be rated, and which permits stakeholders to identify new or unlisted risks.
So what compliance risks have been identified for Commonwealth fisheries? Since the formal risk assessments are not publicly released, this can’t be answered. But the methodology document does offer a set of common compliance risks that are likely quite representative of risks that are regularly identified in the assessments.
Here they are, broken down by category:
The next step in the risk assessment phase is risk analysis. This is where some understanding is developed with regards to those risks identified in the preceding step. For AFMA, this involves rating the consequences associated with each risk, the likelihood of the risk being realized, and the extent to which current enforcement activities reduce the consequence/likelihood of each risk.
This entire process is depicted in the following graphic.
To understand the nitty gritty of this image, you’ll need to understand four key aspects.
First, the risk analysis is also divided into two parts. The first part establishes the “inherent risk”, or the consequence and likelihood of a risk in the absence of current enforcement activities. This part is conducted solely through internal consultation and analysis by AFMA observer section staff, fisheries managers, domestic compliance staff, and intelligence officers. The second part establishes the “residual risk”, or the consequence and likelihood of a risk given current activities. In addition the AFMA staff that took part in the analysis of inherent risks, MACs, industry associations, and data processing contractors also participate.
Second, the risk analysis is conducted using standardized functions and scales. Consequence consists of material impacts, uncertainty, and impacts to the reputation and credibility of AFMA, and can be rated on a five-point qualitative scale: insignificant, minor, moderate, major, and severe. Likelihood, meanwhile, consists of perceived incentives to noncompliance and the morale, or the normative environment that might support/prohibit noncompliance. Similarly, likelihood is assessed on a five-point qualitative scale based on perceived probabilities: rare (≤5%), unlikely (≤30%), moderate (≤50%), likely (≤70%), and almost certain (≤95%).
Third, quantitative and qualitative risk ratings are assigned to each inherent and residual risk. This begins with a risk matrix (below), to which numbers are assigned 1 (low), 2 (moderate), 3 (high), and 4 (severe).
Fourth, and finally, the final scores for residual risks are weighted by the compliance risk history. This means that risks ranked as low by stakeholder input may become a higher priority if information indicates that a high proportion of detected offences occur. Weighting by compliance risk history involves the review of “incidents”, which are defined as single breaches of management rules where enforcement action or investigation is undertaken during the preceding two years. Relevant residual risks will be increased by 0.5 for risks with 5-10 compliance incidents and by 1.0 for those with compliance incidents greater than 10. For risks with 0-5 incidents the level of risk will not be increased.
The final component to AFMA’s compliance risk assessment is risk evaluation, or the making of decisions based on the risk analysis. To do this, a risk level is assigned with the following table:
From here, a full and final fishery compliance assessment can be produced. This is the image below, and for your quantitative overload, I recommend you click on the image for expanded viewing.
And once compliance assessments are produced for each fishery, AFMA determines the overall rating of each risk across all Commonwealth fisheries by averaging the residual risk scores across fisheries.
From here, I’d say that Oz has a really terrific risk assessment methodology, with the exception of some non-transparency that enters into the very final step in prioritizing compliance risks across Commonwealth fisheries. That step is detailed as followed:
The residual risk will then be further amended (where relevant) by the OMC [AFMA’s Operational Management Committee], who will apply their own weightings to determine the final residual risk level:
Average Residual Risk Rating +/- OMC Weighting = Overall Residual Risk Rating
The OMC will consider resource implications and public perception when amending particular risks.
Ultimately, it seems that this final step gives AFMA a lot of wiggle room to put politics before sound management. I am not at a point where I can say this is really an issue, but it does raise a caution flag. And just as importantly, maybe a little politics wouldn’t be so bad? After all, the “big fish” illegal fishers will be paying attention to anything that is published too.
Results for 2013
Though the full results of the compliance risk assessments are kept private, AFMA’s main findings are published in the biannual National Compliance and Enforcement Program that is designed every two years to address the risks prioritized by AFMA.
According to the National Compliance and Enforcement Program for 2014-2015, the domestic compliance risk assessment identified 15 risks across Commonwealth fisheries that were assessed as moderate/high and high, and identified two risks that were also chosen for treatment. These risks are: failure to have a Vessel Monitoring System (VMS) operating at all times, and quota evasion (risk rating: moderate/high).
A Role Model Approach
Ultimately, I’d suggest that compliance risk assessment for Commonwealth fisheries provides AFMA with a sound footing to target key fishery risks. The assessment incorporates the views of multiple stakeholders, adopts a comprehensive approach to risk assessment, and is highly structured.
Certainly, however, you could ask for some improvements, most of all, I’d want to see greater transparency in how the OMC provides a final weighting to residual risks based on “resource implications and public perception”. But I won’t be hard on AFMA here. Though the process is not perfect, it is certainly a shining example of best practice in modern fisheries management. The risks that AFMA has cumulatively disclosed over the years is pretty impressive. And, as you’ll see in the next two posts, AFMA continues to provide examples of best practice management, both in its treatment of compliance risks and management of enforcement data.
I recently was able to talk further with AFMA about their risk assessment methodology and they were able to add further detail on the OMC Weighting Factor, which I identified as a possible point of low transparency. Tod Spencer, the Senior Manager for the National Compliance Strategy, was able to clarify that this final step in the assessment is principally used when:
- There is an obvious misunderstanding of the risk leading to over or under rating of the risk;
- An “approaching risk/factor,” which may not have been evident to stakeholders, should be considered, meaning that the risk has been under-rated (e.g. Newly listed species, Quota changes etc.); and/or
- It is recognized that AFMA can’t address every risk and so may make a decision as to which (of two or more equally rated) risks they are going to target – based on their “perception” of which is more important.
Many thanks to Tod and his team for the additional information!