There are twelve posts in this series. To read Part X, please click here.
If you have been reading this blog recently, you will know we have been considering the problem of how organizations could apply Risk-based thinking (RBT) to Quality Processes. In Parts X through XV, we considered examples of the type of documents needed to support our Six Step process for applying RBT; including suggested document templates for: (1) Statement of Context (External and Internal), (2) Semi-Quantitative Risk Assessment Calculator (3) Risk & Opportunities (R&O) Register, and (4) a Risk Treatment Plan.
In this post, we will look further at a semi-quantitative method of risk assessment that forms one step in our Six Step model, and in particular, at the Semi-Quantitative Risk Assessment Calculator that could be used to help you determine the risk factors in a quality system.
Formal risk assessment can range from simple classification into categories such as High, Medium and Low, through to the use of sophisticated mathematical models referenced in ISO 31010:2009; including for example the Statistical Methods of Markov, Monte Carlo and Bayesian analysis that we described in Part VI of this blog post series. The method described here is a Semi-Quantitative approach to risk analysis. Its purpose is to provide measures for consequences, likelihoods and overall levels of risk, which can be used to generate a picture of 'risky' processes that will facilitate comparison of different risk factors. From this information, managers can more easily determine priority risks.
Let us start though by saying that the risk to an organization's quality is just one component of its business risk. We are not proposing to extend quality risk assessment practice using this technique beyond the quality management system and its component processes. However, we do consider that it is important to understand that product quality should be maintained throughout the product lifecycle1, in order to provide a proactive means to identify and control potential quality issues during development and manufacturing and/or service delivery. This is because quality risk management can improve decision-making and knowing this, customers and third parties will be better assured of an organization's ability to deal with the potential risks. Therefore, we will use this semi-quantitative approach to review risks to quality across the product or service lifecycle, including development, manufacturing, distribution, inspection, and relevant submission/review processes for your products, and for the equivalent steps used in designing and delivery of services.
In this way, a quality risk assessment (QRA) will help us to achieve more effective and consistent risk-based decision-making when planning and considering quality processes. We would also suggest that QRA is aided by the adoption of the formal risk management processes referenced in ISO 31000:2009 that make use of empirical tools and/or documented internal processes/procedures, even though this degree of rigour is not required to achieve compliance with the draft wording of ISO 9001:2015.
The Semi-Quantitative Risk Assessment Calculator provides a useful tool in the systematic process for the assessment, control, communication and review of risks to quality. However, the document format itself has been designed to be a simple to use as possible, reflecting the fact that the level of effort, formality and complexity of documentation should be commensurate with the level of risk. This step in our proposed Six Step QRA method is intended for use by smaller enterprises with relatively simple and often low risk processes to consider, and is not intended to substitute for formal risk management processes adopted by much larger organizations that have to assess complex risks in high risk environments. There is a world of difference in how a global corporation in the oil and gas industry is required to manage risk when designing and operating a petrochemical plant compared to the risks experienced in the context of a small retail operation. The Semi-Quantitative Risk Assessment Calculator is more suited to the needs of SMEs employing fewer than 250 people and not to large-scale, high-risk enterprises in heavily-regulated industry sectors; although even in these environments, a relatively simple, documented risk management process of the type that we are suggesting is, arguably, better than nothing at all. That is to say, in instances where Quality Managers are not already taking a risk-based approach (defined in ISO/DIS 9001:2014) to determining both the type and extent of controls appropriate to an organization's quality processes.
When completing the QRA Calculator form below, the activity should be undertaken by interdisciplinary teams. When the teams are formed, they should include experts from appropriate areas of the organization; e.g. quality unit, business development, engineering, legal and regulatory, production operations, service delivery, marketing, and business analysis.
The decision-makers, under the guidance of the Quality lead, should take responsibility for co-ordinating quality risk management (QRM) across relevant functions and departments, ensuring that the process is defined, deployed, and reviewed, and that adequate resources are made available.
The Step 4 process objective is to calculate the risk factors (RF) for the different quality processes involved in producing a contract deliverable (or market opportunity) on time and in line with the quality policy objectives. Each process should have a defined process owner who is ultimately responsible for implementing policy; that is, what the organization needs to do to ensure that quality is controlled through a clear understanding of risks and opportunities and by monitoring the performance of the process.
In addition to estimating and comparing the different risk factors (RF), the template leaves room for the results of brainstorming sessions that are used to determine the root causes of potential non-conformities so that effective preventive actions in the form of controls can be planned.
One of the appealing aspects of the Step 4 Semi-Quantitative Risk Assessment Calculator is the data that it provides for a graphical output of risks plotted against the background iso-contours representing levels of risk, which highlights which individual risks should take priority when planning and considering your quality processes and allocating resources. All employers, however, should be encouraged to relate their activities to the various risk factors assessed and suggest improvements in working methods, materials, suppliers, and sub-contractors etc to address risk. Likewise, the quality management system should be designed so as to have an established procedure for review of all the activities, including quality risk assessment processes (see Step 6 - Monitoring and Review), in order to identify all possible improvements in methods/materials and standard operating procedures so as to prevent any undesired outcomes.
Simply identifying possible risks and opportunities, and then performing both qualitative and semi-quantitative risk assessments to analyse and evaluate risks, may not lead to any improvement. Risk analysis should be used to anticipate potential problems that could adversely impact quality outcomes to highlight the need for actions to prevent, mitigate or transfer risks that are associated with the organization's context and objectives. You should regularly consider both the outputs of analysis and evaluation (Steps 2, 3 and 4), and the outputs from management review (Step 6), to confirm if there are any areas of underperformance or opportunities that need to be addressed as part of the continual improvement process cycle.
The output is a list of risks to the successful outcome of the quality process with risk factors (RF) determined for each individual risk and for the process as a whole. These values can be plotted to show the relative level of risk (the 'riskiness' being a measure of consequence (C) and likelihood (L) ratings) to facilitate easy comparison and prioritization.
The recommended method for calculating risk factors is expressed as:
Ratings are determined from a pre-defined scale for consequence and likelihood; so, for example, "A" on the sheet represents "almost certain" with a probability of 0.9 on a scale of 0 to 1; whereas "E" is for "Rare" types of risk with the probability of 0.3. "E" could be defined as meaning:
"This risk is judged to be very unlikely to impact in the next 25 years."
Obviously, the descriptions for the various consequence and likelihood indicators will depend largely on the context and the processes involved.
Before we judge whether the Likelihood rating is A, B, C, D E or F, as suggested on the Table below, we should also take into consideration Control Effectiveness in the quality processes in order to assess whether existing Controls would act to reduce the likelihood of a risk occurring.
We would suggest using the following Likelihood ratings, but these could vary depending on your organization's context and the risks in your internal and external environment - see Table 1.0:
TABLE 1.0: Semi-Quantitative Risk Calculator: Likelihood indicators + scoring
| Descriptor | Description | Frequency | Probability | |
| A | Almost certain | Very high risk - controls ineffective / non-existent | One or more times per year / frequent | 0.9 |
| B | Likely | Likely at least once a year | 1 per 1 year | 0.8 |
| C | Possible | Once in a 5 year period | 1 per 5 years | 0.7 |
| D | Unlikely | Once in a 10 year period | 1 per 10 years | 0.5 |
| E | Rare | Risk occurs only in very exceptional circumstances | 1 per 25 years | 0.3 |
| F | Very rare | Extreme outside chance | 1 per 50+ years | 0.1 |
Consequence indicators will of course depend on the context of the organization and the nature of the risks and opportunities that are being assessed; however, indicators such as cost, delivery time, performance values for the manufactured product or service delivery, user satisfaction (as measured, for example, by surveys and feedback from the Sales team), and (5) reputation (of the product and / or the organization's quality), provide us with a starting point when considering the effects of potential non-conformities on the desired outputs of quality processes.
Before we judge whether the Consequence rating is A, B, C, D E or F, as suggested on the Table below, we should also take into consideration Control Effectiveness in the quality processes in order to assess whether existing Controls would act to reduce the consequences (impact) of a risk.
For example, if fire damage is a potential risk, will the existing Fire Safety Controls limit the damage that could be caused to the process outputs? And if the outputs in your situation consist of software that is mirrored on several servers in different physical locations, which are protected by the latest fire detection and response systems controlled by your facilities management team, do you need to assess the likelihood and impact of a 'worst case scenario' as a potential risk to quality? Answer: Probably not! The building fire scenario is surely the concern of your IT and Business Continuity Manager colleagues? However, for Quality Managers, there are much more relevant risk and opportunity scenarios and situation specific indicators of the risks to quality that should be afforded a higher priority.
For example, question marks raised about the quality of outsourced software development work that could adversely impact on the conformity of the organization's products and services2 indicate a risk issue that it is appropriate for the Quality Management team to assess, and if found to have a high risk factor (RF), prioritise and address to ensure that, in the wording of ISO 9001:2015: "externally provided processes, products, and services conform to specified requirements"3. Risk-based thinking should:
a) give assurance that the quality management system can achieve its intended result(s);
b) prevent, or reduce, undesired effects;
c) achieve continual improvement.4
Table 2.0: Semi-Quantitative Risk Calculator: Consequence indicators + scoring
| Descriptor | Description | Level | Score | |
| A | Catastrophe | Irreparable damage to the organization's reputation | Highest | 0.9 |
| B | Extreme risk | Critical non-conformity / loss of human life or injury / major reputation damage | High | 0.8 |
| C | Severe problem | Major non-conformity with lasting reputation damage | Medium-High | 0.7 |
| D | Problem | Minor non-conformity / issue | Medium | 0.5 |
| E | Routine | Low impact - can be managed by standard quality processes | Low | 0.3 |
| F | Minimal | Few/no observable impacts | Very Low | 0.1 |
The template below is an example Semi-Quantitative Risk Assessment Calculator (SQ-RAC) worksheet, adapted from Dale F Cooper.5
Click on the document icon to download the PDF, or visit the Free Templates page in our Documents Library.
Notes:
1 Guidance for Industry, Q9 Quality Risk Management, US Department for Health and Human Services, et al, June 2006, p.2.
2 ISO/DIS 9001:2014, 5.1.2 Customer focus, b), p.27.
3 Ibid., 8.4 Control of externally provided products and services, p.36.
4 Ibid., 6.1 Actions to address risks and opportunities, p.28.
5 Proposed Semi-Quantitative Risk Assessment Calculator (SQ-RAC) worksheet adapted to assess risks in relation to quality system processes from 'An alternative assessment sheet (Table 9.4)' on p.376 of 'Project management guidelines: managing risk with ISO 31000 and IEC 62198'; Dale F Cooper, et al; John Wiley & Sons Inc., March 2014
There are twelve posts in this series. To read Part XII, please click here.