12 Risk management and technical debt

In this chapter

Workarounds and shadow IT both create risks for the organisation and could result in an increase in the technical debt in IT development. Both risk and technical debt are very difficult to quantify but that does not mean they can be ignored. Risk management is especially important in clinical systems where patient safety and optimum patient outcomes are of the greatest importance. This chapter provides an overview of both risk management and technical debt.

—————-

Managing corporate risks

Organisations take risk management very seriously. There is usually either a formal or implied requirement on the Board from shareholders to manage the organisation in a way that minimises the risk to their investment being reduced in value. This is well demonstrated in the SEC 10K annual filing of US quoted companies where Section 1A lists out risk factors. In most organisations there will be a designated risk manager who monitors the state of operational risks from both internal and external perspectives and reports to the Board on a regular basis.

The potential impact of these risks is usually assessed through some form of scoring of the risk. At the most basic level the scoring is based on the product of the probability of the risk and the impact of the operation. The probability might be given a score of 1 – very unlikely to occur up to 5 – highly likely to occur, with a similar scoring for the impact on the organisation.

The fundamental flaw with any scoring based on probability of potential occurrence is that in reality there is little quantitative information on which to base the decision. The impact on the business is easier to judge.

There are three further elements that need to be taken into consideration in the effective management of risk.

1. At what score should the potential risks be moved up through the management levels of the organisation for discussion and appropriate action?
2. The rate of change of a score over time needs to be considered.
3. There needs to be a discussion about the risk appetite of the organisation.

Against this background workarounds could represent a significant risk to the organisation. The focus of the developer and subsequent users of the workaround will be on a short-term gain to them personally. The extent to which the workaround could put the organisation at risk is almost certainly not on their agenda, if only because the focus is on a single process and accomplishing it more effectively. The potential impacts on  down-stream stages of a process may be invisible to them because they lie in a different business unit and/or are hidden behind access security.

Assessing the risk due to workarounds is especially difficult because of their invisibility. Indeed the importance to the organisation of identifying and managing risks is arguably the most important reason for a workarounds strategy.

The two tables below set out a suggestion for a scored assessment of the attitude of IT and of individual employees to the way in which workarounds (including shadow IT) are supported.

Corporate IT assessment

Employee assessment

The appearance of 0 in the final line of each table is not a misprint! If that is the employee score and yet corporate IT has a more positive score then the product of the two is zero as an indication of a lack of communication and transparency.

Workarounds and trade-offs in information security

This is the title of a very detailed review of the ways in which workarounds can give rise to corporate risks. Woltjer (2017), based on a very thorough review of the literature, differentiates between

• Workarounds as actions that are performed when the IS policy does not specify what to do, denoted by the author as ‘workaround-as-improvisation.’
• Workarounds as actions that are done because of perceived gains in other work goals such as effectiveness, efficiency, safety, integrity or work quality, which are perceived as non-compliant to IS policy, which the author denotes as workaround-as-non-compliance.

According to Google Scholar there are only 22 citations to this paper since it was published, and a review of these shows that these citations are to papers primarily on information security policy development and compliance and not specifically to the risks associated with workarounds.

The notable exception is Slabbert (2022) who discusses the specific issues of the risk created by information security workarounds and develops a matrix of risk assessments. In principle these could be extended to applications other than information security but this is not the focus of the thesis. Essi (2023) provides a detailed review of the literature on the security issues of workarounds and also offers a categorisation of workarounds based on this review.

Internal and external compliance

An important issue with assessing the risk associated with any specific process is the extent to which the process is subject to external compliance. This is a major challenge with accounting systems where there will be an internal audit ahead of the external audit for any organisation publishing its accounts.

This issue has been considered in some detail in a series of papers by Drum (2016, 2017) in which the impact of workarounds in accounting can result in very visible risks to the organisation.

Impact on ISO 9001 certification

At the core of ISO 9001 for quality management is that consistent and predictable results are achieved more effectively and efficiently when activities are understood and managed as interrelated processes that function as a coherent system. If these processes are not managed as a coherent system because of workarounds then certification under ISO 9001 is at risk.

ISO calls out a set of actions that an organisation should be taking to achieve and maintain ISO 9001 certification, including

• Defining objectives of the system and processes necessary to achieve them.
• Establishing authority, responsibility and accountability for managing processes.
• Understanding the organisation’s capabilities and determining resource constraints prior to action.
• Determining process interdependencies and analysing the effect of modifications to individual processes on the system as a whole.
• Managing processes and their interrelations as a system to achieve the organisation’s quality objectives effectively and efficiently.
• Ensuring the necessary information is available to operate and improve the processes and to monitor, analyse and evaluate the performance of the overall system.
• Managing risks that can affect outputs of the processes and overall outcomes of the quality management system.

At one stage in my career the firm I worked for was audited for its conformance to ISO 9001 as this was critical to its professional reputation. The preparation for the audit uncovered a substantial list of workarounds where employees had not fully completed the document or had done so by cutting and pasting content from a document for Project A into the related document for Project B. The firm passed the audit though with a number of advisory notes from the audit team. The outcome of the audit process was a substantial improvement in the quality of the project documentation achieved by a considerable amount of training and more frequent internal auditing. Barata (2015) presents an approach to assessing risks in the implementation of ISO 9001.2015 which has more of a process approach than earlier versions of the standard.

Impact on ISO 27001 certification

Another business critical certification is that for ISO 27001 compliance on information security. Hybrid working inevitably introduces workarounds as employees working from home find that processes that worked well in an office environment with networked computers carefully managed by experienced IT security staff can not easily be implemented in a home or other remote environment. This is especially the case with the transfer of files using USB devices, which in a physical office setting are often locked down.

Clinical risks

Compared to the situation in an office environment the potential risk to the well-being of patients in the context of Electronic Health Record applications is an order of magnitude more important and more challenging, primarily because the risk rating could change in minutes, if not seconds, as a patient (for example) has an adverse reaction to a drug which was not correctly recorded on the EHR application.

However it has proved to be very difficult to identify research that specifically considers the risk of workarounds in a clinical setting. There are a significant number of research papers on risk assessment of clinical procedures but from the search result alone it is not possible to distinguish research specifically on the risks associated with workarounds and shadow IT in clinical settings. The exception is an extensive narrative literature review of 220 papers by Baillette (2022) on the impact of Shadow IT in healthcare,

Technical debt

Technical debt can be defined as the design or implementation components that are useful in the short term but can make future change more costly or impossible. The phrase was proposed by Walt Cunningham in 1992 but it is only over the last decade that any significant attention has been paid to the topic. Lennarduzi (2021) has published a comprehensive literature review.

Technical debt has a significant number of elements which are set out by Alves (2014) with the elements which have specific relevance to technical debt highlighted in bold

• Architectural
• Build
• Code
• Defect
• Design
• Documentation
• Infrastructure
• People
• Process
• Requirements
• Service
• Test Automation

The use of the term ‘debt’ in the description might be taken to mean that it is possible to develop a financial metric for the scale of the debt. At a top level it can be defined as a ratio of the cost to fix a software system [Remediation Cost] to the cost of developing it [Development Cost]. This ratio is called the Technical Debt Ratio [TDR].

However the debt metrics are arguably different for each element and cannot be consolidated across multiple elements.

Large corporate IT departments will have developed their own approaches to technical debt but may not have taken into account technical debt related to workarounds, shadow IT and software development.

The bottom line

The very nature of workarounds and shadow IT means that the risks they may generate could well be outside the compliance monitoring policies of the organisation. This situation inevitably increases an overall assessment of IT-related risks. Finally Chapter 13 takes a high level view of the topics discussed in this book

References

Alves, N.S.R., Rios, N., Ribeiro, L.F., Caires, V., Mendes, T.S., & Spinola, R.O. (2014). Towards an ontology of terms on technical debt. Sixth International Workshop on Managing Technical Debt

Baillette, P, Barlette, Y and Berthevasm J-F. (2022). Benefits and risks of shadow IT in health care: a narrative review. Systèmes D’Information & Management, 22(2), 59-96

Barata, J., Da Cunha, P. & Abrantes, L. (2015). Dealing with risks and workarounds: a guiding framework. 8th Practice of Enterprise Modelling (P0EM), Nov, Valencia, Spain, 141-155, ff10.1007/978-3-319-25897-3_10ff. ffhal-01442248 https://hal.inria.fr/hal-01442248v1/document

Drum, D., Pernsteiner, A., & Revak, A. (2016). Walking a mile in their shoes: user workarounds in a SAP environment. International Journal of Accounting and Information Management, 24(2), 185-204

Drum, D., Pernsteiner, A., & Revak, A. (2017). Workarounds in an SAP environment: impacts on accounting information quality. Journal of Accounting & Organizational Change 13(1) 44-64 https://doi.org/10.1108/JAOC-05-2015-0040

Essi, S. & Baker, E.W. (2023) Workarounds from an Information Security Perspective: Literature Review AMCIS 2023 Proceedings. 28.   https://aisel.aisnet.org/amcis2023/sig_sec/sig_sec/28

Kruchten, P., Nord, R. & Ozkaya, I. (2019). Managing technical debt: reducing friction in software development. Addison-Wesley Professional  ISBN: 9780135646052

Lenarduzzi, V., Besker, T., Taibi, D., Martini, A. & Fontana, F.A. (2021). A systematic literature review on technical debt prioritization: Strategies, processes, factors, and tools. Journal of Systems and Software 171, January, 110827

Reboucas de Almeida, R., Kulesza, U., Treude, C., Cavalcanti Feitosaz, D., & Higino Guedes Lima, A. (2018). Aligning technical debt prioritization with business objectives: a multiple-case study. Preprint submitted to the 34th International Conference on Software Maintenance and Evolution (ICSME 18) https://arxiv.org/pdf/1807.05582.pdf

Slabbert, E. (2022). Towards a risk assessment matrix for information security workarounds associated with acceptable use policies. Master’s thesis, Nelson Mandela University. http://vital.seals.ac.za:8080/vital/access/manager/Repository/vital:52811?site_name=GlobalView

Woltjer, R. (2017). Workarounds and trade-offs in information security – an exploratory study. Information & Computer Security, 25(4), 402-420