Risk Management and Contingency Planning
All digitisation projects involve an element of risk as their success is based on the reliability of digital technology, the employment and retention of skilled staff and possibly third party contractors. Even the most rigorously planned projects contain uncertainties, or have elements which have the potential not to go as planned. In many respects the project plan provides an 'ideal world' outline of the technical developments and processes of a project working to time, along a clear workflow carried out by the project team.
Unfortunately in the 'real world', outside of the carefully crafted plan, unexpected events can and are likely to occur. Planning for and managing these unexpected events is thus a fundamental element of a successful digitisation project. This paper serves as an introduction to risk management and contingency planning and outlines some easy-to-apply techniques that can be used to analyse risks, identify ways of dealing with unexpected events most effectively, and minimising their potential effect on the successful production of the digital resource.
2. A Risk Or A Liability?
Risks are generally perceived in a negative way but in fact they are merely events or circumstances that might occur and if they did, would affect the original plan or scope of the digitisation project. They certainly hold the potential to detrimentally affect the project but they can also affect the project in a beneficial way if they are properly planned for, responded to and managed. The “constructive suggestion” is perhaps an example of a risk that can be both beneficial and detrimental. Beneficial in that it may result in a significant improvement in the functionality of the end product, but at the same time detrimental in that it has a heavy demand on resources.
In contrast liabilities are events or circumstances that do adversely affect the success of the project. For example, the discovery that the software employed in a project can not support a particular browser represents a risk to the project. This risk only becomes a liability if it is not effectively managed. However, the use of alternative software could present the project with the opportunity to improve the functionality of its digital resource. Therefore, it is not helpful to always consider risk management in purely negative terms, because the end product of a risk is not always a fundamental problem but an opportunity.
3. Risk Management
Risk management is a systematic way of identifying potential risks within a project, gauging or estimating the probabilities of these risks occurring, to then develop strategies to manage these risks. In the commercial sector considerable resources are allocated to carry out quantitative risk assessments. Such resources are rarely available to publicly funded digitisation projects, nevertheless effective risk management plans can be put in place by adopting a systematic process of identifying and evaluating potential risks and using this analysis to develop strategies to manage and control them. This process can be broken down into three main processes: assessment, implementation and monitoring.
Risk assessment is the process by which potential risks to a project are identified and assessed, and appropriate responses to these risks are developed. Firstly, a list of the uncertainties involved in the project is produced. Secondly, the likelihood of these uncertainties occurring and the relative impact they could have are assessed. Thirdly, the risks are prioritised and strategies developed in order to minimise their seriousness.
4.1 Identifying risks
It is not too difficult to think of a number of 'generic risks' involved in digital resource creation. Listed below are the main potential areas of risks for a vast majority of digitisation projects:
- Creeping scope,
- Tasks involving third parties,
- Use of unfamiliar technologies, or emerging new technologies,
- Any part of the plan based on assumption rather than fact,
- Funders requirements,
- Institutional restraints,
- Insufficient skill levels,
- Recruitment and retention of staff,
- Relevant staff development,
- Supply of equipment,
- Equipment failure,
- Equipment support and knowledge,
- Changes in technology,
- Digital rights management,
- Clear workflow,
- Quality control,
- Undefined goals,
- Compliance to standards,
- Preservation and sustainability of resource,
TASI provides further specific information on areas of potential risk in digitising images. http://www.tasi.ac.uk/advice/managing/risk.html
However, whilst the above list covers some important areas for consideration each digitisation project is unique and risk identification must be project specific. In fact rather than using a list such as the one above to identify risk, which could lead to a rather formulaic assessment failing to take into account the unique parameters of a project, it is better to start by reviewing the project plan to identify potential risks and using a list like the one above as a cross check once a list of project specific risks has been developed.
JISC infoNet (http://www.jiscinfonet.ac.uk/InfoKits/risk-management) recommends that risk identification and risk analysis are not carried out by a single individual; rather the important stakeholders should be involved to some degree in the process. Widening participation in the risk analysis process helps counteract personal bias and can also lead to a more realistic identification of the full range of risks involved in a particular project, as each stakeholder brings their own areas of focus and expertise to bear on the risk assessment. It also ensures that stakeholders are aware of all the risks. By now a daunting number of methods and graphical representations exist to assist project managers and their teams in this process but the key issue is to use those diagrams or methods which can be intelligently implemented according to each project’s requirements. Affinity diagrams such as those used by the National Audit Reports Office (e.g. http://www.nao.org.uk/publications/nao_reports/03-04/03041078_casestudies.pdf
) reports can be easily adapted to digitisation projects, especially when these involve many affiliates to help identify and focus stakeholders’ concerns and priorities as to what the nature and relative importance of risks factors are. Ideally all stakeholders involved in the project should have a very similar perception of what the main project’s risks are prior to the commencement of detail risk analysis.
4.2 Risk Analysis
Risk assessment is not simply about identifying risks so that the project team and stakeholders are aware of them. It also involves assessing the potential severity of these risks, thereby identifying where to most effectively focus attention and resources in managing them.
In order to assess the seriousness of a potential risk it is necessary to estimate the rough probability of it happening and the impact, should it occur, on the project timetable, project costs and end quality of the digital resource.
4.2.1 Assessing Probability
Without the benefit of a quantitative assessment of the probability of an event occurring, an assessment remains largely subjective and has to be based mainly on common sense and experience. As such the assessment has to be ongoing and evaluations of the probability of risks happening revised as the project proceeds and the likelihood of risks happening becomes clearer.
4.2.2 Assessing Impact
Impact is the cost, time or quality effect a risk can have on a project. Whilst not a definitive guide Table 1 provides some basic guidelines on how to weight the impact of an event on a project.
|Very Low||Can be met with current funding||Slight slippage of internal targets||Slight reduction in quality with no overall impact on usability/standards|
|Low||Requires some additional funding||Slight slippage against key milestones or published targets||Failure to include 'bells and whistles' promised to stakeholders|
|Medium||Requires additional funding||Delay affects key stake-holders||Significant elements of scope or functionality will be unavailable|
|High||Requires significant additional funding||Failure to meet key deadlines||Failure to meet the needs of a large proportion of stakeholders|
|Very High||Requires substantial additional funding||Delay jeopardises viability of project||Project outcomes effectively unusable|
For example, in AHDS History's own JISC funded project www.histpop.org (Online Historical Population Reports (OHPR)), one of the main goals of the project was to create a web-based user interface for browsing, searching, viewing and downloading ca. 200,000 images of historical population reports. At the time this project was initialled one major risk to be identified was the adequate preservation of the scanned image data (about 2Tb in all). A poor preservation strategy of these images would have jeopardised the overall viability of the project and ultimately resulted in the project’s failure. Therefore, in the project's own risk assessment a defective preservation strategy for the images was evaluated as a risk that would have resulted in a very high impact. The measures taken to ensure that this did not happen were to rigorously impose data archiving and back-up regimes. However, this risk needs to be kept in proportion to the effect such a loss would have on meeting the users’ needs. The loss of a few pages within a volume of material can be accommodated, while the loss of a complete volume would be perceived as serious omission in the image collection.
Not all risks can be fully identified at the start of a project and incremental measures may need to be taken as the project progresses. An example of such a risk is again drawn front the OHPR project where, as seen in Figure 1, the quality (readability) of the scanned images failed to reach the required level. To ensure that the required level was reached, additional quality assurance resources and processes were introduced which were not originally budgeted for.
Figure 1. Scanned image of a Registrar General Report on Violent Death Statistics in England 1852-56 showing some ‘additional’ unwanted features
4.3 Calculating Severity
Once the probability and impact of a risk has been estimated, the potential seriousness of the risk can be identified. The scales for measuring the probability and impact of risks can be numeric or qualitative, but in each case it is important to be clear what the relative definitions mean. It is perfectly possible to weight risk in terms of high, medium or low; however, JISC infoNet suggests the use of a five-point scale.
|1||Very Low||Unlikely to occur||Negligible Impact|
|2||Low||May occur occasionally||Minor impact on time, cost or quality|
|3||Medium||Is as likely as not to occur||Notable impact on time, cost or quality|
|4||High||Is likely to occur||Substantial impact on time, cost or quality|
|5||Very High||Is almost certain to occur||Threatens the success of the project|
The attribution of numerical scales to risks at this point can be helpful in order to prioritise more precisely their relative seriousness. One way of achieving this is by multiplying a risks probability score by its impact score. In OHPR's risk assessment of the severity of the risk of losing its digital images the probability of it happening was 1 and the impact 5 it therefore was afforded a score of 5.
A vast majority of projects will find such a risk assessment sufficient. However, the drawback to this approach is that it does not take into account the relative potential seriousness of probability and impact. A risk with a high probability of occurring but that has a low impact on the project could obtain the same severity score as a risk that is unlikely to occur but if it did could ultimately threaten the success of the project.
One way of removing the 'averaging out' effect is to increase the relative weight afforded to high impact risks. As seen in Table 3, the numeric value assigned to a risk doubles as you move higher up the impact scale.
|Very Low||Low||Medium||High||Very High|
Thus a risk with a low probability but high impact is identified as being potentially more damaging than a risk with a high probability but low impact.
For example OHPR has assessed the risk of faulty equipment as:
|Risk||Probability (P)||Impact (I)|
If the seriousness of this risk was calculated (PxI) then the risk posed by faulty equipment (score = 6) would be considered more serious than loss of the image data (score=5). If the impact a risk may have on a project is attributed more importance, as outlined in Table 3, then the overall risk to the project of faulty equipment would be half of that of the loss of the digital images.
4.4 Planning a Risk Response
Decisions on the best course to take to in response particular risks should be based upon the risk analysis and ideally should take into account the benefit and cost of the risk response in relation to the probability and impact of the original risk. Ideally those risks with the highest level of seriousness should be considered first.
4.4.1 Types of Risk Response
All techniques to manage the risks fall into one or more of the following five major categories - avoidance, reduction, transference, deferral and retention.
Risk avoidance: Also known as risk removal or risk prevention, risk avoidance involves altering the original plans for the project so that particularly risky elements are removed. It could include deciding not to perform an activity that carries a high risk. Less drastically it could involve altering the activity in such a way that the risk is removed, for example OHPR ensures that multiple back up copies of images are taken thereby avoiding their loss.
Adopting such avoidance techniques may seem an obvious way to deal with all risks. However, often the areas of the project that involve high risks are also the areas of the project that potentially contain the highest worth or the best value for money. Avoiding such risks may also result in removing potentially the 'best bits' of a digital resource, and an alternative strategy that retains these risks may be more appropriate.
Risk reduction: Risk reduction or risk mitigation involves the employment of methods that reduce the probability of a risk occurring, or reducing the severity of the impact of a risk on the outcome of the project. The loss of highly skilled staff is a considerable risk in any digitisation project and not one that can (legally) be totally avoided. Suitable risk mitigation could involve the enforcement of a notice period, comprehensive documentation allowing for replacement staff to continue with the job at hand and adequate management oversight and the use of staff development programmes to encourage staff to stay.
Risk transfer: Risk transfer moves the ownership of the risk to a third party normally by contract. This also moves the impact of the risk away from the digitisation project itself to this third party, for example ensuring that the OCRing of digital text is quality checked by the supplier.
Risk deferral: The impact a risk can have on a project is not constant throughout the life of a project. Risk deferral entails deferring aspects of the project to a date when a risk is less likely to happen. For example managing the expectations users have about the content and delivery of a resource can be time-consuming, one way to reduce this risk is by not making a web resource available until user testing is complete.
Risk retention: Whilst a certain number of the risks to the project originally identified can be removed by changing the project plan or dealt with by transferring the responsibility of the risk to third parties inevitably certain risks have to be accepted as a necessary part of the project. All risks that have not been avoided or transferred are retained or accepted risks by default.
It is therefore important to develop appropriate plans outlining how these residual risks will be dealt with should they occur.
4.4.2 Choosing the right response
Whilst most risk responses fit into the categories listed above, in the real world risk responses are rarely confined to just one category. Decisions about the best course of action to take should be based upon the potential seriousness of the risk and the estimated cost of potential responses. The following examples provide a very rough guide to appropriate responses; however, the level of acceptable risk must be set according to each project’s circumstances:
High Probability, High Impact: this group of risks are more likely to be appropriately dealt with by avoidance or transference. At the very least steps should be put in place to mitigate the risk and they are likely to require the most rigorous contingency planning.
Low Probability, High Impact for this group of risks avoidance responses may be appropriate and the effect of the risks, if they should occur, be mitigated, and possible contingency plans devised.
High Probability, Low Impact. The probability of these risks should be mitigated and possible contingency plans developed.
Low Probability, Low Impact. The effect could be mitigated however the costs of a response might outweigh the potential impact of the risk to the project. In these cases it may be more appropriate to be aware of the risk and to accept the effects it will result in if it should happen.
4.4.3 Contingency Plans
For residual risks that may occur contingency plans should be developed in case they do. Contingency plans should be appropriate and commensurate to the impact of the original risk. In many cases it is more cost effective to allocate a certain amount of resources to mitigate a risk rather than start by developing a contingency plan which, if necessary to implement, is likely to be more expensive. The number of scenarios likely to require a full contingency plan depends upon the project. Contingency planning should not be confused with the normal re-planning necessary to react to minor changes in the developing project plan.
For further information on costing risks and contingency plans see JISC Infokit on Risk Management http://www.jiscinfonet.ac.uk/InfoKits/risk-management/costing-risk
Once the combination of responses to be used for each risk has been decided all the planned methods for dealing with the effects of the risks should be put in place. For those high risks for which an avoidance technique has been chosen, the project plan should be amended accordingly and the remaining risks outlined in a risk log or register.
5.1 Risk Log
The risk log or register is an effective tool which allows project members to record the identified risks, analyse their severity, and outline a response to be taken should they occur. The fields that it could contain are:
- Unique ID
- Expected level of risk once mitigating actions complete
- Early Warning Signs.
The JISC infoNet Project Controls Database includes a risk log that can be downloaded - http://www.jiscinfonet.ac.uk/InfoKits/project-management/pm-controlling-the-project-1.4
6. Risk Monitoring
Identifying, analysing and planning for risk is an important planning stage of any project. However, risk management does not stop once the risks have been identified. Initial risk management plans will never be perfect. Practice, experience, and actual loss, will necessitate changes in the plan and inform decisions about how to most effectively deal with certain risks. Risk identification and analysis should be ongoing throughout the project but particularly at project start-up and milestones. The best risk planning becomes useless unless a clear picture is maintained of how the project and its associated potential risks are developing. A reliable reading or assessment of a project’s situation can only be achieved through iteration, in other words, the continuous repetition of all steps of the risk management process. It is important to keep track of identified risks, to revisit the risk assessment to monitor the effectiveness of the chosen responses and to identify new or changed risks and to make the necessary changes to the risk log. This will mean that when a risk does occur your chosen response will be appropriate and more likely to be implemented effectively. It also means that you can communicate the risk to key stakeholders and demonstrate how it was anticipated and dealt with. A practical example of a risk monitoring action was the creation and implementation, by the OHPR project team, of a purpose design and built process management tool known as ‘the IssueTracker’. This tool consists of a website interfaced to a database which enables the sharing of the management of quality assurance between the OHPR team and the supplier of the image data. The Issue Tracker was specifically designed to fulfil the quality control management requirements of the OHPR project; however it is applicable to other digitisation projects and is now available as a download http://www.histpop.org/download/issuetracker/What_is_the_OHPR_IssueTracker.html.
Risk management is an essential component of managing a digitisation project, which, if applied and employed properly, can bring excellent dividends to the project through the reduction of programmatic and technical risks. It is also a systematic process that should involve the original assessment of risks associated with the project, the implementation of changes to the plans of the project if the risks involved are too high, and the transference and mitigation of high risk areas. For those areas of risk that are acceptable it means the development of appropriate contingency plans. Risk management, however, is not an event but a process and the identification, analysis and tracking of risk should be continued throughout the life cycle of the project.
However, whilst risk management is undoubtedly important in digitisation projects prioritising too highly the risk management processes itself could potentially preclude a project ever starting or completing. This is especially true if other work is suspended until the risk management process is considered complete. Therefore risk analysis should always be considered in conjunction with the overall aims of the project. If a project's value is dependant on a high level of risk then this level of risk must be considered acceptable. Risk analysis and management will always be a mixture of judgement and pragmatism.
8. Further Information
New Generation Managers’ a contribution by Simon Tanner on the 2003 December online magazine of the Chartered Institute of Library and Information Professionals CILIP.
Risk Managers for Information Technologists’ a contribution by Leslie Robinson on the October 2006 online magazine of the Chartered Institute of Library and Information Professionals CILIP.
The OHPR project’s Issue Tracker
Risk Management in Higher Education Report
JISC online resource to help project managers assess and deal with Risk
Risk analysis as part of JISC Project Management guidelines
Access Database to facilitate the recording of Risks, Issues and Changes and to help with monitoring the progress of actions taken
Information paper from Technical Advisory Service for Images
Broadly ranging essay showing problems of risk in use of ICT (pdf link)