Andy Smith
Mine action specialist

 

Evidence based field risk management

 

 

This paper was prepared for presentation at the CTRO Symposium in Croatia in 2020. The symposium was delayed and then cancelled, but the papers were finally published in the CTRO Book of Papers in December 2020. Unfortunately, their staff seem to have used very low resolution images. The correct images are shown here.

Click here to read this paper as a PDF file. The PDF file includes the footnotes.

In Humanitarian Mine Action there are two main types of field risk to be managed. The first is the risk of failing to enhance the safety of those using the land. If we release unsafe land, we increase the risk to people who had previously avoided the area but now believe it is safe to use. The second risk is that of staff suffering explosive related injury during their work.

The IMAS 07.12 Quality Management and 07.14 Risk Management both stress the need for decisions to be based on ‘evidence’. Also, the success of the approach to Land Release detailed in the IMAS relies entirely on the need to gather and act on evidence. An individual’s experience is recognised as evidence, with the constraint that there is a human tendency to believe that something personally experienced must be common when it may not be. A written record extending over time allows individual experience to be assessed in the context of a more objective overview and so is the best evidence available. It is true that there is often a human tendency not to be entirely honest or critical in a written report, but it is hard to conceal the facts in a well detailed report, so the truth often remains manifest to an informed reader.

If Land Release is to be managed without a reduction in safety, it is essential that the systems used to decide when to release land are tested. This should be done by checking whether unsafe land has been released. When explosive hazards are found on released land, these incidents should be investigated to determine how the error arose, so allowing the Land Release system to be refined and improved. This does not happen very often. Few countries treat these incidents in the same way as a demining accident and genuinely investigate with the goal of improving their Land Release systems. For most, finding explosive hazards on land that has been released is an embarrassment because it implies error. Often, no individual is at fault but the new evidence gained from an investigation can allow the imperfect system to be improved – which is central to Quality Management.

wheel_of_advance

Although some INGOs have done so, there is little evidence that the leaders in HMA have adopted the self-critical ‘wheel’ of review and improvement that the IMAS require. Although it is often called a ‘cycle’, the process is not a cycle that returns to the same point: it is a wheel that progresses forward over time. Even if a system were thought 'perfect' today, it can only remain 'fit-for-purpose' with constant evaluation and appropriate revision.

GICHD and UNMAS have not supported the gathering of incident and accident data despite it being critical when evaluating and improving performance in HMA. Their failure may reflect the fact that the evidence would show that our supposed successes in increasing efficiency are not all that they seem. Lowering cost can look like an improvement in efficiency, but that is a falsehood when doing so means that the primary goal of all HMA has to be ignored to get there. I cannot publish proofs that this has occurred but I have often shown evidence that risk management issues have not been given the priority that might be expected in a humanitarian endeavour.

Accidents and incidents

In this paper, a demining ‘accident’ is defined as an unintended explosive-related event that occurs during demining activities. An ’incident’ is defined as the discovery of an explosive hazard on land declared safe for release. Both should be investigated whether or not injury results, and the investigations should be shared so that lessons can be learned.

Orinal database software cover

There is a demining accident and incident database. I started it in 1998 and first published it in 1999. UNMAS gave me a letter of support and I did my best to keep it up to date until 2010 when GICHD launched its RAPID spreadsheet accident record. They did this at a National Directors Meeting in Geneva where MACs were told not to send me reports anymore. I have continued to add data when I get it, but have not been able to gather enough. At every opportunity over the next six years I asked GICHD to reverse this decision and either take over my database or get MACs to send me their accident reports. I was repeatedly told that while individuals recognised its value, there was no money for them to take over the database. In 2017, I approached UNMAS asking for a letter of support for the database that I could send to the MACs when asking for data. They also recognised the value of the database but my request was refused.

In late 2017, the CISR at James Madison University agreed to take over my database and store it in their data repository. They renamed it AID – the Accident and Incident Database for HMA. I was hopeful that they would maintain it but they have not found a way of gathering data, so the only data added since they took it has been collected by me. The AID database at the CISR is effectively a back-up that needs funding support to turn into the useful tool that it should be. And it is a useful tool.

In 2017, I was contracted to produce a range of field risk management training presentations for GICHD – drawing heavily on the evidence in the database of accidents and incidents to do so.

Just this year (2020) I was contracted by UNDP to assist with the revision of the national standards for Lebanon. This involved integrating new Risk Management and Quality Management IMAS in their National Mine Action Standards (NMAS). The need for evidence on which to base decisions was clear, but they did not have most of their own accident and incident data. This is not unusual. NPA, MAG, DDG and HALO have asked for their past records at one time or another, as have two national MACs. (I have always answered these requests at no cost.) So I was able to give Lebanon back more than 50 of their own accident and incident records, add recent reports, and present them with a working database of evidence to use in their field risk management efforts.

page_from_database

The Lebanon Mine Action Centre is impressive despite the inevitable constraints associated with being a military centre in which staff rotation can lead to experience being lost. An easily accessible record of accidents and incidents (and the previous attempts at risk management) was especially useful to them.

Online_training_resource

While in Lebanon, I was asked by NPA to go to Kosovo to advise over PPE needs. Using relevant records from the database, I was able to give pragmatic, evidence-based advice that was well received by the people who will be taking the risks, both managers and field staff. I also left them with a working database of the accident records for Kosovo.

In both Lebanon and Kosovo, the accident record I hold is particularly well detailed and includes some records of incidents when hazards were found on land that the INGOs and commercial companies had declared clear. For this I should thank John Flanaghan and Chris Clark, both pioneers of Quality Management. 

The first version of the database software was poorly designed because I did not know all of the questions that I would need to ask until I was actually using it. I quickly made revisions, but the structure needed to become a ‘relational’ database and that needed a complete software rewrite. When that was funded by the UK government in 2003, that was the only time that the database has received any funding support. The new software was released in 2005 when I also published the records on-line.

online_database

The records have no names of individuals or demining groups, but the best include enough detail about the circumstances surrounding the event to be really useful in risk management and training.

Today, with more than 2GB of data inside it after 15 years, the software has become out of date again. It creaks and labours unless the main dataset is divided, but it still works perfectly with smaller datasets. I also keep the original accident and incident files, which often include far more photographs and ancillary papers than can be included in the database itself. Keeping the original papers allows for anyone with a particular interest to request access.

For the past ten years, I have been offering all of this to anyone who will keep and maintain it and now the CISR has shown interest. At the National Director’s Meeting (NDM) in Geneva in February 2020, there was a side event to discuss the need for accident and incident data. CISR asked me to come and explain the database. I agreed to do so at my own expense but GICHD vetoed my attendance so the invitation was withdrawn. GICHD objected because I have criticised GICHD for its failure to value or support the database in the past, and GICHD cannot abide criticism, no matter how justified. They should remember that only by being critical, and self-critical, do we learn – which is a principle of the Quality Management that they ask others to adopt. Yes, I have made fun of their RAPID spreadsheet alternative which reduces all accident records to a few ill-chosen options in limited drop-down lists. It is inappropriately designed, has been very poorly supported, and is worthless as a field risk management tool. These facts were admitted during the side-meeting at the NDM.

The meeting at the NDM centred around plans by GICHD and the three biggest INGOs to start an entirely new database, designing it from scratch. They will seek funding for software development and then the INGOs themselves will decide what goes into it, and what comes out. While one of those INGOs has a good record for sharing accident data, the others do not. GICHD will control the programme, and its record over valuing the collection and sharing of accident and incident data is not what one might wish. It has never been a good idea to expect competing agencies to police themselves, and it is not necessary.

There may be no one at GICHD today who saw the 2005 revision of the database software when it was released. The UK government channelled its funding through GICHD and I was contracted to design and manage the work. Ironically, given the antipathy towards it, today’s leadership probably does not know that the software bears the GICHD logo and that GICHD was once proud to be a part of the endeavour.

DDAS_with_GICHD_logo

The Intellectual Property Rights are mine, but I have always been willing to cede them freely to anyone who wants to use or improve it while maintaining transparency, honesty, and access. The data inside it belongs to us all.

I cannot keep the database going. CISR has the software and a name-free dataset, so they could – but they would need UNMAS support over gathering data and would need to spend money on data entry and analysis. If they got both, I would happily give them all the back-up data and supporting records that I have collected. For a limited time, I would also be prepared to consider gathering data, showing people how to enter records and showing how to use the database. Alternatively, if GICHD support for this were evident, a tacit acknowledgment of error would be implicit and I would feel obliged by my Quality Management principles to forget our past differences and cooperate with them. There really is a need for management to start leading by example.

Postscript: Since this paper was written GICHD has found a donor and started to develop its own accident database with bespoke software (so no access by others) and without reference to the existing database. Reinventing the wheel, GICHD and the three largest INGOs in demining will dictate what is recorded and what is shared, so effectively policing themselves. GICHD's past record makes it reasonable to predict that the software will go through several expensive iterations before it functions, what is recorded in it will be internal accident investigations that are sanitised to the point of absurdity, and what detailed content there is will not be shared with anyone. This will not help to improve their game, but there is no evidence that they want to do that.


 

HOME