This book presents a Human Factors and Ergonomics evaluation of a digital Mission Planning and Battlespace Management (MP/BM) system. Emphasis was given to the activities occurring within Brigade (Bde) and Battle Group (BG) level headquarters (HQ), and the Human Factors team from the HFI DTC distributed their time evenly between these two locations. The insights contained in this volume arise from a wide-ranging and multi-faceted approach, comprising:
• observation of people using the traditional analogue MP/BM processes in the course of their work to understand how analogue MP/BM is used in practice;
• constraint analysis (Cognitive Work Analysis, CWA) of the digital MP/BM system to understand if digital MP/BM is better or worse than the conventional paper-based approach;
• analysis of the tasks and goal structure required by the digital MP/BM, to understand the ease with which the activities can be performed and identify the likely design-induced errors;
• analysis of Distributed Situation Analysis (DSA), to understand the extent to which digital MP/BM supports collaborative working;
• analysis of the social networks that the digital system allows to form spontaneously (to understand the way in which people choose to communicate via voice and data);
• assessment against EEMUA 201 (Engineering Equipment & Materials Users Association) to understand if digital MP/BM meets with best Human Factors practice in control system interface design;
• assessment against a Usability Questionnaire, to gauge user reactions about the ease or difficulty of using the digital MP/BM system); and
• an environmental survey, to understand the extent to which the Bde and BG environment within which people are working meets with British Standard BS/EN/ISO 11064 Environmental Requirements for Control Centres.
A brief summary of the chapters of the book are presented next, with the detailed description of methods, approach, findings and recommendations within the main body of the book.
Chapter two presents an overview of the Human Factors and Ergonomics discipline and the methods associated with it. The discipline is introduced with a few examples of how it has contributed to improved display and control design. This is consistent with the overall aim of improving the well-being of workers, as well as their work, and the general goal of improved system performance. Two examples in particular resonate with the purpose of this book, both taken from aviation over 60 years ago but still with relevance today. Safety of systems is of major importance in Human Factors and safety critical environments have much to gain from its application. Human Factors and Ergonomics offers unique insights into the way in which people work, through the understanding of the interactions between humans, technology, tools, activities, products and their constraints. This understanding is assisted through the application of Human Factors and Ergonomics methods, which are also introduced. Some of these are pursued through the rest of the book. They offer complementary perspectives on the problem and can be used in an integrated manner.
Chapter three presents observational studies of the tasks people were undertaking in the HQs prior to digitisation. The conventional, analogue mission planning process is examined with the objective of identifying the Ergonomics challenges for digitisation. Prototypes of digital mission planning systems are beginning to be devised and demonstrated, but there has been concern expressed over the design of such systems, many of which fail to understand and incorporate the human aspects of socio-technical systems design. Previous research has identified many of the potential pitfalls of failing to take Ergonomic considerations into account, as well as the multiplicity of constraints acting on the planners and planning process. An analysis of mission planning in a BG is presented, focussing on the tasks and the products produced. This study illustrates the efficiency of an analogue process, one that has evolved over many generations to form the Combat Estimate, a process that is mirrored by forces throughout the world. The challenges for digitisation include ensuring that the mission planning process remains easy and involving, preserving the public nature of the products, encouraging the collaboration and cooperation of the planners, and maintaining the flexibility, adaptability and speed of the analogue planning process. It is argued that digitisation should not become an additional constraint on mission planning.
Chapter four presents the constraint analysis performed on the digital MP/BM. This approach, realised through CWA deconstructs the system into different levels of abstraction:
• Functional Purpose (that is, the reason that the system exists; for example, conduct planning to enact higher command’s intent);
• Values and Priorities (that is, the measures of success for the system; for example, maintain digital MP/BM effectiveness, minimise casualties, reduce time to generate products and so on);
• Purpose Related Functions (that is, the functions the system is performing; for example, coordination of units, position and status, threat evaluation, resource allocations and so on); and
• Object Related Functions (that is, what the physical objects in the system do; for example, data transmission, voice communication, blue positions, red positions, effects schematic and so on).
This Abstraction Hierarchy (AH) was then used as a basis for interviewing staff officers at BG and Bde level, to find out if digital MP/BM was significantly better, the same, or significantly worse than conventional approaches. The findings showed that the system offered little to support planning, with none of the respondents offering a positive opinion of the system’s ability to aid the planning process. Further examination of the results showed that the digital MP/BM estimate process was generally unsupported by the digital system and that in many cases digitisation had a negative effect on tempo, efficiency, effectiveness and flexibility. The participants offered a positive rating for the system’s ability to support battlefield management; however, examination of the results reveals that this positive rating can be mainly attributed to the secure voice radio facility rather than the digital MP/BM elements of the system.
Chapter five presents a deconstruction of the activities performed in the operation of the digital MP/BM system. The deconstruction takes place under the rubric of Hierarchical Task Analysis (HTA) and creates a hierarchy of goals, sub-goals and plans. This analysis produced a task model for each of the seven questions in the Combat Estimate. This offers a much higher fidelity of analysis for the steps involved in producing the Combat Estimate products, as overviewed in chapter three. The HTA was used as the basis for investigating the ease or difficultly with which the operations on the digital MP/BM system could be performed. Examples of the difficulties encountered are presented together with suggested remedies in the redesign of the system. The HTA also formed the foundations for human error identification analysis using the Systematic Human Error Reduction and Prediction Approach (SHERPA) method. The SHERPA taxonomy was applied to the HTA in order to identify likely error modes. Examples of the types of design-induced errors that users may be likely to commit are presented. These errors are also intended to focus attention on remedial strategies and stimulate design solutions.
Chapter six presents an evaluation of DSA during mission planning and execution activities supported by the digital MP/BM system. The analysis was undertaken using a mind mapping approach in order to understand how information and knowledge was spread around the various agents (including the human players, artefacts, products and materials). This analysis was split into three parts: Situation Awareness (SA) requirements analysis, analysis of SA during planning tasks and a corresponding analysis for operational execution tasks. The SA requirements analysis indicated that the system is not designed to support the range of distinct user SA requirements present in the MP/BM system. The analysis of the DSA during the planning phases revealed questions about the timeliness and accuracy of information, the tempo of planning in digital MP/BM, the accuracy of information from digital MP/BM (as staff had to engage in additional checking activities) and the poor support for different SA requirements in the different planning cells. Analysis of the operation execution tasks revealed that the Local Operating Picture (LOP) was often out-of-date or spurious (clarification of Own Situation Position Reports (OSPR) data was required, placing more load on the secure voice channel for updates of the blue force positions) and that there was a low level of trust in the LOP and OSPR (requiring the operations cell to compensate for digital MP/BM’s shortcomings by drawing blue force positions directly on to the Smartboard – but these were wiped off every time digital MP/BM updated or was changed). In summary, it was found that DSA was not well supported by digital MP/BM as different personnel have different SA requirements, which are subject to change, depending upon their tasks and goals at any moment in time.
Chapter seven considers the analysis of networks in digital Network Enabled Technology. The ideas behind self-synchronisation of people in networks adapting to changes in the environment are presented. Social Network Analysis (SNA) offers the means to populate the NATO SAS-050 model of command and control with live data, ascribing network metrics to each of the NATO model’s three primary axes: patterns of interaction (from fully-hierarchical to fully-distributed), distribution of information (from tight-control to broad-control) and decision rights (from unitary to peer-to-peer). This usefully extends the model and enables it to meet several critical needs, firstly, to understand not where a command and control organisation formally places itself in the model but where it ‘actually’ places itself. Secondly, to see how the command and control organisation’s position changes as a result of function and time. And finally, to understand the match between the place(s) the organisation occupies in the so-called ‘approach space’ and how they map over to a corresponding ‘problem space’. In this respect the analysis revealed a mismatch, which powerful examples of emergent user behaviour (in terms of unexpected system use) tried to resolve. In other words, the human-system interaction proved to be highly unstable, but the good news was that the underlying communications architecture was able to facilitate rapid reconfigurations. Using SNA to numerically model the behaviour of the system/organisation provides insight into tempo (with characteristic patterns of reconfigurations in evidence), agility (as modelled by the different areas occupied by the organisation in the NATO model) and self-synchronisation (as evidenced by emergent behaviours). As well as these insights, the modelling work now provides a benchmark for future iterations of the system.
In chapter eight the ‘look and feel’ of the MP/BM’s Human Computer Interface is assessed for compliance with EEMUA 201 guidelines. EEMUA 201 represents accepted industry best practice for the design and operation of control system Human Computer Interfaces. Ideally, the interface for digital MP/BM should be designed to allow staff officers and clerks to conduct their tasks effectively and efficiently. This means in turn that it should conform to their expectations and allow them to find information and perform tasks in a straightforward manner. The EEMUA guidelines are therefore an excellent basis for a review of existing systems or for the design of new systems. EEMUA 201 covers all of the important Human Factors concerns in this setting, such as: the number of screens, navigation techniques, use of windows, screen format and layout considerations. The findings of this analysis show that measured against the 35 EEMUA 201 principles the digital MP/BM only met eight of them. Twelve principles were partially met (for which some improvements to the current system are recommended), whilst a further eight principles failed to be met at all (for which significant shortcomings in design were identified). A further seven of the EEMUA 201 principles were deemed not applicable to digital MP/BM.
Usability assessment was undertaken with a Human Computer Interaction (HCI) questionnaire in chapter nine, which was completed by the staff officers and clerks who used the digital MP/BM at BG (13 respondents) and Bde levels (26 respondents). There were fewer staff in BG, which was reflected in the respondent numbers. The questionnaire comprised nine main sections designed to assess the usability of a particular device or system:
• visual clarity (the clarity with which the system displays information);
• consistency (that is, consistency of the interface in terms of how it looks, the ways in which it presents information and also the ways in which users perform tasks);
• compatibility (that is, the system’s compatibility with other related systems);
• informative feedback (that is, the level, clarity and appropriateness of the feedback provided by the system);
• explicitness (that is, the clarity with which the system transmits its functionality, structure and capability);
• appropriate functionality (that is, the level of appropriateness of the system’s functionality in relation to the activities that it is used for);
• flexibility and control (that is, the flexibility of the system, and the level of control that the user has over the system);
• error prevention and correction (that is, the extent to which the system prevents user errors from either being made or impacting task performance); and
• user guidance and support (that is, the level of guidance and support that the system provides to its end users).
The overall ratings were generally lower at BG, but even at Bde level the overall ratings failed to go beyond neutral. The system was rated particularly low on ‘explicitness’ and ‘error prevention and correction’. This is mainly because the personnel using the system did not find it intuitive, with some losing work altogether due to inadvertent errors.
For the sake of completeness, chapter ten presents assessments of the physical environment within which digital MP/BM was being used. This was not intended to inform the design of digital MP/BM, rather it was to consider if the surrounding environment met with current standards for control centres (that is, BS/EN/ISO 11064 Environmental Requirements for Control Centres). Whilst the comfortable and benign operational environment found in civilian domains, and to which current best practice and guidelines applies, may not be directly relevant to military domains, there remains an inviolable duty of care and opportunities to learn lessons. From a human performance point of view, the command and control environment is generally too cold. Digitisation, and the requirement this brings for sedentary computer-based work, is unlikely to improve this situation. Noise levels approach harmful long-term exposure levels and maintained levels are well in excess of best practice. Air quality (and associated low-level health symptoms) is poor. Lighting would also fail to meet comparable civilian standards; it is too dark overall but has poor directivity meaning that, paradoxically, there is also too much glare. Given the safety critical nature of the tasks being undertaken, comparison against acknowledged best practice sees the present environment as being sub-optimal for safe and efficient human performance.
Chapter eleven of the book presents a summary of all the preceding chapters, drawing all of the main findings together. Conclusions on the extent to which the digital MP/BM system meets Human Factors and Ergonomics criteria are drawn from the Constrains Analysis, HTA, DSA, SNA, System Control and Data Acquistion (SCADA) Analysis, Usability Analysis and Environmental Survey. This represents a very thorough assessment of any new system, but digitisation brings with it additional requirements that have important ramifications, and therefore cannot be undertaken lightly. The recommendations for short-term improvements in the current generation of digital MP/BM system are divided into five sections: general design improvements, user-interface design improvements, hardware improvements, infrastructure improvements and support improvements. Looking forward to next generation digital MP/BM systems, general Human Factors design principles are presented and Human Factors issues in digitising mission planning are considered. Future system design would do well to consider the Human Factors methods presented in chapter two, the understanding gained from mission planning demands and constraints in chapter three, together with the insights gained in the various analysis from chapters four to ten. The design of the digital MP/BM system should not become one of the operational constraints in mission planning and battlespace management. The science and practice of Human Factors has much to offer in terms of resolving this situation, provided that it is applied at the beginning (and then throughout) the system design lifecycle, rather than at the end when its impact is significantly diminished.