|
Preventing design error – a human factors approach
PhD thesis report by Ron Day This report has been written for those people who took part in the surveys or interviews used to gather data for the study, and for others who have expressed interest in knowing more about the research and its results. This brief report summarises just some of the content of the thesis and includes:
Serious accidents occur around the world on a daily basis. Those accidents include every form of transportation – air, rail, road, and sea. They also occur with other safety critical groups that include: Power generation and distribution, Medical procedures, Defence Force activities, and Emergency Service operations including Police, Fire, Ambulance, and Helicopter Rescue. Accidents occur on industrial sites and mining sites. They occur in workplaces everywhere. Sometimes those accidents are caused by extreme weather occurrences such as cyclones or tsunamis, or are the result of terrorist activities. But often blame can be directed at human error or technological fault. Increasingly, as technologies, systems and processes become more complex, undetected errors in the design process emerge as design-induced errors. Statistics suggest that the figure for design-induced rail accidents in Australia may be as high at 10% and the cost more than $30 million a year. Background No matter whether the designer is creating a new mousetrap, an App for an IPhone or a process to guide workers changing the fuel rods in a nuclear plant, a similar design process will be used. Most designs are required to solve a problem, so it is necessary to identify the problem and create a suitable design concept, transfer that concept to a set of specifications, build the solution and test it before distributing it to the users. This process is called the Systems Development Life Cycle (SDLC). Over the last forty or more years, many different SDLCs have been developed. Most can be described as traditional models, and a few more recent ones are known as adaptive models. Traditional SDLCs, such as the Waterfall model, have a set process that most often excludes end-users or minimises their input into the design process. The result can be lengthy delays in delivery, and higher costs due to the need to remove design-induced errors or oversights that prevent the resulting design meeting operational requirements. Adaptive models mandate the participation of end-users, particularly during the formation of a design concept, and testing and implementation stages. For almost forty years Scandinavian countries have legislated for participative design, where end-users are involved in the design of the tools and processes they will use. It is now widely accepted that involving end-users in the design process normally leads to safer design solutions that are cheaper, delivered sooner, and are almost error-free, requiring little or no retrofitting. Research method This study investigated the design processes used by IT designers, engineering firms, and business process developers; particularly those associated with high risk activities. Designers and end-users of technologies, devices, and business systems were surveyed and interviewed, and accident databases from around the world examined. One hundred and eighteen designers and one hundred and thirty eight end-users took part. Seventy six accident databases from around the world were examined. Surveys were handed out at conferences or posted on appropriate group sites on LinkedIn. They were completed by people from many countries, the greatest number of responses coming from Australia, USA and the UK. Findings It was discovered that three quarters of the designers (75.1%) who participated in the study used a traditional SDLC and did not involve ends-users in a recent project. Less than a quarter of designers (23.4%) used an adaptive SDLC where they consulted with end-users. A small number said they did not use an SDLC. End-users however, almost unanimously (98.6%) indicated that they believe they should be involved in the design of the tools they will use, quoting their more detailed knowledge of the tasks that need to be carried out. It was shown that designers have most stakeholder contact with their ‘clients’, i.e. the people who commission them to create a new design and pay for the work. End-users working in large organisations indicated that they often have no more contact with upper-level management (‘clients’) than they do with designers, complaining that those described as ‘clients’ by designers often have a flawed knowledge of workplace operations. Unfortunately they are the people who engage designers and provide the design specifications. The communication breakdown is described in the Disconnect Model (Figure 1). 
Figure 1 Designer/client/end-user disconnect model (Day 2013)
This model shows the communication flows between client, designer and end-user. The communication between designer and client is much stronger than between designer and end-user. The model also indicates the way the original design requirements normally come to the designer from the client, with little or no consultation with end-users. The complex pressures of expectations and constraints (money, time, resources) from both the client and design company leadership can be seen to have an impact on the designer’s ability to conceptualise and build a safe and efficient solution that meets operational requirements. The study also highlighted the fact that errors can occur in every stage of the design process, with the design concept and testing phases being the highest risk areas. Some of those errors were shown to occur because of lack of designers’ knowledge of end-user requirements, some because of lack of attention to detail when transcribing the design concept into specifications, some because of human factors issues, some because of insufficiently rigorous testing, and some because of poorly executed implementation and training of the new design. Business processes operating in design situations were also shown to exacerbate error potential – particularly when they impose constraints, such as limited budget, short delivery time, and lack of appropriate resources. These factors impact on the quality and safety of the designed product. Designers and end-users were asked to identify some of the design-induced errors that occur with computer systems and control devices. There was some agreement between the two groups on the following items:
- No online help - Dual key operation where one key can have several actions depending on whether it is struck on its own or paired with Shift, Ctrl or Alt keys - Operational button hidden behind another screen - Hidden or hard to find functions - Difficulty moving from one operation to another.
- confusing menu choices - unclear or absence of error messages - poorly constructed or missing online help
- controls are placed too far away from the equipment they operate - controls operate in the opposite direction to that expected. Other items mentioned included:
Discussion and Conclusions The results indicated a complex web of interactions that have a bearing on the occurrence of design error in new technologies, but the two activities that have the strongest influence are a flawed design concept and a lack of thorough and comprehensive testing. The flawed design concept occurs when designers are not sufficiently aware of the breadth and depth of operational requirements. As one designer admitted: ‘Misunderstanding of the operational requirements is the biggest cause of design error.’ Errors in testing can occur when the design concept is flawed and testing is misdirected. Testers may have even less appreciation of the requirements of the operating environment than designers. Both these issues need to be addressed by encouraging the participation of end-users in the design process. These are the people who know what is needed in their work. They need to be consulted during the concept formation discussions to ensure the design will address the real requirements, not the perceived requirements of remote designers and ‘clients’ i.e. organisational managers. They are also the people who can test the system against their requirements and indicate where problems exist. With their assistance, errors can be removed before deployment into the work setting. As one end-user interviewee said: ‘We were trying to convince the designers to learn how we do the business rather than work from what they assume we do. We were saying “We know our business. This is the way we need the system to work to allow us to do our jobs effectively and efficiently. Please design a system for us that works in the way we need.”’ The study points to a number of strategies that can be adopted to lower the incidence of design error and create safer designs sooner and for a lower cost. It should be borne in mind that these strategies can be equally useful when reworking or upgrading existing technologies or processes. The list includes: 1. Select an Adaptive Systems Development Life Cycle model for your next design. 2. Ensure that some end-users are included in the stakeholder panel. 3. Be sure to include end-users in the concept formation stage, the testing stage and the deployment stage. 4. Encourage clients to communicate more closely with end-users so their appreciation of operational requirements is accurate. 5. Review the constraints and expectations placed on designers, and ensure time limits, budgets and resources are appropriate to produce a safe result that satisfies operational needs. New tool As a result of this research, I have devised the Design Risk Error Avoidance Measurement analysis tool (DREAM). This tool is being created to assist investigators to identify where errors may occur and calculate a risk co-efficient of your design process. When risk factors have been identified, remedial action can be specified. A modified version of the tool will be available for use during accident investigations to identify whether a design-induced error was a causal factor. Design errors are intruding on safe and effective design all the time. When was the last time you had to deal with a problem with a design or process that caused an incident or accident, or didn’t meet operational needs? Did the design have to be retrofitted/ reworked/ modified? How much time and money did that exercise cost you? What was the cost to your organisation in lost time or loss of sales? If you would like to improve your design implementation performance, I can help. I can discuss solutions with you, train your design team, and conduct a full scale analysis of a project using the DREAM tool. Send an email to [email protected] or call the Safe Design Solutions office on 07 54 981 332 for details and to make a booking. Check out our website at www.safedesignsolutions.com. New book This brief report offers a taste of the results of this study. A much fuller account will be included in a new book Design Error – the enemy within. This book will spell out in detail the kinds of errors that can occur in the design of IT systems, engineering devices and business processes. These errors can, at best result in delivery delays, cost blowouts and the release of unsafe products and services, and at worst cause accidents, injury and death. Strategies for dealing with design error and avoiding unwanted consequences will be described. Ask for your copy at your booksellers later in 2013, or email the author to reserve your copy. Questions and comments, or requests for consultancy services should be directed to Ron Day, Design and Human Factors Specialist via ron@safedesignsolutions.com or call the Safe Design Solutions consultancy office on 07 54 981 332. Check out our website at www.safedesignsolutions.com. |
|
|
