Vol. 5, No.2 August 1998 Selecting Consultants When you choose consultants, be picky. Consider the following criteria: They apply the service (consulting or training session) to the real world problems of your organization. Hypothetical case studies are not used. They provide you with ideas or assistance in transitioning the concepts into use after the initial education. They are willing to customize the service to make it right for your situation. They provide a lifetime warranty on each service. You can call them at any time, and you don't need a new contract for them to answer the phone. They return all phone and E-mail questions, before the next business day. They don't leave you hanging for two weeks on important decisions. Their previous clients are satisfied. You checked. If they don't have the expertise, they admit it, and refer you to someone who does. There is no need to settle for second best. ----Neil Potter

Coping With Risk by Mary Sakry If your project (software product or process improvement effort) is plagued with problems, you might need effective risk management. Risk management is similar to performing preventive health care and buying insurance for your project. It involves identifying potential problems (risks), analyzing those risks, planning to manage them, and reviewing them. On the last page of this newsletter, we have provided you with a copy of our risk process. It is simple, effective and takes 60 to 90 minutes. Risk Identification To identify risks, we must first know what one is. When people begin performing risk management, they often start by listing known problems. Known problems are not risks. Risks are potential  problems.  During risk identification you might notice some known problems. If so, just move them to your current problems list. Then concentrate on future potential problems. Risk identification can be done using a 30-minute brainstorming session. Be sure to invite anyone who can help you think of risks. Be creative and invite the project team, customers, people who have been on similar projects, and experts. Example risks are: "We may not have the requirements right," "The technology is untested", "Key people might leave," "Their server won't restart in situation X," "People might resist the change." Any potential problem or critical project feature is a good candidate for the risk list. Risk Analysis The purpose of analysis is to set priorities and determine where to focus the risk mitigation efforts. Some of the identified risks are unlikely to occur and others might not be serious enough to worry about. During analysis, you will discuss each risk item to understand how devastating it would be if it did occur and how likely it is to occur. Most people use a rating system (see process) to help select the most important ones to work on. For example: if you discussed the risk of a key person leaving, you might decide that it would have a large impact on the project, but that it is not very likely. The risk items that have a high likelihood and a high impact will typically be the ones to select. Risk Management Plan There are two things one can do to manage risk. The first is to take action to reduce (or partially reduce) the likelihood of the risk occurring. For example, some part-time process improvement teams make their deadlines earlier and increase their efforts to minimize the likelihood of team members being pulled off the project due to changing organizational priorities. In a software product, a critical feature might be developed first and tested early. Second, we can take action to reduce the effect if the risk does occur. Sometimes this is an action prior to the crisis, such as the creation of a simulator to use if the hardware is late. At other times it is a simple backup plan such as running a night shift to share hardware. For the potential loss of a key person, for example, we might do either of two things: 1) We could plan to reduce the impact by making sure other people become familiar with that person's work or 2) To reduce the likelihood of attrition, we could give the person a raise, or provide daycare. Risk Review You will want to review your risks periodically so you can check how well mitigation is progressing. You can also see if the risk priorities need to change or if new risks have been discovered. You might decide to rerun the complete risk process if significant changes have occurred on the project. Many people incorporate risk review into other regularly scheduled project reviews.   Risk Management Process 1. Determine scope of the risk session. 2. Select the team and moderator. The moderator explains the risk process to new team members. 3. Identify risks (potential future problems) Brainstorm areas of risk, e.g., Weak areas such as unknown technology. -new, or new to the team (e.g., development tools, target machine) Things that are critical or extremely important to the effort -such as the timely delivery of a vendor's database software, creation of translators, or a user interface that meets the customer's needs Things that have caused problems in the past. -such as loss of key staff, missed deadlines, or error-prone software. Remove invalid or irrelevant items. Current problems should be treated as problems, not risks. 4. Analyze risks. For each risk item: Does the team understand the risk item? -If necessary, split into separate risk items, e.g., -Disk may overload under condition X. - Disk may overload under condition Y. Discuss and determine its scope; - What would the consequences be if this risk item did happen? Determine what the impact would be if the worst happened, using a scale of one to ten. Determine how likely it is that the risk item will occur, using a scale of one to ten. Determine the priority of the risk items and thus which to work on (impact vs. likelihood) 5. Plan to mitigate risks. Select the most important risk issues, such as the top two or three, or top 20%. Brainstorming on actions that could be taken to reduce the likelihood of the risk item occurring. Decide which actions to pursue. Select a person to be responsible for each action chosen. Document the information in the risk management plan. 6. Review risks. Establish how often risks should be reviewed (once a month is typical). Risk reviews can be incorporated into existing project status and phase reviews. Update the list based on risk review sessions. (C) 1998, The Process Group, All rights reserved. Top of newsletter Measure Your Effectiveness... If You Dare by Neil Potter Service providers within a company often struggle when measuring their effectiveness. This article will describe two ways to measure the effectiveness of groups such as a Software Engineering Process Group (SEPG), Process Improvement Team or Software Quality Assurance group. Here I will use the term "process team" to refer to any type of process service provider. Two simple, but commonly overlooked measures are: Customer evaluation of the process team's performance The improvement results of the organization The premise behind these measures is that a team providing the process services is fully responsible for the quality of the service provided and shares responsibility for the improvement results when those services are used. Customer Evaluation of the Team's Performance A customer evaluation form can be used to determine the effectiveness of a process team. Some examples of when you might evaluate your performance would be at the end of moderating a peer review, teaching a risk management process, conducting a requirements gathering session, or helping someone do statistical analysis. The form can be used for one session, or at the end of a group of sessions. The average score for the Overall Session Rating (item #6) can become one data point in a spreadsheet that records the team's performance over time (see graph). If you are feeling queasy about obtaining your customers' opinion of your performance, realize that this piece of  data will give you the most direct feedback about your current effectiveness. Knowing how you are doing is the quickest way to improve. Improvement Results A second way to determine a process team's effectiveness is to look at the improvement results of the organization since the team started. A process team needs to share the responsibility for the organization achieving its improvement goals. Example improvement goals include: Improve development schedule accuracy by 20% Improve product quality by 10% (# defects found per release) Achieve SEI Level 3 or ISO9001 The primary owner of these goals should be the senior manager of the development staff. The process team should have partial responsibility, not sole responsibility. The assumption is that any one group can cause an improvement program to fail, but no one group can ensure its success. In summary, the effectiveness of any service group can be measured. A customer survey is an immediate measure and helps a team improve its performance. Improvement results are a measure of the impact a group has, and should be jointly shared with the senior manager of the organization. Session Evaluation Form Thank you for using the process team for this session. We would be grateful if you would complete the following questionnaire. This will help us improve to better serve your needs. Your Name________________________(optional)   Job Title___________________________ Poor Average Good V. Good  Excellent 1. Clarity of session objectives? 1 2 3 4 5 2. Usefulness of session to your work? 1 2 3 4 5 3. Our knowledge level? 1 2 3 4 5 4. Questions answered clearly & completely? 1 2 3 4 5 5. Considerate and courteous? 1 2 3 4 5 6. Overall session rating? 1 2 3 4 5 Major strengths of the session? Recommended improvements for the session. Additional comments. What additional help would you like? Top of newsletter Quality Is Still Free Philip Crosby wrote Quality is Free in 1979 and I falsely expected this book to be primarily an updated version of that book. If you haven't read the original, I would still recommend reading it before this one. It is more useful if you are trying to understand and apply Crosby's ideas to your organization. If you want to learn about Mr. Crosby's philosophy and career, this new book is the better choice. A few chapters are repeated, but most of the material is new. It gives you a good idea of what it was like for Mr. Crosby as he began trying to move organizations from a quality-control perspective to a quality-management perspective. This book puts a current perspective on his 14 points of quality and covers related concepts such as his Complete Transaction Rating (CTR). CTR provides a simple way of measuring the competence of employees and suppliers. CTR is given a value of 1 if the supplier does what he or she was supposed to do. (That means the supplier completely did what was agreed. This includes components like price, schedule, and quality.) A lot of the book is dedicated to his stories about his company Philip Crosby Associates, and how they try to use his principles. It makes it clear how those of us trying to provide consulting, training, or service can put his ideas into practice. Crosby, Philip B., 1996. Quality is Still Free New York: McGraw-Hill. ----Mary Sakry Top of newsletter © 1998 The Process Group