Delphi method is commonly used when estimating activity durations or other types of quantitative decision making but it can also be effectively used for qualitative applications. One such use case is when you need to qualitatively assess a list of identified risks.
How did you run your last risk identification and assessment session?
As the group was identifying risks, what approach was taken to derive subjective impact or probability values for each risk? A common practice is that the person who identified the risk is also the one who determines its impact or probability. While they might be a subject matter expert, they are also likely to have risk biases which might influence their assessment. Once the risk identifier has provided the probability and impact information, other participants are unlikely to challenge the expertise of the individual.
The same could also happen if a collaborative approach in which risks which have been captured on Post-it notes are grouped onto flip charts representing each impact/probability combination. Once placed, inertia is likely to overcome the desire to make corrections unless an egregious assessment has been done.
Overcoming bias and inertia are two strengths of Delphi, so try this approach the next time you are facilitating a risk session.
- Give each risk session participant a set of cards with each impact or probability value written on an individual card. For example, if you are using the common very low to very high scale, you would give them five cards each.
- As each risk is assessed the person who identified the risk is asked to provide as much context as possible.
- Each team member simultaneously shows the card which best represents the risk’s impact.
- If everyone has selected the same impact value, record it. If there is roughly an equal distribution between two sequential ratings, ask the group to pick one. However, if there are one or two outliers among the ratings, have those individuals explain the rationale behind their rating. Then ask the group to repeat step 3 to re-evaluate the risk’s impact one more time and if there is still not a consensus, pick the value which best represents the group’s decision.
- Steps 3-4 can then be used to assess the probability of the identified risks.
While there is some risk of a strong influencer swaying others to change their evaluation (The Twelve Angry Men scenario), this approach should reduce the likelihood of that occurrence.
So the next time you need to perform qualitative risk assessment, consult the oracle!