Quantifying Subjectivity
AUTHORS: Mark Knight and John Bourguignon
Decision makers rarely choose an alternative based upon a single factor or criterion. Taking the 'best’ course of action for important decisions entails, with very few exceptions, evaluating multiple objectives. The best decision makers always consider more than one objective for important decisions --even when those objectives may compete with each other. But how do you resolve these differences and how do you account for inevitable subconscious bias?
When presented with a choice, there are often objective and subjective aspects that need to be taken into account. Sometimes the inclusion of subjectivity can happen without even knowing that you are doing it. Projects and programs that encounter problems during implementation can usually trace their causes back to poor decisions early on. But when faced with competing tasks, all of which are important, how can you create relative levels of importance that reflect which ones to prioritize?
A criterion is said to be objective when it is based on facts, can be proved easily, and is difficult to deny. Objective refers to an unbiased and balanced statement that represents facts about something. Usually these are things that we can measure so this helps differentiate between options when making decisions.
A criterion is said to be subjective when there is an absence of facts, or when there is not a readily available measure to compare options (and opinions). Ranking of subjective criterion are often driven by ideas or statements dominated by personal feelings, opinion, or preferences, which may be influenced by subconscious bias.
So how can we combine decision-making that involves both objective and subjective criteria and have confidence in our resulting decisions? The answer lies in linear algebra but don’t stop reading here, this is not a math lecture.
Through the years we have learned to lean heavily on spreadsheets to formulate decisions due to their power, simplicity and accessibility no matter how big or complex the decision. Yet no matter how creative we may be, spreadsheet models are framed by expansive criteria normally from the ground up by the most well-intended team and supported by processes that are incapable of objectively modeling input. We are drawn to their convenience and simplicity, yet knowingly flawed results. The good news is that there is a better, proven and simplistic method of modeling decisions.
A process known as Analytic Hierarchy Process (AHP) uses pairwise comparisons of a knowledgeable person to determine the importance of criteria in a decision. It uses matrix algebra involving eigenvalues. In AHP, the inconsistency of judgments is measured by an index based on the principal eigenvalue of the positive reciprocal matrix of judgments. Say what?
What AHP provides is a way not only to prioritize criteria and give then quantifiable weightings but it also looks at the consistency of the comparisons and warns you if the way you have ranked the criteria based on a series or pairwise comparisons is not consistent. It is simple, quantifiable, and effective.
Say for example you were trying to determine where to focus your asset management efforts and your comparisons said that Lifecycle Delivery is more important to you than Risk & Review, and that Risk & Review is more important that Asset Information but you also determined that Asset Information is more important than Lifecycle Delivery? Clearly these comparisons are inconsistent, and it is easy to spot. As an example, it illustrates the point but is rather obviously flawed.
But what if you were looking at all 6 groups of the IAM conceptual model? Now you have 15 comparisons to make (each group compared with every other group) and with a graduated scale for determining relative importance there is no easy way to see how consistent your rankings are. This is where AHP is strong. Without getting into eigenvalues it uses proven math techniques to calculate a consistency index. If the index indicates the comparisons are inconsistent then it is necessary to take a look at the pairwise comparisons. It is also helpful to note why the comparison was scored the way it was in case you need to go back and revisit the scoring.
In the area of decision-making, the concept of priority is fundamental so how priorities are determined influences the choices made. AHP is a simple and effective tool to help with this process.
