This is part of a series providing insights aimed at corporate strategists seeking competitive advantage through better and more accurate decision-making. The full series is available at our special section on Decision Intelligence.  Members are also invited to discuss this topic at the OODA Member Forum on Slack.

This post discusses standards in intelligence, a topic that can improve the quality of all corporate intelligence efforts and do so while reducing ambiguity in the information used to drive decisions and enhancing the ability of corporations to defend their most critical information.

The Importance of Standards in Analysis

Standards in corporate intelligence programs are critical to quality. Standards lead to professionalism of product, higher accuracy and more valuable product that will drive decisions.

But standards must be appropriately applied. Not all standards are relevant to all efforts. As you will see below, many standards that work in the US Intelligence Community are totally wrong for use in corporate America, and vice versa. The wrong standards can result in a waste of time and perhaps even a stifling of analysts that would be counterproductive. In the worse case, standards applied wrong could lead to the wrong conclusions and decisions that threaten corporate success.

This leads to rule number one on standards in intelligence: Do no harm!

Expectations From an Analytics Standards Program

Analytic standards apply to how information is sourced and scrutinized, some parts of how analysis is conducted, and how conclusions are reported. Here is our recommendations for goals for corporate intelligence analysis product standards:

• The standards put in place in your organization should be of high enough fidelity to be used in training for the workforce and for stating requirements for external providers.
• Standards should be clear and easy to capture in corporate policy and to teach to the workforce.
• Standards in analysis should only be applied where experience shows they can add value and are needed, and even then should be applied with insights from experience.

With these goals in mind we present our list of analytic standards below.

The Core Corporate Analytic Standards

The list of standards below is based largely on standards in place in the US Intelligence Community, which has published the results of years of research into what works and what does not in a community-wide directive (ICD 203). Additional sources of the standards below are include the extensive experience of the OODA team in the needs of corporate analytic efforts.

Objective and Independent: Corporate intelligence processes should deliver product that is free from harmful bias or unwarranted guidance from anyone. Analysts should never be pressured to change assessments to serve an outcome that reality does not indicate is true. Analysis should be based on facts, judgements, or when, required, even assumptions or intuition. All this should be captured and qualified, but never should they be shifted based on something a boss or other wants changed. But here is a big difference in how this standard should be applied in government and in the corporate world: In government this independence is taken to such an extreme that intelligence analysts becomes too detached from reality and not coupled enough to decision-making. Intelligence is so detached from decisions that they usually do not provide any sort of net-assessment or input on the many things decision-makers might want to consider to shape the future. In government, intelligence usually informs policy, and almost never makes it. In corporate America, the best intelligence efforts drive strategy and does not shy away from making suggestions on the right course of action.

Timely: Analytic products must be produced, disseminated and made available in time to impact decisions. The perfect product delivered after a decision is made is worthless.

Based on all relevant sources: Analysts charged with examining a topic need to be empowered to use any data and information they need to drive their conclusion.

Provides insight into credibility of sources and data: This can include describing factors like the reputation of the provider of data.

Provides insight into methodologies used: The methodologies used to produce analysis should be succinctly expressed in the work product so consumers can use this as they evaluate their own belief on the quality of conclusions.

Express probabilities and uncertainties in standard ways: When analytical products must express likelihood or odds of an event or uncertainty it should be done in as repeatable way as possible. Your organization may want to tailor how these are done, but as a starting point, consider the way the intelligence community expects conclusions be discussed, as shown in the table below:

 

 

 

Always Distinguish Between Facts, Assumptions and Judgements: Analytic products need to distinguish between these so readers of the product are not misled. Assumptions are defined as suppositions used to frame or support an argument. Judgements are defined as conclusions based on underlying information, analysis and assumptions.

Incorporate Analysis of Alternatives: the analysis of alternatives is the systemic evaluation of differing hypothesis to explain events, explore near term outcomes and imagine possible futures to mitigate surprise and reduce risk. Analytic processes should identify and assess plausible alternative hypothesis. This is especially important when major judgements must deal with significant uncertainties or complexities or when the event being discussed could have a high impact.

Identify Indicators: When producing products which give assessments of future events or potential decisions, analysts should strive to provide insight into what might be seen that might indicate the event in question is coming to pass.

Demonstrate Customer Relevance and Address Implications: Whenever possible, analytic product should get to the “so what” of the analysis, in a language relevant to the decision-maker leveraging the analysis. Analysts should also, to the greatest extent possible, express “what’s next” for the situation, since this is almost always of relevance to customers.

Use clear and logical arguments: To the greatest extent possible, analytic products should present the main conclusions up front, but then provide the logic and facts and assumption supporting those judgements in a clear and logical way so consumers can dive deeper into an examination of why the judgement was made.

Explain change to or consistency of analytic judgements: When new data, new judgements or changing situations cause a change to analytic judgements they should be highlighted.

 

Concluding Thoughts

Analysis depends on the power of the human brain. The most important analytical tasks tap into mysterious unbounded processes like creativity and imagination and intuition. The importance of these impossible to regulate creative processes to analytic output means the approach to standards must be carefully done. The first step in doing this is to leverage the lessons learned from the intelligence community and interactions in corporate settings captured above.

 

Additional Resources:

Decision Intelligence and Establishing an Intelligent Enterprise: The greatest determinant of your success will be the quality of your decisions. We examine frameworks for understanding and reducing risk while enabling opportunities. Topics include Black Swans, Gray Rhinos, Foresight, Strategy, Stratigames, Business Intelligence and Intelligent Enterprises. Leadership in the modern age is also a key topic in this domain.