Intelligence Analysis as Discovery of Evidence Hypotheses and Arguments Connecting the Dots 1st Edition by Gheorghe Tecuci, David Schum, Dorin Marcu, Mihai Boicu 1316655598 9781316655597

1 Intelligence Analysis: ‘‘Connecting the Dots’’

1.1 How Easy Is It to Connect the Dots?

1.1.1 How Many Kinds of Dots Are There?

1.1.2 Which Evidential Dots Can Be Believed?

1.1.3 Which Evidential Dots Should Be Considered?

1.1.4 Which Evidential Dots Should We Try to Connect?

1.1.5 How to Connect Evidential Dots to Hypotheses?

1.1.6 What Do Our Dot Connections Mean?

1.1.7 Whose Evidential Dots Should Be Connected?

1.2 Imaginative Reasoning in Intelligence Analysis

1.2.1 Imaginative Reasoning

1.2.2 What Ingredients of Analysis Are to Be Generated by Imaginative Thought?

1.2.3 Generating Main Hypotheses to Be Defended by Evidence and Argument

1.2.4 Generating the Evidential Grounds for Arguments

1.2.5 Generating Arguments Linking Evidence and Hypotheses

1.3 Intelligence Analysis as Discovery of Evidence, Hypotheses, and Arguments

1.3.1 Intelligence Analysis in the Framework of the Scientific Method

1.3.2 Evidence in Search of Hypotheses

1.3.3 Hypotheses in Search of Evidence

1.3.4 Evidentiary Testing of Hypotheses

1.3.5 Completing the Analysis

1.4 Review Questions

2 Marshaling Thoughts and Evidence for Imaginative Analysis

2.1 Sherlock Holmes and Investigation or Discovery

2.2 Mycroft Holmes and Evidence Marshaling

2.3 Marshaling ‘‘Magnets’’ or Attractors

2.4 Types of Marshaling Magnets

2.4.1 Believability Magnet

2.4.2 Chronology Magnet

2.4.3 Question Magnet

2.4.4 Hypothesis Magnet

2.4.5 Argument Magnet

2.4.6 Eliminative Magnet

2.4.7 Scenario Magnet

2.5 Use of the Marshaling Magnets

2.6 Review Questions

3 Disciple-CD: A Cognitive Assistant for Connecting the Dots

3.1 System Overview

3.2 Obtaining Disciple-CD

3.3 Hands On: Working with Knowledge Bases

3.3.1 Overview

3.3.2 Basic Operations

3.4 Knowledge Base Guidelines

Guideline 3.1. Work with only one knowledge base loaded in memory.

Guideline 3.2. Create a knowledge base and save successive versions.

3.5 Hands On: Browsing an Argumentation

3.5.1 Overview

3.5.2 Practice

3.5.3 Basic Operations

4 Evidence

4.1 What Is Evidence?

4.2 The Credentials of All Evidence

4.2.1 Relevance

4.2.2 Believability or Credibility

4.2.3 Force or Weight of Evidence

4.3 Assessing the Relevance, Believability, and Inferential Force of Evidence

4.4 Basic Operations with Disciple-CD

4.4.1 Hands On: Define and Evaluate Evidence

4.4.1.1 Overview

4.4.1.2 Practice

4.4.1.3 Basic Operations

4.5 Advanced Operations with Disciple-CD

4.5.1 Hands On: From Information to Evidence

4.5.1.1 Overview

4.5.1.2 Practice

4.5.1.3 Advanced Operations

4.6 Review Questions

5 Divide and Conquer: A Necessary Approach to Complex Analysis

5.1 Holistic Approach to Analysis

5.2 Divide and Conquer

5.3 Assessing Complex Hypotheses through Analysis and Synthesis

5.4 Inquiry-Driven Analysis and Synthesis

5.5 Types of Reductions and Corresponding Syntheses

5.5.1 Necessary and Sufficient Conditions

5.5.2 Sufficient Conditions and Scenarios

5.5.3 Indicators

5.6 Problems with Argument Construction

5.7 Basic Operations with Disciple-CD

5.7.1 Hands On: Was the Cesium Canister Stolen?

5.7.1.1 Hypothesis in Search of Evidence: Illustration

5.7.1.2 Hands-On Overview

5.7.1.3 Practice

5.7.1.4 Basic Operations

5.7.2 Hands On: Development and Evaluation of an Argument

5.7.2.1 Overview

5.7.2.2 Practice

5.7.2.3 Basic Operations

5.7.3 Hands On: Analysis Based on Previously Learned Patterns and Synthesis Functions

5.7.3.1 Overview

5.7.3.2 Practice

5.7.3.3 Basic Operations

5.8 Advanced Operations with Disciple-CD

5.8.1 Hands On: Abstraction of Analysis

5.8.1.1 Overview

5.8.1.2 Practice

5.8.1.3 Advanced Operations

5.8.2 Hands On: Hypothesis Analysis and Evidence Search

5.8.2.1 Overview

5.8.2.2 Practice

5.8.2.3 Advanced Operation

5.8.3 Hands On: Justifications of Assumptions

5.8.3.1 Overview

5.8.3.2 Practice

5.8.3.3 Advanced Operation

5.8.4 Hands On: Top-down and Bottom-up Argument Development

5.8.4.1 Overview

5.8.4.2 Practice

5.8.4.3 Advanced Operations

5.9 Analysis Guidelines

Guideline 5.1. Define analysis trees in natural language using simple questions

Guideline 5.2. Identify the specific instances, the generic instances, and the constants

Guideline 5.3. Learn and reuse reduction patterns

Guideline 5.4. Define short hypothesis names for the abstract reasoning tree

5.10 Review Questions

6 Assessing the Believability of Evidence

6.1 Believability: The Foundation of All Arguments from Evidence

6.2 Classification of Evidence Based on Believability

6.3 Tangible Evidence

6.3.1 Real Tangible Evidence: Authenticity

6.3.2 Demonstrative Tangible Evidence: Authenticity, Accuracy, and Reliability

6.3.3 Examples of Tangible Evidence

6.4 Testimonial Evidence

6.4.1 Competence

6.4.1.1 Access

6.4.1.2 Understandability

6.4.2 Credibility

6.4.2.1 Veracity or Truthfulness

6.4.2.2 Objectivity

6.4.2.3 Observational Sensitivity

6.4.3 Types of Testimonial Evidence

6.4.4 Examples of Testimonial Evidence

6.5 Missing Evidence

6.5.1 Uncertainties Associated with Missing Evidence

6.5.2 Example of Missing Evidence

6.6 Authoritative Records

6.7 Mixed Evidence

6.7.1 Analysis of Mixed Evidence

6.7.2 Examples of Mixed Evidence

6.8 Deep Believability Analysis

6.9 Advanced Operations with Disciple-CD

6.9.1 Hands On: Believability Analysis

6.9.1.1 Overview

6.9.1.2 Practice

6.9.1.3 Advanced Operation

6.10 Review Questions

7 Chains of Custody

7.1 What Is a Chain of Custody?

7.2 A Case Involving Chains of Custody

7.3 A Chain of Custody for Testimonial Evidence

7.4 A Chain of Custody for Demonstrative Tangible Evidence

7.4.1 Chain of Custody for a Photo Given Directly to the Analyst

7.4.2 Chain of Custody for a Written Description of a Photo Given to the Analyst

7.5 Analyzing a Chain of Custody

7.6 Drill-Down Analysis of Chains of Custody

7.7 Review Questions

8 Recurrent Substance-Blind Combinations of Evidence

8.1 Harmonious Evidence

8.1.1 Basic Forms of Harmonious Evidence

8.1.2 Patterns of Evidential Harmony

8.2 Dissonant Evidence

8.2.1 Basic Forms of Dissonant Evidence

8.2.2 Patterns of Evidential Dissonance

8.3 Redundant Evidence

8.3.1 Basic Forms of Redundant Evidence

8.3.2 Patterns of Evidential Redundance

8.4 Why Considering Evidence Combinations Is Important

8.5 Basic Operations with Disciple-CD

8.5.1 Hands On: Who Has Stolen the Cesium Canister?

8.5.1.1 Overview

8.5.1.2 Practice

8.6 Review Questions

9 Major Sources of Uncertainty in Masses of Evidence

9.1 Incompleteness

9.1.1 What Is Incompleteness of Evidence?

9.1.2 Examples of Incompleteness

9.2 Inconclusiveness

9.2.1 What Is Inconclusiveness of Evidence?

9.2.2 Examples of Inconclusiveness

9.3 Ambiguity

9.3.1 What Is Ambiguity of Evidence?

9.3.2 Examples of Ambiguity

9.4 Dissonance

9.4.1 What Is the Dissonance of Evidence?

9.4.2 Examples of Dissonance

9.5 Imperfect Believability

9.5.1 What Is Imperfect Believability of Evidence?

9.5.2 Examples of Imperfect Believability

9.6 Basic Operations with Disciple-CD

9.6.1 Hands On: Does a Terrorist Organization Have the Cesium Canister?

9.6.1.1 Overview

9.6.1.2 Extracting Evidence from Information

9.6.1.3 Practice

9.7 Review Questions

10 Assessing and Reporting Uncertainty: Some Alternative Methods

10.1 Introduction

10.2 General Classes of Probability and Uncertainty

10.3 Enumerative Probabilities: Obtained by Counting

10.4 Nonenumerative Probabilities: Nothing to Count

10.5 Epistemic Probability (1): The Subjective Bayesian View

10.5.1 Likelihood Ratios

10.5.1.1 Analysis Using Likelihood Ratios

10.5.1.2 Examples

10.5.2 Bayesian Networks

10.5.2.1 Constructing the Argument Structure

10.5.2.2 Forming the Key List

10.5.2.3 Identifying the Likelihoods and Prior Probabilities

10.5.2.4 Using the Bayesian Network

10.5.2.5 Utility and Feasibility of Bayesian Network Analyses

10.6 Epistemic Probability (2): Belief Functions

10.6.1 Belief Functions and Evidential Support

10.6.2 Examples of Assigning Evidential Support

10.6.3 Dempster’s Rule for Combining Partial Beliefs

10.7 Baconian Probability and the Importance of Evidential Completeness

10.7.1 Variative and Eliminative Inferences

10.7.2 Importance of Evidential Completeness

10.7.3 Baconian Probability of Boolean Expressions

10.8 Imprecision and Fuzzy Probability

10.8.1 Fuzzy Force of Evidence

10.8.2 Fuzzy Probability of Boolean Expressions

10.8.3 On Verbal Assessments of Probabilities

10.9 A Summary of Uncertainty Methods and What They Best Capture

10.10 Basic Operations with Disciple-CD

10.10.1 Hands On: Will a Bomb Be Set Off in Washington, D.C.?

10.10.1.1 Overview

10.10.1.2 Practice

10.11 Review Questions

11 Analytic Bias

11.1 Basic Interpretations of the Term Bias

11.2 Biases of the Analyst

11.2.1 Biases in the Evaluation of Evidence

11.2.2 Biases in the Perception of Cause and Effect

11.2.3 Biases in Estimating Probabilities

11.2.4 Hindsight Biases in Evaluating Intelligence Reporting

11.3 Some Frequently Overlooked Origins of Bias

11.3.1 HUMINT Sources

11.3.2 Persons in Chains of Custody of Evidence

11.3.3 Consumers of Intelligence Analyses

11.4 Biases and the Evaluation of Analysts

11.5 Recognizing and Countering Biases with Disciple-CD

11.6 Review Questions

12 Learning and Reusing Analytic Expertise: Beyond Disciple-CD

12.1 Introduction

12.2 Learning Agent Shell

12.3 Learning Agent Shell for Evidence-based Reasoning

12.3.1 Disciple-EBR

12.3.2 Disciple-CD

12.4 Development of a Cognitive Assistant

12.5 Evidence-Based Reasoning Everywhere

Glossary of Terms

References

Appendixes

1 Methodological Guidelines

2 Hands-On Exercises

3 Operations with Disciple-CD

Index