The Inevitability of Intelligence Failure: Sources, Locations and Causes

November 2013

Intelligence failure can be defined as ‘a misunderstanding of the situation that leads a government (or its military forces) to take actions that are inappropriate and counterproductive to its own interests’ (Schmitt & Shulsky, 2002: 62). Explorations of what could or should have been done differently form the basis for millions of pages of government investigations, newspaper articles, books, journals and entire academic careers.

A uniting factor which leads an event to be considered an intelligence failure is that it was or came close to a national disaster, whereas ‘the record of success is less striking because observers tend not to notice disasters that do not happen’ (Betts, 1978: 62). If one takes this definition of intelligence failure and turns it around, an ‘intelligence success’ occurs when a situation is understood properly and leads a government (or its military forces) to take actions appropriate and productive to its own interests. Concern with intelligence failures always far outstrips that of any intelligence successes, which may not become known until long after the event, if ever.

Intelligence failures are inevitable because there are limits to what intelligence can accomplish. Its proponents are often guilty of overselling its capabilities and its ‘consumers’ and observers are guilty of misunderstanding them (Gill & Phythian, 2006: 104-5). This essay will use a framework based upon the work of Betts, Schmitt and Shulsky, and Gill and Phythian, among others, to outline the different ‘sources’ or ‘locations’ of intelligence failure, where they occur in the ‘intelligence cycle’, their ‘causes’ and other factors, followed by the use of the 2003 Iraq WMD controversy as a case study to briefly illustrate real-world examples to show why intelligence failures are inevitable.

Sources & Causes
There are three general locations which can be identified as sources of intelligence failure: ‘Collection’, ‘Analysis’ and ‘Decision-Makers’ (Gill & Phythian, 2006: 103-4, citing Betts, 1978). These categories broadly separate persons involved in the ‘Intelligence Cycle’ by their positions or functional roles within it. CIA defines the Intelligence Cycle as the ‘process by which information is acquired, converted into intelligence, and made available to policymakers’ (Johnson & Wirtz, 2008: 49).
Collection

The first step in the Intelligence Cycle is to develop directions as to what information is necessary, known as ‘requirements’, and how it will be collected based upon the needs of the eventual consumers of the intelligence, usually senior policymakers (ibid.). Based upon this guidance, intelligence agencies then begin Collection, also known as ‘espionage’, which can be defined as, ‘the practice of using spies to collect information about what another government or company is doing or plans to do’ (Williams, 2011: 1165). Put simply, Collection involves obtaining the information or ‘raw intelligence’ which is required to meet Requirements developed at the first step in the Intelligence Cycle. Collection is performed by all means available to intelligence agencies, including human intelligence, signals intercepts, imagery and scientific and technical measurement, among others, and relates to one or more ‘types’ of intelligence, including military, political, economic or cultural intelligence (Johnson & Wirtz, 2008: 51).

Collection Failure
Collection Failure can be seen as the ‘unavailability of information when and where needed’ (Hatlebrekke & Smith, 2010: 151, citing Schmitt & Shulsky, 2002: 64-7;) or part of Betts’ ‘Pathologies of Communication’–the lack of timely collection of information (Betts, 1978: 62-3). Essentially, Collection Failure occurs because the information necessary to respond successfully to the situation was not collected or not when needed.

Inevitability?
Planning and directing intelligence requirements and Collection must still deal with the fact that, in the famous words of U.S. Defence Secretary Donald Rumsfeld, ‘There are no "knowns." There are thing we know that we know. There are known unknowns. That is to say there are things that we now know we don't know. But there are also unknown unknowns. There are things we don't know we don't know’ (North Atlantic Treaty Organisation, 2002), a sentiment similar to one expressed earlier by CIA’s Sherman Kent when he stated there are ‘Things which are knowable but happen to be unknown to us, and…things which are not known to anyone at all’ (Kent, 1964). Simply put, Collection Failure can occur when necessary information, known to exist and known to be needed, is not available due to some limitation. Or it can occur because it was not even considered, dismissed or there were no indicators it was necessary to gather such information. Nonetheless, as Betts (1978: 61) states, ‘In the best-known cases of intelligence failure, the most crucial mistakes have seldom been made by collectors of raw information’.

Analysis
Station three in the Intelligence Cycle is the ‘processing’ of the collected information, followed by the fourth step of Analysis and creation of intelligence ‘products’, such as reports, findings and estimates. This is followed by the fifth step, Dissemination—distributing the intelligence product to decision-makers in a proper format in a timely manner (Johnson & Wirtz, 2008: 49).

The analysis stage begins with an evaluation of the truth of the collected material and the validity of the process which acquired it (Butler Panel, 2008: 527-8). For some forms, such as imagery and signals intelligence, this will be easy; for others, especially human intelligence (usually obtained second-hand from an informant by a case officer) it is more onerous to establish. There may be the wish of the collecting officer to have his source believed, giving rise to the need for an independent appraisal of the information (ibid.). Accuracy in reporting or quoting the information from the source to the analyst must be ensured. It must be checked that the source has actual access to the information or an acceptable explanation of how it was obtained. It must be considered if the source has some ulterior motive for providing the information or if they are involved in a counter-intelligence operation. Their previous track record must also be considered (ibid.). If the veracity of the information is not thoroughly tested and established, anything that follows may be derived from false information and will be fruit from a poison tree.

Actual Analysis of the information, after its validity has been established, involves appraising the value of the information in its own right, deciding how much weight should be assigned to it and compiling it into ‘meaningful strands’ based upon what it relates to. These ‘strands’, compiled from all available sources of related information, are then further used to develop ‘estimates’ (‘assessments’ in the UK) of a particular international situation or set of circumstances (ibid.: 528). They can be rapid, low-level appraisals or they can be the premier product composed of the collective wisdom of the entire intelligence community, as with America’s National Intelligence Estimates (Johnson, 2008: 344). The product is then disseminated and/or briefed to decision-makers in an appropriate format.

Collecting information is difficult enough; deciding what should be done in light of it and how it should be presented creates even more problems. Sherman Kent, ‘father figure of CIA analysis’, felt that estimates consist of, ‘knowledge, reasoning and guesswork’ (Johnson, 2008: 344). The pitfalls of analysing information to attempt to piece together accurate appraisals, predictions and/or advice based upon information of varying types and of contestable accuracy or value, often in a short amount of time, in order to develop an intelligence product are apparent. Estimates are used to develop national policy regarding situations vital to national security. If information, analysis or estimates are wrong or wrongly presented, the policy response will be wrong as well, with serious consequences (Gill & Phythian, 2006: 106-112).

Analysis Failure
Analysis Failure is a broader category. It includes ‘Tendency to concentrate on ‘usual suspects’ for ideological or practical purposes’ (Gill & Phythian, 2006: 104), ‘Opinion governed by “conventional wisdom” without supporting evidence’, ‘”mirror imaging”, in which unfamiliar situations are judged upon the basis of the familiar’ (Hatlebrekke & Smith, 2010: 151, citing Schmitt & Shulsky, 2002, 64-67) and the ‘Paradox of Perception’—failure to properly balance pre-conceived notions based upon previous and historical experience against an unbiased look at information and failure to balance sensitivity of warnings between insufficiency and alarmism (Betts, 1978: 62-3). It also includes failure in ‘effectively communicating with Decision-Makers’, Betts’ other entry under ‘Pathologies of Communication’ (ibid.).

In sum, Analysis Failure can be located in the Processing, Analysis and Production, or Dissemination stages of the Intelligence Cycle. Essentially, Analysis Failure occurs because information is not properly validated or it is improperly dismissed; the wrong degree of emphasis is placed on collected information; opinion developed through collected information is skewed by practical, ideological, or some other cognitive bias, or; where intelligence products do not effectively communicate the necessary information collected to Decision-Makers, either through an ineffective portrayal of the information collected or untimely dissemination to them.

Inevitability?
Writing in defence of the wrongly-concluded 1962 NIE on the likelihood of Soviet nuclear missiles being stationed on Cuba, Sherman Kent (1964) explained the process of estimates thus:

“If NIEs could be confined to statements of indisputable fact the task would be safe and easy. Of course the result could not then be called an estimate. By definition, estimating is an excursion out beyond established fact into the unknown--a venture in which the estimator gets such aid and comfort as he can from analogy, extrapolation, logic, and judgment. In the nature of things he will upon occasion end up with a conclusion which time will prove to be wrong. To recognize this as inevitable does not mean that we estimators are reconciled to our inadequacy; it only means we fully realize that we are engaged in a hazardous occupation.”

If the circumstances for which Decision-Makers require intelligence were unambiguous, then intelligence services could hand them Kent’s ‘indisputable facts’ and leave them to it. However, there are few circumstances in international politics today which possess such clarity that Decision-Makers, often elected officials with little or no security experience, can decide on their own with ‘raw intelligence’. Today the problem may be too much information, as opposed to not enough, when one considers the mass amounts of imagery and signals information collected, processed and analysed (Irwin, 2012). Intelligence analysis remains a necessity.

Schmitt and Schulsky (2002: 72) argue, ’The heart of the problem of intelligence failure, [is] the thought processes of the individual analyst.’ Attempts to counteract the human element in analysis through systemic or procedural reforms, rather than fixing these flaws, may actually serve to build overconfidence in them afterwards through a belief that the problem has been solved and won’t recur (Betts, 1978: 61). However, history shows that the same mistakes continue to be made. At bottom, intelligence analysis is still a flawed human process and will always be as flawed as the human beings conducting the analysis. So long as there are ambiguous situations requiring, as Kent (1964) puts it, ‘guesswork’, there will inevitably be Analysis Failures.
Decision-Makers

In the final step of the Intelligence Cycle, the product, having been disseminated, has been received by its ‘consumers’ and they have informed themselves of the intelligence. They may have further questions or desire more information. This leads to the development of new Requirements, leading back to the first stage of the Intelligence Cycle, which begins anew (Johnson & Wirtz, 2008: 49).

Decision-Makers, a mix of high-level elected officials, executive appointees and military officials, then actually use the intelligence produced to inform their decisions. Making decisions that are vital to protecting a nation’s security are inherently difficult, with or without accurate intelligence. Historically, intelligence failures are most frequently located with Decision-Makers (Betts, 1978: 62-3). Policymakers often make one of Johnson’s ‘seven sins of strategic intelligence’—ignoring intelligence that which does not conform to their view of a situation (Johnson, 1982: 182-4). If intelligence is misunderstood or ignored, the policy decision-makers pursue may lead to an intelligence failure.

Decision-Maker Failure
‘Decision-Maker Failure’ is the ‘subordination of Intelligence to policy’ (Schmitt & Shulsky, 2002: 64-67; Hatlebrekke & Smith, 2010: 151) and may influence the initial Planning and Direction stage of the Intelligence Cycle or be exhibited in how or if Decision-Makers use intelligence to make policy. It also occurs when pressure from Decision-Makers is brought to bear on those involved in analysis at the Processing, Analysis and Production or Dissemination stages. According to Betts (1978: 61), ‘In the best-known cases of intelligence failure, the most crucial mistakes have seldom been made by collectors of raw information, occasionally by professionals who produce finished analyses, but most often by the decision-makers who consume the products of intelligence services.’

Decision-Maker Failure often occurs as a result of ‘Politicisation’, one of the central problems of intelligence. In the words of Robert M Gates as Director of Central Intelligence (Gates, 1992):

“Politicization can manifest itself in many ways, but in each case it boils down to the same essential elements: almost all agree that it involves deliberately distorting analysis or judgments to favor a preferred line of thinking irrespective of evidence. Most consider `classic' politicization to be only that which occurs if products are forced to conform to policymakers' views. A number believe politicization also results from management pressures to define and drive certain lines of analysis and substantive viewpoints. Still others believe that changes in tone or emphasis made during the normal review or coordination process and limited means for expressing alternative viewpoints, also constitute forms of politicization."

Essentially, Decision-Maker Failure most often occurs because of some form of ‘politicisation’ by Decision-Makers where intelligence is deliberately distorted, ignored or selectively applied.

Inevitability?
According to Johnson (1983: 182), ‘No shortcoming of strategic intelligence is more often cited than the self-delusion of policymakers who brush aside-or bend-facts that fail to conform with their Weltanschauung.’ In Western democracies, Decision-Makers are either elected officials or senior civil servants influenced by elected officials. Politics is their job and it affects all aspects of it, including making security decisions based upon intelligence. Politics affects every form of government, from totalitarian dictatorships to communist collectives. As von Clausewitz is often quoted, ‘War is the continuation of politics by other means.’ Attempting to totally remove politics from war and national security is an impossible task. As stated above, Decision-Maker Failure through politicisation is the most common reason for intelligence failure. So long as politics are involved in security decisions, there will inevitably be intelligence failures.


Other Factors

Systemic Factors
Systemic Factors which may lead to intelligence failure include internal bureaucratic obstacles and failure to share information or cooperate with other agencies (Gill & Phythian, 2006: 104). Garciano and Posner (2005: 159) describe this as, ‘lack of prompt and full sharing of intelligence information within intelligence agencies, between different agencies, and between federal, state and local government levels.’ These Systemic Factors may occur at any phase of the Intelligence cycle and lie within Collection, Analysis or with Decision-Makers. If Collection resources or collected information is not shared it cannot be properly analysed and will not make its way to Decision-Makers, leading to intelligence failure.

External Factors
‘External Factors’ are always present. They include the, ‘Intrinsic difficulty in identifying targets’ and the fact that states or organisations which are or may be the target of intelligence agencies cannot be expected to remain passive to intelligence operations against them (Gill & Phythian, 2006: 104-5). Simply put, sometimes it is difficult or impossible to collect some forms of information, or, as Kent (1964) puts it, ‘Something literally unknowable by any man alive.’ Knowing the intent of an adversary before even he has formed it is an example of this impossibility. An intelligence failure for one state is often the result of an intelligence success by another. External Factors most often affect Collection, but they may also affect Analysis and Decision-Makers at any stage of the Intelligence Cycle, especially if the particular intelligence failure is the active effort of a foreign intelligence service, such as infiltration, double-agents, strategic deception or a counter-intelligence operation.

Case Study: Iraq WMD
The false belief and assertion by the U.S. and UK governments that Saddam Hussein’s Iraq possessed WMD capabilities in the lead up to the 2003 Iraq War is an instructive case which illustrates examples of each of the different causes of intelligence failure. Some of these are explored here, though there are more which could be cited.

Collection Failure
According to Morrison (2011: 520), one of the reasons for the intelligence failure relating to the existence of Iraq’s WMD capabilities was a failure to set a Requirement for Collection of political intelligence to place in context technical information on Saddam’s WMD programs. Collection was instead focused on gathering information about the existence of the programs themselves. This led to failure to consider the question in light of Saddam’s ‘political system, fears, and intentions’ (Jervis, 2006: 41). Collecting such information could have offered an explanation as to why Hussein would refuse to disavow WMD programs and cooperate with UN inspections despite not having active WMD programs. Setting this Requirement may have served to balance the dominant presumption that Iraq had WMD, the justification for the war which turned out to be false.

Analysis Failure
UK and U.S. analysis of Iraq’s WMD relied heavily upon unreliable human intelligence sources which were not properly vetted and, in some cases, have since proven to be fabrications. Information was also being quickly disseminated to policymakers without being properly validated or analysed first (Morrison, 2011: 520). In his study of the major U.S. and UK post-mortems on Iraq, Jervis (2006) cites many examples of Analysis Failure: ‘ICs’ judgments were stated with excessive certainty’ (ibid.: 14), ‘no general alternative explanations for Saddam’s behavior were offered’ (ibid.: 15), ‘lack of imagination’ to develop these alternative views (ibid.: 17), ‘failure to challenge assumptions’ regarding existence of WMD programs and a situation ‘whereby assessments were based on previous judgments without carrying forward the uncertainties’ (ibid.: 22), among others. These examples illustrate just some of the places where analysis went wrong.

Decision-Maker Failure
The case of Iraq’s WMD provides a clear example of politicisation of intelligence. The Pentagon’s Office of Special Plans (OSP) was specifically established by the Bush administration to develop assessments based upon the assumption that Iraq had WMD and its products were used in preference to other Intelligence Community assessments that contradicted that hypothesis (Ryan, 2006: 304-5). A ‘Red Team’ analysis conducted by CIA’s WINPAC, designed only as an initial ‘devil’s advocate’ view in the face of evidence Iraq did not have WMD, was also selectively used by the administration despite strong evidence from the rest of the Intelligence Community its conclusion was wrong. Assessments by agencies such as U.S. Department of Energy and the State Department’s Bureau of Intelligence Research provided strong, clear arguments against the central argument made in assessments preferred by the Bush administration, but they were pushed aside by Decision-Makers (Conway, 2012: 490-1).

Systemic Factors
Garciano and Posner (2005: 149) point out the WMD Commission’s finding that there was not enough cooperation or information sharing between agencies, specifically in regard to reports that called into question the credibility of the information by the source ‘Curveball’, whose assertion that he had been involved in active WMD programs in Iraq became a central piece of evidence in the Bush administration’s push for war. Curveball (Rashid al-Janabi) himself has since admitted his evidence was a fabrication (NBC News, 2011). If more than one agency had access to these reports, there may have been more questions asked. However, bureaucratic obstacles and security standards stood in the way. There is a lack of information sharing between U.S. intelligence and law enforcement agencies, seen to have separate functions, which may have led to vital information not being shared and considered (Cilluffo et al, 2002: 70-1).

External Factors
Saddam Hussein and his regime were not inactive players in their fate. There was a systematic denial and deception campaign by the regime and their failure to cooperate with and eventual ejection of UN inspectors supported the belief that Iraq had something to hide regarding its WMD programs. Wherever evidence was not available to prove or disprove hypotheses, Saddam and his regime could be blamed for blocking attempts to collect the necessary intelligence (Jervis, 2006: 27-8). In his debriefing by the FBI, Saddam admitted that he wanted to maintain the façade of possessing WMD to counter enemies, especially Iran (Federal Bureau of Investigation, 2008). Of course, had Saddam been open regarding WMD and cooperated with UN inspections, the likelihood of war would have been greatly reduced. Intelligence is, ‘a game between hiders and finders, and the former usually have the easier job’ (Jervis, 2006: 11).

Conclusion
As Gill and Phythian (2004: 105) point out, ‘the limits of intelligence dictate that intelligence failure is inevitable. Partly, this is a consequence of the impossibility of perfect predictive success; partly it is a consequence of decision-makers’ (politicians with regard to states) natural tendency to err on the side of caution by subscribing to worst-case scenarios, or to simply ignore intelligence that does not fit their own preferences.’ As discussed, there are other locations within the Intelligence Cycle where things can go wrong and there are other possible causes and factors leading to intelligence failure. The debacle of Iraq’s WMD provides many real-world illustrations.

Jervis (2006: 11) makes clear that, ‘Any specific instance of intelligence failure will, by definition, seem unusual, but the fact of the failure itself is quite ordinary’. If the result of the failure is a great national disaster, it will seem to be a great failure. However, factual examinations of how failures occur inevitably show common, ordinary failures by people and organisations that often occur in situations outside of the intelligence context. Jervis usefully compares the Iraq WMD investigations to enquiries into the 1986 Challenger shuttle disaster and abuse allegations in the Catholic Church (ibid.: 4-5). It is only the consequences of intelligence failure that make them appear much larger than they are—the stakes are simply much higher (Betts, 1978: 62). Despite all of the post-mortems conducted on intelligence failures, there is little evidence their conclusions or the reforms enacted afterwards as a result have translated into an elimination or even significant reduction of failures (ibid.).
These human failures are impossible to ever eliminate for good, especially those of a political or cognitive nature. Intelligence failures are inevitable because they seek to make clear ambiguous, complex situations using information that may be difficult to obtain, validate and understand, either through the resistance of the target or the nature of the information itself, and then present information in a digestible manner to Decision-Makers, who may do as they wish with the intelligence produced. Even minor errors at any point in the Intelligence Cycle may throw off the entire enterprise. The fallible and flawed nature of human beings attempting to put together some unknown puzzle with an incomplete picture means mistakes will inevitably occur at some point in the process. There is an old military maxim: ‘what can go wrong, will go wrong.’ Betts (1978) and Gill and Phythian (2006: 105) believe we must accept that intelligence failures are inevitable. Indeed, given the nature of the work of intelligence and all that can go wrong, it would be more extraordinary if intelligence failures did not occur.





BIBLIOGRAPHY
Barrett, D. (2010) “Why Intelligence Failures Are (Still) Inevitable.” Diplomatic History 34 (1): pp. 207–213.

Betts, R. (1978) “Analysis, War, and Decision: Why Intelligence Failures Are Inevitable.” World Politics 31 (1): pp. 61–89.

———. (2002) “Fixing Intelligence.” Foreign Affairs 81 (1): pp.43–59.

———. (2007) “Two Faces of Intelligence Failure: September 11 and Iraq’s Missing WMD.” Political Science Quarterly 122 (4): pp. 585–606.

Butler Panel of Inquiry (2008) “The British Experience with Intelligence Failure.” In Intelligence and National Security: The Secret World of Spies, edited by Johnson, L. and Wirtz, J., 2nd ed., pp. 526–536. Oxford: Oxford University Press.

Cilluffo, F., Marks, R., and Salmoiraghi, G. (2002) “The Use and Limits of U.S. Intelligence.” Washington Quarterly 25 (1): pp. 61–74.

Conway, P. (2012) “Red Team: How the Neoconservatives Helped Cause the Iraq Intelligence Failure.” Intelligence and National Security 27 (4): pp. 488–512.

Federal Bureau of Investigation (2008) “Interviewing Saddam: FBI Agent Gets to the Truth.” Federal Bureau of Investigation – Stories. http://www.fbi.gov/news/stories/2008/january/piro012808 [Accessed 4 January 2014].

Garciano, L., and Posner, R. (2005) “Intelligence Failures: An Organizational Economics Perspective.” The Journal of Economic Perspectives 19 (4). http://www.jstor.org/stable/4134960 [Accessed 4 January 2014].

Gates, R. (1992) “Guarding Against Politicization.” Central Intelligence Agency. https://www.cia.gov/library/center-for-the-study-of-intelligence/kent-csi/volume-36-number-1/html/v36i1a01p_0001.htm [Accessed 5 January 2014].

Gill, P., and Phythian, M. (2006) “Why Does Intelligence Fail?” In Intelligence in an Insecure World, 1st ed., pp. 103–124. Cambridge: Polity.

Hatlebrekke, K., and Smith, M. (2010) “Towards a New Theory of Intelligence Failure? The Impact of Cognitive Closure and Discourse Failure.” Intelligence and National Security, 25 (2): pp. 147–182.

Irwin, S. (2012) “Too Much Information, Not Enough Intelligence.” National Defense Magazine. http://www.nationaldefensemagazine.org/archive/2012/May/Pages/TooMuchInformation,NotEnoughIntelligence.aspx [Accessed 6 January 2014].

Jackson, P., and Scott, L. (2004) “The Study of Intelligence in Theory and Practice.” Intelligence and National Security 19 (2): pp. 139–169.

Jervis, R. (2006) “Reports, Politics, and Intelligence Failures: The Case of Iraq.” Journal of Strategic Studies 29 (1): pp. 3–52.

Johnson, L. (1983) “Seven Sins of Strategic Intelligence.” World Affairs 146 (2): pp. 176–204.

———. (2008) “Glimpses into the Gems of American Intelligence: The President’s Daily Brief and the National Intelligence Estimate.” Intelligence and National Security 23 (3): pp. 333–370.

Johnson, L., and Wirtz, J. (2008) “Part II: Collection.” In Intelligence and National Security, 2nd ed., pp. 49–55. Oxford: Oxford University Press.

Kent, S. (1964) “A Crucial Estimate Relived.” Studies in Intelligence (Central Intelligence Agency). https://www.cia.gov/library/center-for-the-study-of-intelligence/kent-csi/vol8no2/html/v08i2a01p.htm [Accessed 4 January 2014].

Morrison, J. (2011) “British Intelligence Failures in Iraq.” Intelligence and National Security 26 (4). http://dx.doi.org/10.1080/02684527.2011.580604 [Accessed 6 January 2014].

NBC News (2011) “‘Curveball’: I Lied about WMD to Hasten Iraq War.” NBC News. http://www.nbcnews.com/id/41609536/ns/world_news-mideast_n_africa/t/curveball-i-lied-about-wmd-hasten-iraq-war/#.Us12Fp5_u8A [Accessed 5 January 2014].

North Atlantic Treaty Organisation (2002) “Press Conference by US Secretary of Defence, Donald Rumsfeld.” NATO HQ. http://www.nato.int/docu/speech/2002/s020606g.htm [Accessed 5 January 2014].

Renshon, J. (2009) “Assessing Capabilities in International Politics: Biased Overestimation and the Case of the Imaginary ‘Missile Gap.’” Journal of Strategic Studies 32 (1). http://dx.doi.org/10.1080/01402390802407475 [Accessed 5 January 2014].

Roman, P. (1995) “Strategic Bombers over the Missile Horizon.” Journal of Strategic Studies 18 (1): pp. 198–236.

Ryan, M. (2006) “Filling in the ‘Unknowns’: Hypothesis-Based Intelligence and the Rumsfeld Commission.” Intelligence and National Security 21 (2): pp. 286–315.

Schmitt, G., and Shulsky, A. (2002) Silent Warfare: Understanding the World of Intelligence. Washington, D.C.: Brassey’s.

Williams, R. (2011) “(Spy) Game Change: Cyber Networks, Intelligence Collection, and Covert Action.” George Washington Law Review 79 (4): pp. 1162–1200.

Contracting Out U.S. National Security

This article originally appeared in The Guardian on 2 July 2013.

Despite the cheers and jeers at leakers such as Julian Assange, Bradley Manning and Edward Snowden over the last few years, it would seem access to such information is less protected than ever. Serving intelligence officers are allowed to sell their skills to corporations. High-level intelligence work, once closely guarded, is farmed out to contractors, who leak information like a sieve.

The Obama White House has conducted more leak investigations than any previous administration, now including the leak of a cyber-warfare campaign against Iran by a top US army general, James Cartwright. We are selling our national security in an effort to save a buck – and we will continue to pay for it in other ways.

In 2010, Eamon Javers reported on CIA's controversial "moonlighting" policy, which allowed active intelligence officers to seek permission to work in the private sector on the side if they made full disclosure and it did not conflict with their duties. Generally, the same standard applies to all federal employees who are not political appointees.

But not all federal employees are CIA officers. No statistics of how many took advantage of the policy were forthcoming, but the policy itself was disturbing enough to lead to questions before Congress.

Why is this allowed? Defenders hold it is important to stop the "brain drain", where the best and brightest depart the agency for better pay. Supporters argue the level of compensation at CIA doesn't stack up against other federal or private-sector employment. This "discrepancy" leads talented employees to look elsewhere – for roles in which their skills fetch better prices. The CIA thus finds itself in competition with the private sector to keep personnel on whom it has expended a great deal of money and effort in training.

Demand for intelligence work has ballooned since 9/11, though government belt-tightening in recent years has had an effect. Washington has decided, rather than expanding intelligence services in a permanent sense, to farm out the increased workload to private contractors. This has advantages, as for any business seeking to outsource operations. When the need for the work reduces, it is easier to allow a contract to expire than it is to lay off excess federal employees or to keep them on, under-employed.

But there are drawbacks as well. Rather than paying solely for the service, the price also reflects the contractor's own costs – their HR, finance, management and marketing departments' wages, office costs, even redundancies. It is questionable if contracting saves anything at all. One doesn't have to look hard to find stories of wasted costs, foul-ups or even corruption in security contracting.

Just last week, USIS, the contractor responsible for performing background checks for security clearances – including Edward Snowden's – was accused of misleading the government as to the thoroughness of their investigations. Lawmakers have been told that thousands of background checks may have been improperly conducted. (The firm has made no comment, but issued a statement saying it was co-operating with an investigation.)

Private intelligence contractors look to hire employees who already have active clearances and intelligence experience – requirements not easily obtained elsewhere than an intelligence agency. The problem becomes circular. Farming out intelligence work to contractors creates its own demand. Agency employees leave for jobs with higher pay because the government itself has created a market for them, creating the very "brain drain" it seeks to stem. This is because bean-counters are worried about the effect on the budget – namely, of salary, benefits and pension obligations to employees. In the CIA's defence, these decisions are made in Washington, DC, not Langley, Virginia.

A second argument is that no harm is being done and these individuals have dedicated years to the country and are entitled to "feather their nests". However, the US military shares the same commitment to the country – yet service personnel get paid even less, and veterans are coming home to high unemployment rates.

While the job has its risks, most CIA employees still have desk jobs. The more academic nature of the job does not automatically entitle analysts to higher pay. There are also brainy folks at the Department of Interior or Forest Service who are overworked and underpaid. Many other less fortunate Americans would be happy to have such pay and benefits as CIA officers enjoy.

That our national security apparatus feels it must compete with the private sector to attract the best talent now means attracting the wrong kind of talent. Those who work in such a vital role should not do so for the salary, even if that is a great recruitment tool. One recruit who signs up for the right reasons is worth five who sign up for others. This should be familiar to US intelligence officers from the CIA's chief architect, Allen Dulles, who believed the most dependable sources are those not in it for pay.

The net effect on security is that it has become impossible for America to keep secrets. Security information is being handled by private companies poaching the intelligence community's talent and selling it back to the government, who paid to develop it in the first place. Government budget cuts and a failure to invest in intelligence infrastructure have created this problem.

National security challenges do not simply go away because we are having budget fights. If we are not willing to pay for this now, we'll pay for it later – in another, more sinister sense. Some things are worth paying for; our national security is one of them.

U.S. Military owes more to combat Joes than a four-star General

This article originally appeared in The Guardian on 14 November 2012.

The pressure on those in positions of great responsibility is tremendous. Although constantly surrounded, they are, in fact, alone. Men like General David Petraeus and General John Allen could never live up to the myths that surround them, pedestals they never asked to be placed upon as fame-seekers do – though, certainly in Petraeus' case, they haven't shied from it. To expect them to live up to the standards of perfection others have set for them is an impossible task no one could succeed at. They're not saints. They're soldiers. Marines.

Yet, at some point, "celebrity generals" pass out of the realm of being troopers and into being politicians. Similar uproar occurred when President Obama required General Stanley McChrystal, another admirable commander, to resign after he and his staff publicly criticised the commander-in-chief, a well-known "don't" in the military. That was a case of over-confidently taking shots in the press. The Petraeus affair is worse, in that it may have presented a real threat to national security as an opportunity for a foreign intelligence service to exploit.

I'm a two-tour army veteran of the Iraq war. I've spent the majority of my adult life as a soldier. We're no angels. Guys and girls like me come in any colour, from inner-city or rural America, places like the south side of Chicago or Lovelady, Texas. We're regular middle- or lower-class kids (average age: 19). Some joined for college money; some to get away from home; others out of a sense of duty. We like to fight, we like to swear, get tattoos, and we enjoy a good drink. We remain pretty much anonymous. We don't have biographers or make it onto the social circuit.

We're the absolute backbone of the country – Uncle Sam's misguided children. We're the men and women without any of the advantages in life. We're not supposed to succeed, but we do. In fact, it's the very flaws in each of our individual characters that make it all the more extraordinary what the US military has achieved in its 236 years. As we're reminded every Veteran's Day, victory was never certain and the sacrifices by our troops are great. These were the men and women fighting in Afghanistan while this melodrama was going on back home.

One can see how allowances can be made for the personal indiscretions of such people. Yet, I somehow can't help but feel let down by the expanding circle of this controversy, which has brought down General Petraeus and threatens to do the same to General Allen. I expected more of them, and the behaviour thus far revealed seems petty and melodramatic – a military soap opera complete with affairs, threatening notes, twins and fancy parties.

There are some who believe that this affair is nothing compared to the work General Petraeus has done for the country. Despite the great respect I have for the general, I find myself less willing to grant such allowances.

It is a military principle that a leader cannot impose a standard on others he doesn't impose on himself. This situation started out innocently enough, but seems to have gotten out of control. As director of an intelligence agency, the general would certainly have seen the danger this scenario presented if he had been looking at it from the outside. Director of National Intelligence James Clapper recognised that – and required Petraeus' resignation.

General Petraeus is widely respected by the military and veterans. But he is utterly exalted in both liberal and conservative Washington circles, which see him as a sort of noble warrior-philosopher, with credentials from West Point, Princeton and Georgetown. I was taught in the military that none of us is indispensable. We're all cogs in the wheel, and two minutes after we walk out the door, there will be an adequate replacement. The machine keeps going. The success or failure of operations in Iraq and Afghanistan, and of the United States as a whole, do not depend upon the fate of one or two men, no matter how exemplary their service. This country is stronger than that.

In fact, as study of our recent military campaigns has gone on, we have arguably been successful in many respects, despite the failures and inadequacies of some of our top military commanders and elected leadership. The credit for our limited successes should go to the great mass of regular, everyday troops who have been called upon to be not just soldiers, but police officers, doctors, aid workers, public works directors and social scientists.

The tough kids with high school diplomas got it done, not the administration in Washington, Ivy League professors, the brass in the Pentagon or generals at garden parties. These average Americans have done more than was asked of them and theirs should be all the credit.

The combat Joes are the ones who have been let down by this scandal. General Petraeus shouldn't have been placed on an exalted pedestal, even if he welcomed it. That very isolation may have clouded his judgment and created the risk of a security breach. In the end, though, he is a man just like other men: he is a cog in the wheel and the United States will go on without General Petraeus.