How Many CIA Agents Does It Take To Produce a Telephone Directory?
03.26.2008 - 10:33 AMOne of the most important functions carried out by U.S. intelligence is analysis of the vast quantities of data collected by the sixteen agencies of the intelligence community (IC). To do this job, the IC needs a lot of analysts. Who and where are they?
That has long proved to be a remarkably difficult question for senior officials to answer.
Ten years ago, one such official came up with the seemingly simple idea of creating a database that would record the names, locations, and specialties of all the analysts working in the IC. The computers were duly programmed, but the data were never entered. What went wrong?
Thomas Fingar, the director of national intelligence for analysis, and the chairman of the National Intelligence Council, has offered a fascinating explanation in a recent talk at the Council on Foreign Relations. The database, he explained, had been
billed as a “This is where we’ll go if we need to build a task force. We needed a Serbo-Croat speaker to send out to East Armpit.” And people didn’t put that data in. Others — managers — saw it as a free-agent list. “If I advertise what talent I’ve got, somebody . . . will try and steal it.”
The lesson here is that, try as one might to persuade people working in large organizations to cooperate for the common good, nitty-gritty career incentives will always and forever trump everything else.
Now, in the aftermath of 9/11 and other intelligence fiascos, the intelligence community is once again trying to create a telephone book. Fingar, who is running the initiative, has put out the word that being in the telephone directory is important: “If you’re not in it, it means one of two things: You don’t know anything, or your boss thinks you don’t know anything.” On top of that, a more draconian signal was sent out: “If you’re not in Fingar’s database, you’re not in a funded position.”
The results have been nothing short of astonishing. Reports Fingar:
I suddenly discovered I had 1,200 more analysts than I knew I had, even by estimating. But we can now reproduce phone book, e-mail directories. If you need to find an expert on economics in the Andean region, you can find out where they are, how to contact them. And people are using it.
Hearty congratulations are due the intelligence community. After ten years, it now has a telephone book. But where in the world is Osama bin Laden?
| »Back to Connecting the Dots | »Back to Commentary |
March 26th, 2008 at 12:07 PM
I regret to say that I have yet to applaud or even trust anything Fingar does. He is the intelligence-cooker behind the 2007 Iran NIE, and was moved to the DNI/NIC from INR at State, where he labored to undermine Bush’s GWOT policy.
Rassling down the IC by threatening agencies with funding cutoffs if they don’t yield up their personnel to the DNI is an interesting task for the “DNI for Analysis.” In most other organizations, this task would fall to a Director responsible for resource programming.
Incidentally, the threat of losing your people when you can least afford to because they have special skills is very real. The military does keep close track of its uniformed members’ special skills with a database (compliance from the uniformed is a given). You wouldn’t think forward-deployed forces would be vulnerable to having their special-skilled people plucked from them for other tasks (usually less pressing for national security), but it happens. I was flagged in the military personnel database my entire career as having French language capability, and more than once, the Atlantic Command tried to have me sent back from overseas deployments (e.g., in the Persian Gulf during the Iraq sanctions/no-fly zone years, and in the Mediterranean during the Balkans operations) to do a six-month hitch with the UN in Haiti. It took flag-level intervention to make Atlantic Command (4 stars to my boss’s one star) back off, and look elsewhere to fill its requirement.
The department-level managers who want to protect their manpower are not the wieners Fingar makes them out to be. They know perfectly well that their tasking load will not be lightened when their personnel are snatched away. People with relevant skills are rarely sequestered somewhere unrelated to their skills anyway: a manager who loses an Arabic linguist with special knowledge of Syria is one whose department already works on Syria. Fingar’s determination to have that linguist at HIS beck and call, rather than someone else’s, cannot truthfully be depicted as a national security interest. It’s just bureaucratic BS.
March 26th, 2008 at 1:43 PM
I’d like to both agree and disagree with Dyer at the same time. My 10 years in government were spent outside the IC, but the problem senior officials had identifying expertise in my area was very real. Yes, there was always a risk that the most competent people would be stolen away, at least for awhile. But if you want officials to craft informed policies, would you rather they have access to the best available expertise or be stuck with the lesser lights (and rank incompetents) that managers don’t care to protect — or in come cases, very much would like to lose?
Managers also have less honorable bureaucratic reasons for hiding good people — in particular, to be able to take credit for their work and, if possible, prevent them from going elsewhere. Keeping them out of the “phone book” serves both purposes rather well.
As I understand it, the DNI is supposed to be running a coordination function. If the DNI has to go through layers and layers of obscure channels just to locate expertise, never mind use it effectively, and each layer is aggressively turf protected, he is sure to fail. Nobody in his right mind would want that job.
Dyer’s legitimate complaint is not about the data base itself; it’s about the ability and willingness of some (always senior) people to abuse it. I grant that. But it means there is an unavoidable tradeoff that must be made between enabling the DNI to do his job on the one hand, and protecting subordinate IC offices (and employees) from abuse on the other.
March 26th, 2008 at 2:10 PM
Mr. Belzer — I do appreciate the perspective you outline. What I might (faintly) applaud if I approved of the DNI leadership, I can see all the faults in when I don’t.
I am quite confident of this: as long as Mr. Fingar and others in DNI dedicate themselves to spinning intelligence as they unquestionably did in the 2007 Iran NIE, the organizational coup of getting the personnel database filled in will NOT produce any improvements in the making of policy, or in the intelligence output. Task groups’ products are all limited in advance by the political boundaries of their tasks, and it is not the experts-on-call who set them. It’s the senior leadership.
March 26th, 2008 at 4:03 PM
As a manager involved in analysis, I always hoped that I would “get” three years from every hire. That is, three years within the division before they were recognized and offered better positions. How did they get recognized? By allowing them to brief executive management and take credit for their work. My “credit” was the hiring of these quality people in the first place. Unfortunately, I found that contributing your best and brightest to “joint” initiatives was NOT the norm for other managers and the analysts I put forward were recognized even more quickly! While I seemed to be in a constant “hiring mode”, the continued success of these qualified people was rewarding. Finally, I would question the ability of many of the senior management to effectively grasp “work management”. Many times exc management requested more resources than they required and many an analyst requested to be returned to his/her position because they were “BORED”!
March 26th, 2008 at 8:37 PM
Mr. Dyer,
I am not an IC-junkie, so I really can’t comment intelligently on the dispute over these particular people. I’ve been reading here and elsewhere legitimate reasons for concern, which I accept at face value.
My interest in this topic began with the publication of the 2007 NIE Summary. From my perspective as a risk analyst in a wholly different arena — human health risk — the NIE Summary struck me as remarkably free of policy bias. Reasonable people might quibble with the choice of subjective probabilities, but it seemed to be that they were destined to be subjective and dressing them up with fancy mathematics (which could have been done) would not change that.
I have read many of the complaints that the interesting part of the problem was assumed away in a footnote. But for me, it was spectacular that such a footnote was included. In the human health risk assessments I see and review, such details are simply left out. Moreover, it is standard practice to embed within an assessment of human health risk a deep and abiding policy preference that no human health risk ought to be tolerated. This is achieved by developing a risk estimate that is purposefully biased. For example, cancer risk is estimated by, among other things, using the 95th percentile upper-bound of a maximum likelihood estimate. It is virtually certain that actual cancer risk is less, and it is likely to be orders of magnitude less. Yet this is how it is done, and policy officials are left with little discretion to make sensible decisions.
Ironically, it might make sense to use such a technique for estimating the risk posed by Iran’s nuclear weapons program, because the consequences are so dire. (In cancer risk assessment, analysts worry about incremental increases in lifetime cancer risk as small as 0.000001, when the baseline risk of cancer exceeds 0.25.) I don’t want to know the expected value date by which Iran can deploy nuclear weapons. I want to know something much closer to the earliest possible date — like the worst-case scenario for cancer risk.
In any case, I did not see embedded policy bias in the NIE Summary, and I praised it for that clean distinction between science and policy. When I compared the 2006 and 2008 NIEs, they seemed to have the same message: The collective judgment of the IC is that Iran will succeed by the middle of the next decade. That the press got the wrong message is hardly surprising, of course.
Please take a look at my posts on the NIE and give me your feedback, either on my blog or here. Go to http://neutralsource.org/content/search/?search_query=iran and read the top 4 items.
Regards,
Richard Belzer
March 27th, 2008 at 12:31 AM
Mr Schoenfeld, I agree with you that “The lesson here is that, try as one might to persuade people working in large organizations to cooperate for the common good, nitty-gritty career incentives will always and forever trump everything else.”
This could be especially true of the C.I.A. Look at the treatment they have received by Congress and perhaps a couple of Presidents. Across the board budget cuts, reorganizations, agglomeration of Agencies and attempts to streamline the resulting organizations are very risky undertakings in government bureaucracies. Additionally, I suppose as in most other government agencies, diversity incentives replaced the merit system.
it is doubtful that Congress, in its attempt to undo the damage caused by separating the intelligence functions has improved the lack of communication or performance. Congress obviously does not know that NIH –Not Invented Here–, applies at all levels of the civil service, from the smaller sections up to the plum positions. Agglomerating agencies in a casual attempt at integration does not in itself lead to improvements.
The recent public dispute between the Agency realists, the neocons and Bush indicate a breakdown in discipline and possibly morale. Can the CIA and the intelligence community be put back together or improved? In my opinion, and I don’t have experience with the Defense nor Intelligence agencies, not likely! That is, unless there is a more severe terrorist attack here. If the major terrorist organizations hold off from another 9/11, and the Democrats sweep the next election there is a danger that the CIA and NSA will ultimately wind up as just another agency like the General Services Administration or be pruned down and stuck in the State Department.
As I wrote above, I have no experience with the Defense Department nor Intelligence community. The above remarks are based on my experience in the civil service starting in the Eisenhower administration, and my understanding of current events.
March 27th, 2008 at 1:31 AM
Mr. Belzer — Since I am retired Naval Intelligence officer, you will probably take as a given that Miles’ Law applies, in my case, to at least some extent. That’s OK. It is in-depth knowledge of the IC that yields my conclusion the 2007 Iran NIE was a political document, rather than a mere risk assessment. That doesn’t mean it wasn’t prompted by a valid event; it means that what it focused on, and what it left out, were not hallmarks of good intelligence writing but of politics.
My very first comment, after reading your essays on the NIE as risk analysis, is that the NIE appears to have misled you as to what the baseline event captured by the “intelligence” was. In your part 4, you analyze the possible-but-not-necessary inconsistency of the 2005 and 2007 estimates, regarding Iran’s susceptibility to sanctions pressure, in terms of the following propositions:
“a. The costs to Iran to sustain its nuclear weapons program increased significantly since 2003″;
or,
“b. The benefits to Iran from sustaining its nuclear weapons program decreased significantly since 2003.”
However, the observable event that prompted the NIE was Iran’s suspending the weaponization aspect of its nuclear programs IN 2003 — an event that was not influenced by changes in either costs or benefits SINCE 2003. To consider cost and benefit factors since 2003 as relevant to an analysis of the 2003 event on which the NIE was based, one would have to assume that Iran has, indeed, persisted in maintaining the suspension of 2003, and that whatever may have prompted the original suspension, more recent influences also matter.
The NIE presents no analysis, arguments, or references to positive intelligence for its brief assessment that the suspension continues. Continuation of the suspension is, in fact, very little addressed in the NIE, and yet is by far the most important question raised by it. We may all agree that Iran suspended weaponization in the fall of 2003 (and I do), but this NIE is remarkably uncommunicative on the obvious next questions: does that suspension constitute information about Iran’s ultimate intentions, and has weaponization been resumed since then? On the latter, an unsupported, unelaborated “probably not” is all we get. The NIE spends its time trying to set the 2003 suspension in a diplomatic context, and offers no justification for the almost off-hand estimate that the suspension remains in force.
In a properly written intelligence document, the whole focus would be on whether Iran intends to acquire nuclear weapons, and whether the suspension is still in force — because those are the key questions for policy. Intelligence is not abstract risk assessment, it is an attempt to answer questions for policy. It’s actually irresponsible to leave a big, honking implication for policy, and not address it more directly and clearly than this NIE addresses whether the Iranian suspension in weaponization continues — and what it means about Iranian intentions to acquire nuclear weapons. About this last, the NIE says only “we do not know whether it (Iran) currently intends to develop nuclear weapons.”
This is very poor intelligence support, however it may be judged as risk analysis. Whether Iran intends to develop nuclear weapons is precisely what the IC is supposed to be estimating. Merely saying “we do not know” is unprofessional; there should at least be an analysis of indicators. The 2003 suspension might be germane to answering this question, but by itself it is not an answer. This NIE, however, unprofessionally implies – but ONLY implies — that it is. The NIE leaves an implication that the 2003 suspension means Iran may not, in fact, intend to acquire nuclear weapons, but without making that case explicitly, and without a meaningful analytical discussion of it. Leaving unaddressed implications to do their own work, without explicitly committing yourself to one interpretation or another, is the very definition of spin. It is a political approach; it is NOT a valid intelligence practice.
What to make of this badly constructed, off-focus NIE is illuminated, for a career intelligence officer, by a caveat placed (in boldface) in its Scope Note, whose agnosticism is profoundly bizarre:
“This NIE does NOT assume that Iran intends to acquire nuclear weapons. Rather, it examines the intelligence to assess Iran’s capability and intent (or lack thereof) to acquire nuclear weapons, taking full account of Iran’s dual-use uranium fuel cycle and those nuclear activities that are at least partly civil in nature.”
In the context of abstract risk analysis, it may require no explanation that, after ten years of officially assessing that Iran intends to acquire nuclear weapons, the national intelligence community should insert — from out of the blue — such a clear-the-decks caveat in an NIE. But suddenly shifting to a wholly different posture without explanation is not a standard practice of the IC. This Scope Note caveat is a blast from the Twilight Zone, for anyone who has ever written an intelligence estimate. If the analytical decks must be cleared for some reason, that is explained and justified — it is not merely stated as a disembodied principle, as if no one in the customer base had any right to have a policy stake in previous estimates (and therefore to be owed a better explanation).
The two red flags for me, in analyzing this NIE, are the out-of-nowhere Scope Note caveat, and the NIE’s implication – which it then fails to explicitly address — that the 2003 suspension is an answer to the policy question of whether Iran intends to acquire nuclear weapons. Both features of the NIE are colossal breakdowns in intelligence professionalism. It is exactly the role of intelligence to be accountably explicit about its implications, and to answer the questions of policy — with analysis, at least, if not with definitive assertions. Intelligence has been politicized when it puts out a product that does not answer any policy question, while leaving for its audience an unaddressed implication; and one that, leaving unaddressed, the IC apparently hopes to avoid being held accountable for.
I note that none of this could be attributable to the NIE executive summary being unclassified. The executive summary will not be misleading about the classified version, however much detail it may eliminate. Since “Does Iran intend to acquire nuclear weapons?” is THE key question for our national policy, an answer expressing the same basic sense will appear in both versions, if there is such an answer. Too many members of Congress and senior executives see both versions; no conflict between them could go unnoticed. We may be certain that the bottom line expressed in the executive summary – “we do not know” – is what appears in the classified NIE. Moreover, if the classified NIE contained an explicit discussion of the implication about the 2003 suspension and Iranian intentions, the unclassified summary would contain the discussion’s conclusions, even if the details could not be elaborated.
March 27th, 2008 at 9:21 PM
Thank you, kindly. This is very helpful and informative. My risk analysis community (the Society for Risk Analysis) reaches pretty widely, but it does not include IC, so I readily admit a knowledge gap here. Regarding my posts on the NIE, please understand that Neutral Source’s mission and charge is to refrain from injecting policy biases in its analyses. It would be a tad hypocritical to complain about this problem in others but engage it in ourselves.
Since 1983 (the NAS “Red Book”), the accepted paradigm in risk analysis is more or less as follows:
1. Science (i.e., facts and estimates about uncertain facts) ought to be clearly distinguished from policy (i.e., opinions about what ought to be done about the facts, including perhaps choosing to ignore them). The critical thing is not to misrepresent policy as if it were science. The temptations to do so are, of course, very strong. Paraphrasing Moynihan, everybody is entitled to their own opinions but not to their own facts.
2. There is a gray area between science and policy that we call judgment, a bad term because it is imprecise about whether it is scientific or policy in nature. It refers to inferences from facts that do not necessarily require the injection of policy, but nevertheless are highly susceptible to the influence of policy. In practice, a lot of effort is devoted to probing the question of how much judgment is scientific and how much is policy. Unfortunately, consumers of risk assessments do not get to interrogate the risk assessors to find out. (Many times in my government career I would be loved to waterboard an agency risk assessor to find out.)
3. The objective is to produce risk assessments in which informed customers cannot discern an embedded policy bias. That is, the risk assessment ought to be accepted as “factual” by all sides in the policy debate — in this case, irrespective of one’s views about (a) how serious a national security threat it would be if Iran deployed nuclear weapons and/or (b) what ought to be done to mitigate or adapt to this risk. If a demonstrable embedded policy bias can be detected, then the risk assessment is low quality and should not be disseminated. A high degree of uncertainty is not per se evidence of low quality; it may just reflect more ignorance than knowledge. A risk assessment that says we know less than we do is not objective any more than one that exaggerates knowledge and downplays ignorance.
In practice this paradigm breaks down at several places. For example, objective risk assessment per se does not have policy neutral effects. One side or the other will benefit more in any particular case. There is a widely held belief that competing factions can agree on objective risk assessment because it serves their long-term interests to have information that is free of policy bias. However, this belief appears to be incompatible with observable facts. Some policy views cannot be sustained without resort to biased risk assessment, and partisans for such policy views will either not agree to be bound by objective risk assessment or they will reneg on their agreements.
Nonetheless, the paradigm is useful because it seems to be the least-bad a priori rule of practice that we can devise. We do not yet have very good incentives for rewarding objectivity and punishing bias, nor do we have effective tools for effecting internal discipline. Trying to (partially) solve these problems are research interests of mine, but the going is slow and tough. The creation of an IC expert directory — the post that started this discussion — seems to me to be the kind of effort I would have undertaken to better align incentives, and of course I have only the noblest of motives.
Still, the three most salient observations I have remain:
1. The text was rich enough to permit experienced analysts such as you to find fault in it. In that sense (and perhaps that sense alone), it was good for it to be published.
2. The most important issue is the predicted date of nuclear weapons deployment, and the 2006 and 2008 versions do not seems to differ much. Perhaps that means the NIE gave too much attention to the secondary issues that were covered in the press so relentlessly.
3. Which brings us to a huge problem: The NIE had to be published because the IC (or some element thereof) or a customer was incapable or unwilling to keep it out of the New York Times. If this problem is not solved, intelligence will become a product that government no longer can credibly produce. Policy- and decision-makers will have to go outside the government to learn what the government should strive to know.
I have the impression (informed only by anecdote and casual empiricism) that the problem of legitimate secrecy has gotten much worse since the office of the DNI was created. That makes intuitive sense to me because forcing disparate views into a consensus throws away valuable information. Those who believe that their views are getting short shrift (which, ironically, could be every IC agency and expert) have strong incentives to upset the system by leaking. It also violates my economics training, which tells me that it is better to have multiple providers of intelligence acting in competition rather than a monopoly.
Had it been up to me, I would have “solved the IC problem” by increasing competition amongst the various agencies, not forcing them to paper over their differences. But what do I know?
March 28th, 2008 at 1:58 AM
I admire the analytical perspicacity that led you quickly to a point I’ve made here at CTD before: that a key factor prompting the publication of this NIE was a concern to avoid being accused of NOT making Iran’s 2003 weaponization suspension public.
That said, the 2003 suspension was enough of a pretext to update the NIE. The question was really how to couch the update. The current DNI leadership chose to feature the single 2003 data point, leave out discussion of the other elements of Iran’s nuclear programs, and not address the implications of the 2003 suspension implications analytically, merely leaving them to have a political impact.
The opposite decision could have been made, and it would have been more professional: to explicitly update the assessment of Iran’s intentions and overall progress, in light of all the new intelligence since the 2005 NIE: the 2003 suspension, progress in uranium enrichment, ballistic missile programs, Russian statements and actions, and Iran’s posture with IAEA and the EU-3. No possible professional consideration accounts for the DNI’s decision not to do this.
Another point you bring up is an excellent one, which is that policy bias seems to be inherent in risk assessments. This is something we are only just learning about the inevitable character of preemptive policy, and the intelligence it bases itself on. We should not be surprised at it, though, because when you think about it, defining risk in the first place is done in the context of priorities and preferences. It has to be; where there is no bias in favor of one state or outcome, there is no concept of risk. Wanting to avert or manage risk implies a preference about outcomes, otherwise we wouldn’t even bother putting in the effort.
In the brave new world of preemptive policy, we have thought very little about defining risks and policy triggers. If I asked Americans the question, “When will we know Iran’s nuclear program poses a great enough risk that we need to take action, and what action should we take?” I would get many, many answers, almost none of them precise. We are content to vaguely suppose Iran’s nuclear program poses a threat, and that intelligence will tell us when that threat has become “too great.”
But nowhere have any criteria been defined for this assessment. We have no common idea whatsoever what we even mean when we say the Iranian nuclear threat could become too great. Too great for what? What peril to our national preferences is suggested here? What action is implied? The concept of risk has no meaning outside of these policy biases, but we act on the assumption of risk – assigning intelligence to “monitor” it — without defining them.
Our political leaders berate each other with opposing views on aspects of this question, but without getting us any closer to a common, popular understanding of it. Instead, partisans are free to speechify as if their perspectives are obvious and universal – and intelligence analysts are free to set their own context of national preferences and implied action. We are quick to assume that this is because the particular incumbent of the Oval Office is a weak leader of his own bureaucracy. There may be some truth to that, but I think the more important factor is the situation itself: assigning intelligence to effectively tell us when to preempt a threat, as a substitute for having a defined policy on when to do it.
Foreign policy is always resistant to being cornered by prior definitions, of tripwires and triggers, so it will not help us out here by demanding them. Lack of accord among our political leaders feeds the trend against them. But without these definitions, policy relinquishes the wheel to its servant, the risk monitor. In default of guidance from policy, the risk monitor – intelligence — will use its own definitions of the biases that give its project form and meaning. We have actually seen this phenomenon wax and wane in the IC since the late 1940s; it’s not peculiar to the 2000s or the Bush administration.
The technology that gives us global surveillance and expeditionary capability has given us options for preemption that no one ever used to have. It has also enlarged the effective policy purview of intelligence’s detection and analysis disciplines (on which preemption decisions are predicated), at the expense of our duly constituted political leadership.
We need not despair. I believe that what we are seeing is evidence for a modern extension of the long-held axiom that a civilian political leader should know how to supervise and direct war: how to give his generals strategic direction and executable guidance. Having this supervisory skill over intelligence will, in my view, be necessary for national political leaders in the future, as we make more policy decisions that are based on intelligence findings. It is not enough to ask intelligence to “monitor and assess the risk of Iran’s nuclear program.” If the purpose for monitoring is to contemplate the need to take national action, the political leader will have to both choose his intelligence leaders wisely, as he would his closest advisers, and frame his policy questions to intelligence in terms he has deliberately chosen, and is prepared to act on accountably. Framing the policy boundaries of risk assessments by intelligence will be as supremely important as we know framing war objectives to be. Indeed, it already is.
March 28th, 2008 at 9:39 AM
In the broader (i.e., non-IC) risk world, risk is defined as probability x outcome, with both probabilities and outcomes being uncertain and outcomes being either good or bad. That is, it is not the case that policy preferences drive the definition of risk. (In human health risk assessment, it is commonplace to assume away positive outcomes and worry only about the negative. However, this is a tortured constraint on the definition that exists because so many of its practitioners are trained in toxicology — the science of killing things — and they are subliminally embarrassed by it.)
There is a large subfield in risk analysis under the dominion of Bayesians, and I would put myself squarely in that camp. For us, it is professional malpractice to throw out data. New data should be used to update the storehouse of knowledge, not to supplant it. I understand your criticism of the NIE is that what they did was discard the storehouse and focus solely on the single new datum indicating that weaponization had been suspended.
If so, that is a major (but remediable) process failure. Risk managers should not direct risk assessors to look at new data in isolation, and risk assessors should refuse to do so if so directed. Similarly, if the decision to do this arose within the IC, then it was the responsibility of risk managers to reject the draft work product as analytically defective for its intended purposes and reassign the career managers responsible for having permitted it.