E

spionage and covert intelligence activities have long uneasily coexisted with American constitutional principles and democratic values. Appalled by the Black Chamber, a code-breaking office created during World War I, Secretary of State Henry Stimson famously demanded in 1929 that it be shuttered because “gentlemen don’t read each other’s mail.” Pressed to approve the peacetime continuation of the Office of Strategic Services, the World War II spy agency, Harry Truman refused and warned about creating an American Gestapo. Yet, as secretary of war during World War II, Stimson made abundant use of cryptanalysis. And Truman ultimately approved the creation of the Central Intelligence Agency and oversaw a burgeoning national-security apparatus.

By its very nature espionage is morally dubious. It involves deception, secrecy, underhanded and often illegal activities. Its practitioners are often unsavory human beings. But every nation in the world resorts to it to protect its interests and thwart its enemies. Democracies, no less than dictatorships, have of necessity used spies and attempted to fend off enemy spies, but usually with more scruples. Invariably, when spy agencies overstep their bounds, democratic societies agonize over the costs of national security and the extent to which they have become like their foes.

The latest manifestation of this tension was the reaction to Edward Snowden, a contractor for the National Security Agency who fled first to Hong Kong and then Russia in 2013 with tens of thousands of downloaded top-secret files detailing the agency’s worldwide surveillance of telephone calls and email messages. Unlike many traditional spies who sell or provide the secrets they have purloined to foreign countries, Snowden shared them with journalists. That has led to a fierce ongoing debate about intelligence overreach, endangered national security, and the irony that someone who insisted his only purpose in purloining classified information was to provide openness and transparency has found refuge in a closed, oppressive society that routinely violates human rights and privacy. Whatever Snowden’s motives or legal culpability, his revelations starkly demonstrated that the remarkable technological advances in the 20th century that have enabled governments to intercept electronic communications pose significant challenges to the Fourth Amendment to the Constitution, which ensures protection against unreasonable searches and seizures without probable cause. Also, by exposing the collection of radio communications of Taliban insurgents in Afghanistan, he endangered the lives of American servicemen.

Stephen Budiansky, a journalist who has written extensively about codes and code-breaking, traces the successes and failures of American signals intelligence, SIGINT, during the Cold War. His new book, Code Warriors, focuses largely on the covert war with the Soviet Union, with excursions into conflicts with its allies, North Korea, North Vietnam, and Cuba. With its emphasis on state actors, Code Warriors does not deal with the NSA’s activities to counter terrorism, efforts that have increasingly defined its mission in the past two decades.

Budiansky is scathing about the activities of American intelligence agencies, suggesting that they have “often failed spectacularly at crucial moments, and had left in their wake an often sordid trail of transgressions against law, morality, decency, and basic American values.” He is somewhat less critical of the National Security Agency but does argue that its successes have coexisted with a culture of extreme secrecy and obsessive desire to collect whatever information it could, and that this culture led it to violate individual privacy and avoid legislative oversight without much thought of the consequences.

Perhaps the most impressive achievements of SIGINT during and immediately after World War II were the remarkable successes in breaking German, Japanese, and Russian codes. In England, code breakers at Bletchley Park capitalized on Alan Turing’s brilliant insights to unlock the secrets of the German Enigma machine and read high-level military communications, enabling Allied forces to anticipate enemy movements and plans. In America, separate Navy and Army cryptographic units broke both German and Japanese military codes, making possible the American victory at the Battle of Midway.

In 1943, an odd array of employees at Arlington Hall in Virginia, site of the Army Security Agency, began work on “the Russian problem” in even greater secrecy than normal in the famously secretive world of cryptography. Even Great Britain was initially kept out of the loop. Not only was Russia an ally; earlier work on Russian codes had gotten nowhere. But in 1940, the Army code breakers had persuaded RCA, which was a major carrier of commercial telegrams, to allow one of its officers to copy all international cables—hundreds of thousands of encoded and enciphered Russian messages were available. And, following the Pearl Harbor disaster, the new mantra was “Get everything.” Rumors that Stalin might sign a separate peace deal with the Nazis and a firm belief that the wartime alliance with the Soviet Union was unlikely to survive victory gave further impetus to the effort.

A team of cryptanalysts was formed, including several raw recruits like Cecil Phillips, a teenage chemistry student at the University of North Carolina whose only exposure to code-breaking came from once owning a Little Orphan Annie decoder ring, and Gene Grabeel, a high-school home-economics teacher who was hired for a job about which neither she nor her recruiter had any inkling. The team began the laborious task of trying to break a seemingly unbreakable code.

Because of Soviet errors using certain types of pads to encipher messages, cryptanalysts managed to begin making breaks in the Soviet code in late 1943, but progress was painfully slow. It was not until late 1946 that Meredith Gardner—a linguist proficient in Spanish, German, Russian, Bulgarian, Hebrew, and Japanese—figured out the “spell table,” by which the Soviets spelled out proper English names. It became apparent that many of the intercepted messages did not deal with diplomatic or trade matters, but were about Soviet espionage in the United States.

Over the next few years, dozens of important Soviet spies were uncovered, including Klaus Fuchs, Judith Coplon, the Rosenbergs, Ted Hall, Harry Dexter White, Larry Duggan, and Donald Maclean. The project, code-named Venona, confirmed the stories told by such defectors from Soviet intelligence as Whittaker Chambers and Elizabeth Bentley. It was regarded as so sensitive that even President Truman was not briefed about how the information about Soviet spies had been obtained.

As the Cold War took shape, the importance of SIGINT only in-creased. HUMINT, or human intelligence, failed miserably to provide information about the Soviet Union. Efforts to deploy agents in Eastern Europe ended in disaster, due to betrayals by Soviet moles like Kim Philby. Penetrating the Soviet Union itself would take decades longer. Reading Soviet communications, not only from Venona but other systems, was crucial to American policymakers. And that meant ensuring that the Soviets did not learn their communications had been compromised.

To avoid exposing Venona, most of the spies it revealed were never prosecuted. One who was, Judith Coplon, had both her convictions reversed—even though she was caught in the act of passing classified information to a Soviet agent—because FBI agents unable to mention Venona left the impression while testifying that she had come under suspicion as a result of wiretaps without a warrant. All the precautions turned out to be worthless, however. It happened that William Weisband, a Russian translator at Arlington Hall, was a longtime Soviet agent; in November 1948, following his warnings, the USSR made wholesale changes in all of its encryption systems that made them unreadable.

That bit of counter-espionage inaugurated a decade-long hiatus in America’s ability to read Soviet messages. Weisband was likely the most destructive spy in American history; he crippled America’s ability to get a full picture of the USSR just before the latter began to prepare for the Korean War. Tellingly, even after Weisband was exposed in 1950, he was not prosecuted, preserving secrecy even though the Soviets knew everything there was to know about Venona. Budiansky notes that the secrecy about Venona provided space for demagogues like Joseph McCarthy to make irresponsible charges about Soviet espionage and led many liberals mistakenly to defend real spies like Alger Hiss, Harry White, and Lauchlin Currie.

Weisband was not the only spy to betray SIGINT operations. The CIA built the Berlin Tunnel to tap into Soviet cables in East Berlin carrying military communications. Fighting a bureaucratic war with the NSA, the CIA did not inform its rival until the tap was operational, and then it limited the NSA’s access to the material. The project was quickly betrayed by British spy George Blake. Demonstrating that petty jurisdictional battles were not limited to the United States, the KGB (whose lines were not tapped), never informed the Soviet military intelligence agency (the GRU) or the East German Stasi about the tunnel’s existence, allowing them to bleed secrets, until it could plausibly “discover” the tunnel without exposing Blake a year later.

Bureaucratic wrangling significantly weakened the effectiveness of American signals intelligence. Until 1952, when the National Security Agency was set up, each of the three military services had its own signals unit, and they squabbled over resources, tasks, and prerogatives. The Army and Navy units had totally different cultures; the former was staffed largely by civilians, the latter with more military. Before World War II, in fact, jurisdictional warfare resulted in them alternating days on which to decode Japanese diplomatic traffic. Even after NSA was created, different elements of the intelligence community fought over turf. SIGINT was considered information, not intelligence, so the CIA, G-2, Office of Naval Intelligence, and the State Department all insisted that the NSA should not provide analysis of the data it generated—while the NSA resisted providing huge amounts of raw data to other agencies.

Deprived by Weisband of the ability to read high-level Russian messages, the NSA focused on scooping up electronic wisps and signals from radar, photo reconnaissance, and clear, low-level military messages. The amount of traffic it intercepted was massive: 1.3 million messages a month by 1953. By 1955, NSA intercept stations around the world were sucking in such enormous amounts of data that 37 tons of paper tape and printouts were generated per month. The mountains of information required more and more powerful computers that, in turn, inspired hopes that brute machine force could penetrate the devilishly complicated Soviet codes.

Despite the devastating setback caused by Weisband, techniques like traffic analysis did provide crucial information. SIGINT gave Allied forces advance notice of North Korean troop movements, enabling outnumbered forces in Pusan to fend off their attacks. Information gathered in this way provided General Douglas MacArthur key information about the concentration of Chinese troops prior to their entry into the war; MacArthur tragically ignored the danger, with a loss of thousands of soldiers. But American air superiority in Korea was ensured when Arlington Hall intercepted communications between Russian MIG pilots and their ground controllers. (That Russians were engaged in combat with American forces during Korea was kept secret for 25 years.)

In Budiansky’s telling, Vietnam was the nadir of the agency’s performance. The NSA provided misleading information to policymakers about the Gulf of Tonkin incident, failing to make clear that some intercepted material cast doubt on whether North Vietnamese torpedo boats had deliberately attacked an American ship. Pressured by an administration determined to use the incident to get congressional authorization for an expanded war, the NSA engaged in a cover-up and altered translations to support the preferred narrative. Overconfidence, poor security, and disdain for North Vietnamese signals capability led to major lapses that allowed the North to learn about forthcoming American air strikes. The NSA missed signs of the 1968 Tet Offensive. That same year, the North Korean capture of the USS Pueblo, a Navy electronics ship with NSA electronic listening gear, laid bare the poor coordination between the two agencies, demonstrated lapses in training and security, and gave Communists access to top-secret American technology.

By the late 1960s, the development of fiefdoms within the NSA that were insulated from accountability, and growing inefficiency fueled by cultural clashes between its military leaders and its largely civilian work force led to declining morale. While it escaped most of the fallout from the Watergate investigations, the NSA entered a new world in 1978 with passage of the Foreign Intelligence Surveillance Act, which required outside approval of electronic surveillance of foreigners in the United States and American citizens overseas. The act set up a system of legislative oversight and outlawed dragnet collections of electronic data in the United States. Meanwhile, the development of new supercomputers by 1979 once again gave the NSA a purchase on decrypting Soviet messages.

Some of the NSA’s failures cited by Budiansky were as much the result of the blindness or stubbornness of political and military leaders like MacArthur or Lyndon Johnson. Nothing new there; Joseph Stalin had advance warning from his intelligence services of Operation Barbarossa but stubbornly refused to believe that Adolf Hitler would attack the Soviet Union. No matter how good the information supplied by intelligence, policymakers have to have good judgment and be prepared to have their prejudices and assumptions disconfirmed.

Budiansky’s account also reminds us that spies can do enormous damage. The USSR might well have eventually ended the vulnerabilities that enabled American cryptanalysts to read their messages. But Weisband’s betrayal (and Blake’s) deprived America of crucial information that could have changed the course of history.

Even with its shortcomings, the NSA had one paramount achievement noted by Budiansky. SIGINT provided assurance that America would not be subjected to surprise attack. At key moments, like the Cuban missile crisis, it gave decision makers confidence that the Soviets were backing off, avoiding escalation of situations that could have led to the use of nuclear weapons.

The NSA’s technical capacity to scoop up information and violate privacy is worrisome. But in a world where terrorists can jet around the globe, communicate with sympathizers via the Internet, and employ frightening weapons to kill civilians, using SIGINT to catch them before they act justifies some of the controversies generated by the existence and collection of intelligence information through electronic means. The history of SIGINT is the history of conflict in the era of mass communication, and the story is well told in Stephen Budiansky’s fascinating book.