On 1 June 1931, Herbert O. Yardley’s The American Black Chamber stunned the world by exposing American codebreaking activities. Among the diplomatic messages uncovered by his Cipher Bureau, “the most important and far-reaching telegram” as Yardley put it was an instruction to the Japanese plenipotentiary sent to the Washington Naval Conference in 1921 (see another article).
While there have been statements that the book caused Japan and other nations to revise their codes at once, Kahn (2004) undermined such a myth by showing the number of solutions in the British archives of secret messages of Japan and Germany did not decrease after the publication of the book (Kahn p.131-136) (A skeptical view on such a traditional “offhand statements” is also presented by the editor of ASA, Wayne G. Barker (ed.), The History of Codes and Ciphers in the United States during the Period between the World Wars, Part I. 1919-1929 p.137). Materials in Japanese archives appear to support Kahn’s statements that Japan had been updating its diplomatic cryptosystems every couple of years and did not immediately change to new ones.
The U.S. government tried to prosecute Yardley under the Espionage Act, but he had not broken the law and could not be prosecuted. The Espionage Act was rewritten a year later.
Herbert Yardley was the surveillance state’s first betrayer, as loathed by insiders in his day as Edward Snowden is in ours. His book spilled secrets on a scale that a pre-Snowden-leak NSA described as follows:
In today’s terms, it would be as if an NSA employee had publicly revealed the complete communications intelligence operations of the Agency for the past 12 years–all its techniques and major successes, its organizational structure and budget–and had, for good measure, included actual intercepts, decrypts, and translations of communications not only of our adversaries but of our allies as well.
The same historical analysis declares that Herbert Yardley is “by all odds the most colorful, controversial, enigmatic figure in the history of American cryptology.”
He worked as a cryptanalyst for three countries, was commended by the U.S. government for his cryptanalytic achievements, then saw the same government summarily abolish his organization and, with it, his job. For later publicly revealing his success in cryptanalysis and secret writing, he was generally acclaimed by the press but reviled by the cryptologic community.
He wrote melodramatic spy novels and radio programs, and traveled the country speaking on cryptology and espionage. He hobnobbed with movie stars, famous authors, a future presidential candidate, and a future prime minister and winner of the Nobel Prize. He played championship golf; he played winning poker all his life and wrote a best-selling book on the subject. Motivated, probably, by bitterness and a need for money, he apparently sold cryptanalytic secrets to a foreign power, with results that, together with his other exposes, affected the course of U.S. cryptology for the next decade.
What’s more, he published his book and lent his expertise to foreign governments partly because he lost his job just before the 1929 stock-market crash and needed money and work. Put another way, if you survey the evidence-free accusations that surveillance-state apologists lob at Edward Snowden, you’ll find that the father of American cryptology actually did perpetrate those very transgressions.
Little wonder that James Bamford, the leading journalistic chronicler of NSA history, spends the second chapter of The Puzzle Palace looking back at Yardley’s life. (The first chapter of that book is summarized in part one of this series.) Yardley’s triumphs and serve as proof that cryptology can be of tremendous strategic value. For example, after cracking the code Japan used in cables to its embassy, Yardley was able to intercept a message about ongoing treaty negotiations with the U.S. that would determine the ratio of naval assets that Japan, Britain, and the U.S. would maintain. Thanks to spying, American negotiators knew that Japan would ultimately agree to their initial position.
But the peacetime surveillance operation that Yardley built also contained cautionary tales and surveillance abuses that have been repeated in very recent history.
The end of WWI presented an institutional threat to Yardley’s surveillance agency: The end of cable censorship made it harder to get material to read, and the Radio Communication Act of 1912 applied again. “No person engaged in or having knowledge of the operation of any station or stations should divulge or publish the contents of any messages transmitted or received by such station,” the federal law stated, “except to the person or persons to whom the same may be directed …” Bamford explains the context and ensuing events:
This was a very significant step for the United States, since it represented the first international convention of its type to which the country had adhered. To the Black Chamber, however, it represented a large obstacle that had to be overcome—illegally, if necessary.
By the time Yardley returned to the United States in April 1919, the State Department was already busy trying to establish a secret liaison with the Western Union Telegraph Company. It was hoped that Western Union would cooperate with the Black Chamber in providing copies of needed messages. For six months the State Department got nowhere; the Radio Communication Act provided harsh penalties for any employee of a telegraph company who divulged the contents of a message. Then Yardley suggested to General Churchill that he personally visit Western Union’s president, Newcomb Carlton. The meeting was arranged in September, and Churchill, accompanied by Yardley, raised with President Carlton the delicate matter of his secretly supplying the Chamber, in total violation of the law, copies of all necessary telegrams. After the men “had put all our cards on the table,” Yardley would later write, “President Carlton seemed anxious to do everything he could for us.”
Under the agreed on arrangements, a messenger called at Western Union’s Washington office each morning and took the telegrams to the office of the Military Intelligence Division in Washington. They were returned to Western Union before the close of the same day.
In the spring of 1920 the Black Chamber began approaching the other major telegraph company, Postal Telegraph, with the same request. Officials of this company, however, were much more disturbed by the possibility of criminal prosecution than were their counterparts at Western Union. For this reason, negotiations with the Black Chamber were carried on through an intermediary, a New York lawyer named L.F.H. Betts. All letters were carefully written so that no outsider would be able to understand what was really being said, and to camouflage the negotiations even further, Betts in one case communicated with General Churchill through the general’s wife.
In the end an agreement was reached, and that left only the smaller All-American Cable Company, which handled communications between North and South America. Yardley, later that same year, began negotiations with it through W.E. Roosevelt and Robert W. Goelet, who himself had been a commissioned officer in Military Intelligence during the