On 1 June 1931, Herbert O. Yardley’s The American Black Chamber stunned the world by exposing American codebreaking activities. Among the diplomatic messages uncovered by his Cipher Bureau, “the most important and far-reaching telegram” as Yardley put it was an instruction to the Japanese plenipotentiary sent to the Washington Naval Conference in 1921 (see another article).
While there have been statements that the book caused Japan and other nations to revise their codes at once, Kahn (2004) undermined such a myth by showing the number of solutions in the British archives of secret messages of Japan and Germany did not decrease after the publication of the book (Kahn p.131-136) (A skeptical view on such a traditional “offhand statements” is also presented by the editor of ASA, Wayne G. Barker (ed.), The History of Codes and Ciphers in the United States during the Period between the World Wars, Part I. 1919-1929 p.137). Materials in Japanese archives appear to support Kahn’s statements that Japan had been updating its diplomatic cryptosystems every couple of years and did not immediately change to new ones.
The U.S. government tried to prosecute Yardley under the Espionage Act, but he had not broken the law and could not be prosecuted. The Espionage Act was rewritten a year later.
Herbert Yardley was the surveillance state’s first betrayer, as loathed by insiders in his day as Edward Snowden is in ours. His book spilled secrets on a scale that a pre-Snowden-leak NSA described as follows:
In today’s terms, it would be as if an NSA employee had publicly revealed the complete communications intelligence operations of the Agency for the past 12 years–all its techniques and major successes, its organizational structure and budget–and had, for good measure, included actual intercepts, decrypts, and translations of communications not only of our adversaries but of our allies as well.
The same historical analysis declares that Herbert Yardley is “by all odds the most colorful, controversial, enigmatic figure in the history of American cryptology.”
He worked as a cryptanalyst for three countries, was commended by the U.S. government for his cryptanalytic achievements, then saw the same government summarily abolish his organization and, with it, his job. For later publicly revealing his success in cryptanalysis and secret writing, he was generally acclaimed by the press but reviled by the cryptologic community.
He wrote melodramatic spy novels and radio programs, and traveled the country speaking on cryptology and espionage. He hobnobbed with movie stars, famous authors, a future presidential candidate, and a future prime minister and winner of the Nobel Prize. He played championship golf; he played winning poker all his life and wrote a best-selling book on the subject. Motivated, probably, by bitterness and a need for money, he apparently sold cryptanalytic secrets to a foreign power, with results that, together with his other exposes, affected the course of U.S. cryptology for the next decade.
What’s more, he published his book and lent his expertise to foreign governments partly because he lost his job just before the 1929 stock-market crash and needed money and work. Put another way, if you survey the evidence-free accusations that surveillance-state apologists lob at Edward Snowden, you’ll find that the father of American cryptology actually did perpetrate those very transgressions.
Little wonder that James Bamford, the leading journalistic chronicler of NSA history, spends the second chapter of The Puzzle Palace looking back at Yardley’s life. (The first chapter of that book is summarized in part one of this series.) Yardley’s triumphs and serve as proof that cryptology can be of tremendous strategic value. For example, after cracking the code Japan used in cables to its embassy, Yardley was able to intercept a message about ongoing treaty negotiations with the U.S. that would determine the ratio of naval assets that Japan, Britain, and the U.S. would maintain. Thanks to spying, American negotiators knew that Japan would ultimately agree to their initial position.
But the peacetime surveillance operation that Yardley built also contained cautionary tales and surveillance abuses that have been repeated in very recent history.
The end of WWI presented an institutional threat to Yardley’s surveillance agency: The end of cable censorship made it harder to get material to read, and the Radio Communication Act of 1912 applied again. “No person engaged in or having knowledge of the operation of any station or stations should divulge or publish the contents of any messages transmitted or received by such station,” the federal law stated, “except to the person or persons to whom the same may be directed …” Bamford explains the context and ensuing events:
This was a very significant step for the United States, since it represented the first international convention of its type to which the country had adhered. To the Black Chamber, however, it represented a large obstacle that had to be overcome—illegally, if necessary.
By the time Yardley returned to the United States in April 1919, the State Department was already busy trying to establish a secret liaison with the Western Union Telegraph Company. It was hoped that Western Union would cooperate with the Black Chamber in providing copies of needed messages. For six months the State Department got nowhere; the Radio Communication Act provided harsh penalties for any employee of a telegraph company who divulged the contents of a message. Then Yardley suggested to General Churchill that he personally visit Western Union’s president, Newcomb Carlton. The meeting was arranged in September, and Churchill, accompanied by Yardley, raised with President Carlton the delicate matter of his secretly supplying the Chamber, in total violation of the law, copies of all necessary telegrams. After the men “had put all our cards on the table,” Yardley would later write, “President Carlton seemed anxious to do everything he could for us.”
Under the agreed on arrangements, a messenger called at Western Union’s Washington office each morning and took the telegrams to the office of the Military Intelligence Division in Washington. They were returned to Western Union before the close of the same day.
In the spring of 1920 the Black Chamber began approaching the other major telegraph company, Postal Telegraph, with the same request. Officials of this company, however, were much more disturbed by the possibility of criminal prosecution than were their counterparts at Western Union. For this reason, negotiations with the Black Chamber were carried on through an intermediary, a New York lawyer named L.F.H. Betts. All letters were carefully written so that no outsider would be able to understand what was really being said, and to camouflage the negotiations even further, Betts in one case communicated with General Churchill through the general’s wife.
In the end an agreement was reached, and that left only the smaller All-American Cable Company, which handled communications between North and South America. Yardley, later that same year, began negotiations with it through W.E. Roosevelt and Robert W. Goelet, who himself had been a commissioned officer in Military Intelligence during the war. Regardless of whether All-American cooperated, by the end of 1920 the Black Chamber had the secret and illegal cooperation of almost the entire American cable industry.
American cryptology had lost its virginity.
Despite its willingness to break the law to ensure its institutional survival, the Black Chamber would face significant cuts to its secret budget in ensuing years, and in the winter of 1929, the secret agency faced the prospect of a new president. “To Yardley, any change in Washington was viewed as a potential threat to his Black Chamber, and he advised his liaison at the State Department not to reveal the existence of the organization to the new Secretary for a few months, in the hope that any idealism [Henry] Stimson may have had before taking office would be tempered by reality.” When Secretary of State Stimson finally found out, his reaction “was immediate and violent,” Bamford writes. He declared the Black Chamber “highly illegal” and is reported to have said, “gentlemen do not read each other’s mail.” Soon after, Yardley lost his job.
Not long after that, having spent years insisting his work must be kept absolutely secret for the sake of national security, he spilled all his secrets in print, enraging the government, which nevertheless declined to prosecute him. The State Department lied to the public by disavowing the truth of the book’s contents. And Yardley subsequently wrote another work that focused on Japanese codes, making heavy use of actual messages he’d intercepted from Japan. It was “the first and only manuscript in American literary history to be seized and impounded for national security reasons by the United States government.”
As Bamford goes on to relate, Congress then passed a law that survived for decades, only slightly amended. The language that FDR signed on June 10, 1933:
… whoever, by virtue of his employment by the United States, shall obtain from another or shall have custody of or access to, any official diplomatic code or any matter prepared in such code, and shall willfully, without authorization or public authority, publish or furnish to another any such code or matter, or any matter which was obtained while in the process of transmission between any foreign government and its diplomatic mission in the United States, shall be fined not more than $10,000 or imprisoned not more than ten years, or both.
