A computer mouse is illuminated by a projection of a Chinese flag in this photo illustration, October 1, 2013.
Tim Wimborne / Reuters

Three years ago, Edward Snowden, a CIA employee, leaked documents revealing the U.S. government’s extensive surveillance of foreign and domestic phone calls. On the anniversary of that explosive disclosure, media outlets are no doubt preparing to revisit questions about data privacy and civil liberties.

What the world might be missing, however, is the way in which surveillance technologies have already extended beyond the interception of phone calls, e-mails, and text messages. In fact, it isn’t too far off to imagine governments being able to spy on your e-mails, Facebook posts, and tweets before you send them. Nor is it unrealistic that states and private firms could intercept and read .docx files, .rtf files, and indeed all text files produced using ostensibly non-transmitting programs.

Finally, it is possible that such surveillance could eventually be conducted in real time, where text documents could be intercepted even before one pressed “save” or committed a file to the cloud.

All of this is speculative, but that doesn’t make it unreal.

People carrying mobile phones walk past a banner supporting Edward Snowden, a former contractor at the National Security Agency (NSA), at Hong Kong's financial Central district, June 18, 2013.
People carrying mobile phones walk past a banner supporting Edward Snowden, a former contractor at the National Security Agency (NSA), at Hong Kong's financial Central district, June 18, 2013.
Bobby Yip / Reuters

EVERY KEY YOU STROKE
For the past decade, I have been working on the history of modern Chinese information technology, with the results due to appear in a pair of books that MIT Press is publishing next year. I had no idea that in the course of my research I would stumble upon something relevant to present-day debates about state surveillance, data privacy, and the Snowden leak that shocked the world. But I did.
In a nutshell: Chinese computing has a “keylogging” function effectively baked in. The technology came about for non-malicious (and rather brilliant) reasons, but it has made it theoretically possible to spy on Chinese computer users in real time—even in cases in which one is using seemingly offline and non-transmitting text programs such as Microsoft Word, NotePad, TextEdit, and more. And current trajectories in media technology strongly suggest that this vulnerability will soon spread beyond China, becoming, in effect, the new normal.

To see why, one needs to understand keyloggers and Chinese computers—and how the two came together.

A keylogger is a program or a physical device installed in a computer that collects data on every key the user depresses. Typically associated with malicious intent—such as stealing credit card information, passwords, or other personal data—keyloggers also have a long history in government surveillance. In a landmark case from 1999, FBI agents employed a keylogging device for perhaps the first time, targeting the computer of Mafia kingpin Nicodemo Salvatore Scarfo, Jr., to determine his passwords.

In the early years of keylog spycraft, the prime challenge was not building the keylogging device but installing it. In the case of Scarfo, Jr., the FBI had to rely on good old-fashioned cloak-and-dagger techniques, breaking into the crime boss’ office not once but twice—first to install the device and then to get it back.

With the proliferation of malware, it has become far easier to install keyloggers, but the challenge to would-be keylog spies still remains nontrivial. In 2015, researchers at the University of Illinois at Urbana-Champaign demonstrated that a SmartWatch could be transformed into an imperfect keylogger by using the device’s accelerometer and gyroscope data. As the theory read: if one could determine how a user’s hand moved across the keyboard surface, it stood to reason that one could determine which keys the user struck.

As devilishly brilliant as this and other workarounds are, Chinese computing offers up a far better way to determine exactly what a computer user has entered on his or her keyboard. In China, all you would need to do is access the keylogger that is part of every personal computer and device in the country—and, technically speaking, every computer and device in the world.

HOW CHINESE COMPUTING WORKS

Computers in China are exactly the same as those in the United States, right down to the QWERTY keyboard. Were you to park yourself at one or another of Beijing’s hip new work-share spaces, you would find the same kind of entrepreneurial millennials hard at work on their QWERTY devices, getting ready for hopeful meetings with venture capitalists.

The QWERTY keyboard in China is not what it seems, however. In Chinese computing, the letters on the QWERTY keyboard are not used in the classic “what-you-type-is-what-you-get” way but as a means of providing instructions to a piece of software known as an Input Method Editor (IME), which uses keyboard instructions to determine the Chinese characters that will then appear on the screen.

Consider how one might use the QWERTY keyboard to input some Chinese terms and phrases relevant to this discussion: “integrity,” “public opinion,” and “data privacy.”

To enter the two-character Chinese word for “integrity” (chengxin 诚信), the four-character term for “public opinion” (yulun daoxiang 舆论导向), or the four-character phrase for “data privacy” (shuju yinsi 数据隐私), the industry-leading IME by Sougou offers up many possible input strategies for the user. One could enter complete phonetic spellings for each passage—in this case, “c-h-e-n-g-x-i-n,” “y-u-l-u-n-d-a-o-x-i-a-n-g,” etc.—or one could use various shortcuts and abbreviations. With just the first letter of each character—“c-x” and “y-l-d-x”—Sougou input is intelligent enough to provide a limited set of candidates. For “c-x,” the Sougou pop-up menu provides “integrity” alongside other two-character Chinese words that also begin with “c” and “x,” such as “plagiarism” (chaoxi 抄袭). For “y-l-d-x,” the IME provides “public opinion” and one other alternative, “Yale University” (yelu daxue 耶鲁大学). Being presented with such options, it is simple enough for the user to select “integrity” over “plagiarism,” and so forth.

Because of the way IMEs function, all Chinese text input on the computer is essentially a form of localized telecommunication. A person in China using Microsoft Word is not sending messages to a third party, of course, but talking to himself. The person using the program is sending out alphabetically coded transmissions into the Input Method Editor, which then processes the code and sends back messages to the user in plaintext Chinese.

CLOUD INPUT

Such input systems have been a mainstay of Chinese computing for more than 60 years, ever since Samuel Caldwell, a professor of engineering at MIT, invented the first experimental Chinese computer. In fact, Chinese input predates computing, even, with Caldwell drawing his inspiration from the mechanical MingKwai Chinese typewriter, invented by best-selling author and linguist Lin Yutang in the 1940s. Following Lin and Caldwell, each generation of Chinese computer scientists and engineers have made input central to their design.

Beginning in the 2000s in particular, Chinese computing has harnessed the ever-increasing processing power of personal computers to speed up the input process dramatically, with all Chinese input relying heavily and intelligently on predictive text, autocompletion, shortcuts, and abbreviations. The QWERTY keyboard in China is thus “smart” compared to the what-you-type-is-what-you-get counterpart in the alphabetic world. All of this has turned Chinese into perhaps the fastest language on earth, in terms of computer input.

In just the past few years, however, something profound happened: Input joined the cloud. Unlike IMEs from the 1980s through the 2000s, where the entire process took place inside the computer, so-called cloud input systems by Sougou, Baidu, QQ, Microsoft, and others have begun to harness enormous Chinese-language text corpora, as well as ever more sophisticated natural language processing algorithms. In 2013, Microsoft researchers touted the growing power of its Chinese IME, and on its website, Sougou has boasted of far greater accuracy and performance for its cloud-based IME. “Long sentence accuracy”—the ability for an IME to convert a long and complex sequence of alphabetic letters into an accurate, multi-character Chinese passage—has grown from 62.5 percent on locally stored IMEs to 84 percent with cloud input, Sougou touted, while “short sentence accuracy” was reported to grow from 91.52 percent to 96 percent.

A map of China is seen through a magnifying glass on a computer screen showing binary digits in Singapore in this January 2, 2014 photo illustration.
A map of China is seen through a magnifying glass on a computer screen showing binary digits in Singapore in this January 2, 2014 photo illustration.
Edgar Su / Reuters

SECURITY QUESTIONS

If many have taken notice of the increased speed and accuracy of cloud input, few if any have pointed to the profound questions of data security these systems raise. If IME programs are, at their core, keyloggers, cloud IMEs are keyloggers connected to the Web. In the context of cloud input, every single key on the QWERTY keyboard might as well read “send,” “post,” or “tweet.”

As billions of keystrokes pulse back and forth across fiber optic cables, they become susceptible to the very kinds of surveillance publics have started to fear in recent years. What is more, unlike English-language computing, where WiFi-augmented suggestions are largely limited to the Google search bar and Web browsing, in China, these text suggestions are becoming a core part of all text input.

One might ask, of course, whether someone could evade cloud input surveillance simply by using a third-party computer. Surely there’s no way to identify who produced a particular keylog without positive identification. But that might not be right.

As part of the Digital Humanities Asia program at the Stanford University Center for Spatial and Textual Analysis (CESTA), my team and I have been analyzing input keylogs in an attempt to understand the various kinds of logics and strategies that go into the input experience.

Although research is ongoing, what we have discovered thus far is the surprising degree to which input keylogs vary, and the way that different computer users seem to possess their own, distinct input strategies—techniques that they have come to rely on when inputting Chinese. The security and privacy implications of even these preliminary findings are profound.

Prior to the age of personal computing, the forensic analysis of typewriters and typewritten documents formed an important element within both foreign and domestic spycraft. Typed documents vary in subtle ways, depending upon the make and model of typewriter with which they are produced. Even different brands of typewriter ribbons leave subtly different “signatures” on documents. In the United States, as well as in the former communist bloc, surveillance outfits exploited these subtle differences to assess the potential authorship or provenance of otherwise attributable documents. (For a popular take on typewriter forensics, readers are invited to watch the 2006 film The Lives of Others.)

Input transcripts possess their own signatures, our preliminary research suggests, making “input forensics” entirely imaginable. At Stanford, we invited research subjects to input an identical Chinese passage—a short Tang dynasty poem by Wang Wei (701–761 ce) called “Parting”—using whatever IME each individual preferred. What we found was striking: even for the opening two stanzas of the poem alone, measuring only ten characters in length, the keylog of each individual diverged almost immediately:

下馬飲君酒,問君何所之?
Translation: “Dismount the horse and have some wine / I ask, to where are you going?”
Input Keylog, Person #1: xiama_yinjunjiu_,wen_jun_hesuozhi2?
Input Keylog, Person #2: xiamayinjunjiu2,wenjunhesuozhi2?

Notes: The underscore (_) indicates that user pressed the space bar, which is used to select the first character candidate shown in the Sogou input pop-up window. The digit 2 indicates that user chose the second Chinese character candidate listed in the Sougou pop-up menu.

As seen in the first example, some individuals entered Chinese characters one at a time: inputting “w-e-n” and then selecting the desired character using the space bar (_). Others entered longer sequences of letters corresponding to a passage containing more than one Chinese character (e.g., “x-i-a-m-a-y-i-n-j-u-n-j-i-u”), relying more heavily upon the IME’s predictive text capabilities.

As passages grew longer, moreover, variations between different computer users grew rapidly. What this suggests is that, when aggregated across the hundreds of thousands (if not millions) of keystrokes the average computer user produces annually, a distinct fingerprint begins to take shape—one that might be at least as revealing as the most advanced identification techniques developed in the era of typewriter forensics. When cross-referenced against geotagging data, Web cookie data, and other data-acquisition methods, moreover, input fingerprints could serve as a powerful forensic technology.

In the age of cloud input, then, it would not necessarily be enough for a computer user simply to switch machines in order to evade surveillance. His or her input fingerprints would likely follow. And everyone who has ever written an e-mail in anger, or drunk-texted, knows that there is a world of difference between a text before and after it is transmitted. But what if that difference effectively disappeared? What if every Word document you wrote could be intercepted as well, even the ones you never saved, including the entire drafting process? What if every deleted passage, every rephrasing, and every revision formed as much a part of the surveilled world as every one of your definitive statements? Could these deleted yet preserved texts ever be used against you?

In his 1956 work The Minority Report, Philip K. Dick gave shape to the chilling concept of “precrime”—of infractions detected (and prosecuted) in advance of their actual occurrence. The rise of cloud input raises questions: If Wei Jingsheng was sentenced to prison largely on account of his 1978 essay “The Fifth Modernization,” and if Liu Xiaobo was incarcerated in part due to his co-authorship of the 2008 manifesto, Charter 08, might not future, born-digital dissidents be detected in advance, treated not unlike terrorists whose plots “foiled” before they happen?

What is more, there is no reason to imagine that these speculations are limited to Chinese input or the Chinese language—far from it. Arguably, it is only a matter of time before IT companies and users worldwide adopt and harness the power of the smart keyboard, the way that China began to decades ago.

As I prepare to press “send” on an e-mail to my editor, then, transmitting my reflections above as a .docx attachment, I am fully aware of the greater-than-zero possibility that my words could be intercepted and read by parties in government and the private sector. .