You’ve heard the story many times. The stock market is rigged. A highly secretive group of opaque financial institutions is making billions of dollars from socially useless high-frequency trading—placing and withdrawing stock orders hundreds of thousands of times per second—with all those profits coming, in one way or another, from the rest of us. The biggest losers of all? Small, mom-and-pop, or retail, investors, who cannot hope to compete.

Perhaps the best-known proponent of this narrative is the author and financial journalist Michael Lewis. In his 2014 book, Flash Boys, Lewis painted the stock market as a battle in which the good guys were losing to the bad guys. The book sold well and even instigated a handful of criminal investigations into high-frequency traders (HFTs), none of which bore any visible fruit. For the truth is that even with the rise of high-frequency trading since the early years of this century, actual mom-and-pop investors have never had it so good. Armed with online accounts offering trades for minuscule fees, they see their transactions go through instantaneously, without the sorts of delays that can allow the market to move against them before their order is filled. If the stock market is broken, it’s not broken in a way that is obvious to retail investors.

Yet Lewis was right to worry about HFTs; he just misidentified their main victims. This is the revelation at the heart of Walter Mattli’s masterful Darkness by Design. Great books make you reexamine your assumptions, and this one delivers in spades. It not only offers a compelling critique of how the stock market has evolved over the past 15 years; it also forces readers to reconsider the idea that competition is good and monopolies are bad. What has truly tilted the playing field in favor of a handful of financial behemoths and HFTs, Mattli argues, is the growing fragmentation of stock markets, a process actively encouraged by misguided government regulators. The biggest losers of that development are not retail investors, who tend to be fairly well-off, but pension funds, insurance companies, and other major institutional investors.

Those financial behemoths are, in fact, the proverbial little guy. One of the paradoxes of financial terminology is that terms such as “retail investor” and “small business owner” connote the relatively impecunious, whereas in fact those investors and owners are disproportionately likely to be in the top one percent of the wealth distribution. The big investors—pension funds, insurance companies, mutual funds, and exchange-traded funds—are much more likely to be holding the wealth of the 99 percent. Ordinary investors are being ripped off every day; they just don’t see it, because it is happening behind the scenes of their life insurance policies and their index-fund investments.

Outside the NYSE, October 2019
Brendan McDermid / Reuters


Mattli is a political scientist, and his great insight is to consider the stock market more as a political entity than an economic one. To Mattli, markets are first and foremost “political institutions governed by power relations.” Different members have differing preferences when it comes to market structure and rules. When those members have similar amounts of power, the result is often a democratic compromise in which the greater good prevails. But when financial institutions garner for themselves an outsize degree of power and influence, they can end up skewing the market structure in their favor, at the expense of ideals such as liquidity and trustworthiness. That, in a nutshell, is what has happened in the global stock market—with the largest banks and brokerage companies refashioning markets to serve their own ends.

Mattli’s book is the result of years of research into the history of the New York Stock Exchange and its member companies. Granular detail about market regulation might not sound like the stuff of a great read. But Mattli spent a lot of time in the NYSE’s archives and interviewed many of its former employees and traders. As a result, Darkness by Design has an uncommon richness to it.

Stock exchanges have abdicated even the pretense of having a governance structure with any teeth.

Take a story that neatly illuminates how much has changed for the worse under today’s regulatory regime. Bob Seijas, a 33-year employee of the NYSE, told Mattli about a coworker who in the 1980s was fined $50,000 (well above $100,000 in today’s dollars) because he left his post to go to the men’s room for eight minutes and gave inadequate instructions to his assistant. The man in question worked as a specialist—an employee at the exchange who serves as an intermediary between buyers and sellers. Part of his job was to buy into selling pressure—buying stocks even as their prices were falling so as to ensure smoothly continuous trading. But when the specialist went to the bathroom, his assistant didn’t keep buying, and the price of the stock he was charged with overseeing fell sharply, by 75 cents. Seijas later defended the specialist, saying that the man had spent four hours performing superbly before taking a bathroom break. A colleague simply retorted, “Don’t tell me he stopped at 20 red lights and only passed one.”

Indeed, the specialist himself likely expected a penalty and understood that if negligence went unpunished, the consequences for his chosen vocation would be much worse than a one-off $50,000 hit. From the 1980s all the way to the early years of this century, any such breach of protocol was almost certain to be punished, reinforcing the trust that all participants had in the market.

Specialists played a central role in maintaining that trust. They understood trading patterns, knew who the big buyers and sellers were, and knew how best to match the two without affecting prices. They made money, but they did so transparently, surrounded by traders who watched their every move. Attempts to front-run the market—buying or selling ahead of a client’s pending order to pad one’s own profits at the expense of the client—were almost always detected. The result was a market in which, as Mattli writes, almost everybody was “socialized into the value system of the Exchange” and had strong financial and reputational incentives to live up to those values.


Those days are over. When the NYSE was a monopoly, before 2005, a single rogue specialist could destroy the reputation of the entire franchise, and so the exchange was always working to improve its governance standards. But the NYSE is no longer the only game in town. Today, there are 23 different registered “national securities exchanges” in the United States, of which the NYSE is merely the second largest, accounting for about 12 percent of the total U.S. market. It competes directly with exchanges bearing names such as MIAX, Cboe BYX, and Nasdaq MRX (not to be confused with Nasdaq BX, Nasdaq GEMX, Nasdaq ISE, or Nasdaq PHLX—none of which is the main Nasdaq exchange that ordinary investors know about). And because it has to compete, the NYSE has gone from a powerful norm setter and regulator in its own right to just another market participant, trying to bolster its position at any cost. Today, stock prices move up or down by 75 cents almost every minute of every day, and the NYSE has neither the ability nor the inclination to stop that from happening.

David Plunkert

“In the new era of fragmented markets,” Mattli writes, “costly investments in good governance and commitments to fairness, equality, and transparency have to be balanced against an overriding new mandate to attract liquidity to survive.” Exchanges do everything they can to attract the business of the major players, who do millions of trades per second, often accommodating them in ways that benefit those players at the expense of other participants in the market. Although no playing field is entirely level, today the market is much more tilted toward a handful of ultra-sophisticated traders than it ever was during the days of the NYSE’s monopoly.

The state bodies monitoring the exchanges suffer from the same lack of cohesion, with predictable results: when an economic sector is governed by multiple regulators, actors will constantly engage in regulatory arbitrage, rewarding the most lenient regulators while diverting their activities away from the most stringent. Before the 2008 financial crisis, for instance, two U.S. bank regulators—the Office of Thrift Supervision and the Office of the Comptroller of the Currency—competed with each other to attract banks, which could choose which agency’s regulation to submit to. That never made much sense, and lawmakers merged the two as part of the 2010 Dodd-Frank Wall Street Reform and Consumer Protection Act. But to this day, the Securities and Exchange Commission (SEC) and the Commodity Futures Trading Commission (CFTC) compete with each other to regulate markets. (Blame congressional politics: the CFTC is governed by the House and Senate Agriculture Committees, whereas the SEC is governed by the House Financial Services Committee and the Senate Banking Committee.)

In earlier days, the concentration of market power at the NYSE made up for this regulatory confusion. When it came to stock trading, the exchange proved a much more capable regulator than the SEC or any other federal agency ever did. The NYSE enforcement arm had deep institutional knowledge. It knew, for instance, that if a broker placed a trade in IBM stock at 12:04:45 PM, he would need at least 22 seconds to walk over to a different specialist to place a different trade. The NYSE used this kind of information to conduct forensic examinations of suspicious transactions, examinations that the SEC would find completely impossible to perform.

Today, however, the regulators are on their own; the individual exchanges have all but abdicated even the pretense of having a governance structure with any teeth. And as Mattli points out, “The creation of exploitative schemes by particularly powerful actors to benefit themselves is rational in a system of bad governance because the chances of getting caught are tiny and the reputational or material consequences of such behavior are largely insignificant while the profits from such schemes are high.”


What caused this enormous change? The short answer is the Regulation National Market System, or Reg NMS, a rule promulgated in 2005 by the SEC in the name of market efficiency. It ostensibly modernized markets by moving stock trading away from the NYSE and toward numerous other exchanges, but it also marked the death of the old NYSE. Up until that point, the exchange was a mutual society: firms could buy seats, and the exchange was owned by its members. After 2005, it demutualized, stopped selling seats, and became just one among many exchanges, most of which were owned and operated by enormous global broker-dealers—think Credit Suisse, Goldman Sachs, and Merrill Lynch—that had spent limitless hours and dollars on lobbying the SEC to push Reg NMS through. Rather than being a utility owned by its members, the NYSE was now a profit-maximizing entity like all the other exchanges.

High-frequency traders are the embodiment of socially useless financial activity.

On top of there being competition among the many new exchanges, every major broker-dealer also engages in “internalization”—effectively acting as its own mini-exchange and fulfilling orders with its own inventory of shares rather than sending them on to any exchange at all. Not so long ago, if you phoned up a broker and placed an order to buy 100 shares of IBM, that order would likely be filled on the NYSE. Today, HFTs compete with one another to pay your broker for the privilege of taking the other side of your trade. This fragmentation benefits HFTs, who are constantly searching for order flows that they can keep for themselves rather than having to compete for them on an open market. It also helps the major global securities firms that orchestrated the end of the NYSE monopoly in the first place, since they are paid for—or take direct advantage of—the retail order flow that they generate. Between them, these huge companies now have a market share north of 70 percent.

The big test of any stock market is whether it has depth: whether it’s possible to buy or sell a large number of shares in a small amount of time without moving the market. Traders will naturally flock to such a market, creating even more depth—a virtuous cycle that results in monopolies, such as the one the NYSE enjoyed until 2005. The NYSE’s monopoly, in turn, allowed it to be technically innovative, introducing everything from the first stock ticker (1867) and the first trading-floor telephones (1878) to a system capable of processing four billion shares a day (1999). No other stock exchange in the world could come close.

Today’s internalization, by contrast, has created a classic tragedy of the commons: big banks free-ride on the NYSE’s ticker, trading at the prices it publishes in real time, without contributing to its liquidity. The consequences became clear during the “flash crash” of May 2010, when billions of dollars of value suddenly evaporated, only to reappear minutes later. Without the deep liquidity and oversight of the old NYSE there was no one to prevent thousands of stocks from collectively plunging and then rebounding. Worse still, that kind of event happens every day in individual stocks; the only unusual thing about the flash crash was that it took place in many stocks simultaneously.

An ornament on the New York Stock Exchange's official Christmas tree, December 2008
An ornament on the New York Stock Exchange's official Christmas tree, December 2008
Mike Segar / Reuters

As the flash crash proved, today’s market lacks depth. Large investors want to move billions of dollars in and out of the stock market but cannot do so without prices moving against them, their orders being front-run by HFTs. The HFTs who benefit from this system are the embodiment of what Adair Turner, then chair of the United Kingdom’s Financial Services Authority, famously characterized as “socially useless” financial activity. They reinvest their profits into machines that can trade in microseconds rather than milliseconds; those profits would surely serve a higher purpose if they were invested in other parts of the economy. And as these outfits become bigger and more sophisticated, they trade increasingly complex financial products—all invented by banks—across dozens of markets and jurisdictions. No regulator can hope to keep up, so these highly secretive companies effectively operate with no code, no morals, and no values. Their only motivation is profit.

Investors once hoped that so-called dark pools would offer a way out of the depth problem. Dark pools exploded in popularity after 2005, since large institutions could no longer count on the NYSE’s specialists to provide ample liquidity and found themselves being outpaced by HFTs on smaller exchanges. Because orders placed in dark pools are not visible to other traders until they have been executed, the hope was that HFTs would not be able to make money front-running these transactions. In reality, however, even dark pools are infested with HFTs, whose trade volume the pools rely on to remain profitable.


The HFTs are in control of the markets now. They are the must-have customers for any exchange, because they drive most of the volume and liquidity in the market. The exchanges, many of them created to serve the HFTs, cannot themselves prevent the latter’s dominance. Nor can regulators, who are confined to single markets in single countries, whereas HFTs roam globally. By the time a regulator has found a vaguely suspicious transaction, the algorithms HFTs use have long since moved on to something new.

Even when blatantly illegal activity happens right under their noses, regulators generally ignore it. From 2006 to 2010, the NYSE gave HFTs a physical trading-speed advantage by openly allowing them to place their trading computers right inside the exchange.

Even when illegal activity happens right under their noses, regulators generally ignore it.

This practice was, as Mattli notes, a patent violation of securities law. But instead of punishing the NYSE, the regulators simply waited for the exchange to ask permission, which eventually it did. Then the SEC granted that permission. Other cases involve special order types, or SOTs—extremely arcane forms of placing a trade, designed to give HFTs an extra advantage over real-money investors. On rare occasions, the SEC has levied fines on exchanges for implementing SOTs without permission, but the fines are tiny compared with the profits the SOTs generate.

Mattli has a whole chapter on various forms of market manipulation that are unequivocally harmful but ubiquitous. There are the ways that banks have allowed HFTs into dark pools even after promising large investors that they would not, for instance. There is quote stuffing—placing millions of essentially fake orders for stocks, at prices far enough removed from the market price that the orders won’t ever be filled—which makes it impossible to see how much liquidity there is in any given security. That happens 125 times per day, on average, across 75 percent of all U.S.-listed equities. And there is spoofing—investors placing and then immediately withdrawing orders near the market price that they never actually intended to see through—which also happens every day in every major stock.

The nefarious activity is clear to all, as is the lack of any real enforcement. The regulators are not only captured by the big banks; they are also completely out of their depth technologically. By some counts, the Financial Industry Regulatory Authority, a private regulator overseen by the SEC, monitors 50 billion market events per day. Its computers flag about one percent of those—500 million events per day—and a single flag can create weeks of work for a team of regulatory investigators. The vast majority of suspicious transactions likely go uninvestigated.

A couple of glimmers of hope remain. The European Union has made decent strides in improving investor protections with a 2018 directive called MiFID II, a new version of the Markets in Financial Instruments Directive, which forces exchanges to be much more transparent about conflicts of interest in their disclosures to investors. In 2012, France implemented a 0.1 percent tax on the value of canceled or modified orders, which is a strong disincentive to engage in quote stuffing or spoofing. And there are even occasional discussions, so far confined largely to academia, about moving to so-called discontinuous markets, where stocks would be allowed to trade a mere ten times per second—slow enough that HFTs could not front-run orders.

Ironically, the greatest hope of all may be that the technological arms race between HFTs and exchanges will become so astronomically expensive that it will force the world’s biggest exchanges into megamergers with one another, resulting in a new global monopoly spanning countries and markets. The idea of a single trading venue for stocks, bonds, currencies, and derivatives, operating 24 hours a day, oblivious not only to regulators but also to time zones, admittedly sounds terrifyingly dystopian. But the lesson of Mattli’s book is that sometimes giants can be relatively benign. It is when they break apart that chaos results.

You are reading a free article.

Subscribe to Foreign Affairs to get unlimited access.

  • Paywall-free reading of new articles and a century of archives
  • Unlock access to iOS/Android apps to save editions for offline reading
  • Six issues a year in print, online, and audio editions
Subscribe Now