Dark Pools: The Rise of the Machine Traders and the Rigging of the U.S. Stock Market

'Dark Pools' is a fun, intelligent, beach weekend read.Apart from the obvious Wall Street / HFT focus, the book struck me as a cross between War Games, The Matrix, and Terminator: Rise of the Machines. You immediately enter a world of high tech mayhem, with super-algos, blaster bots, and hunter-sniper cloaking devices duking it out at the speed of light.Re, War Games parallels, you get an iconoclastic, vaguely teenage anti-hero - Josh Levine, the misift-hacker-genius creator of Island - who pursues an idealistic vision of "making markets free," with no impure thoughts of capitalistic gain, until one day his mutated ECN creation all but comes alive and says: "Good morning, Professor Falken. Do you want to play a game?"Re, Matrix parallels, by the end of the book we have fast-forwarded from the humble beginnings of electronic trading to the near birth of AI (artificial intelligence)... the "desert of the real" (in this case the real being markets)... and the grand vision of supervillainesque networked undersea substructures, monitoring global data flows from strategic ocean points all around the globe.Re, Terminator, in the final stretch I kept waiting for Patterson to write: "As future tech historians will note, SkyNet showed signs of self-awareness on X-X date, 2012..."It was all a bit much - but in a good way. As Keynes once said, "Words ought to be a little wild, for they are the assault of thoughts on the unthinking." This book will definitely get you thinking about the impact of high frequency trading on markets.My two cents: At the end of the day, high frequency traders are the new market makers... the superfast replacements for the hand-signaling floor traders and post-sitting NYSE specialists of old. Yours truly predicted as much would happen in a review of "The Predictors" by Thomas Bass, titled "The New Market Makers?" circa 2005 (still available via my Amazon review page).As I wrote nearly seven years ago:"These guys occupy a very specific niche in the market ecosystem. Before the onslaught of computers, human floor traders provided vital liquidity to the markets (and got paid plenty well to do so). As physical exchanges lumber towards extinction, `smart' algorithms are filling the shoes of floor traders, extracting profits tick by tick with high volume, high frequency strategies. These automated players are thus becoming the new liquidity providers and market makers of the 21st century. Daytraders and scalpers may find themselves swept up in a technological arms race, but longer term traders and investors have little to fear... it's a different game."Such is why this review is titled, "Meet the New Market Makers (Same as the Old Market Makers)". In a lot of ways, despite all the technological advancement, the biggest things haven't changed.Take the infamous "Flash Crash" of May 2010, for example. When you understand the role that high frequency traders play these days - in terms of facilitating the majority of volume and liquidity in markets - it makes sense to expect chaos when they all "pull their bids" at once.From the perspective of a freak occurrence where a large portion of the HFT community "backed away," exposing ridiculously far-off placeholder bids that were never meant to be hit, the market makers of the 21st century acted just like the market makers of the last century amidst the crash of 1987. They left a void at a point of severe dislocation, just as the old school guys did so long ago.Perhaps now that GETCO - which stands for Global Electronic Trading Co, the most supervillain-like shop of them all - has assumed official market maker duties in many household names, such bid pulling will not be a future problem.The question remains: Are the new guys worse than the old guys? I'm skeptical.Anyone nostalgic for the old days of physical pits and human specialists may not remember the day-to-day reality of such a system.As an international commodity broker in the late 1990s, I had the privilege of phoning into the pits on behalf of hedge and commercial clients, to yell at some guy named Vinny or Frankie or Sol - inevitably the brother-in-law or cousin of the Refco floor trader who executed our order - to try and get restitution on a criminally bad fill. This kind of thing happened far too often.And as for being an NYSE specialist? Talk about a license to print money. There is a reason such jobs were handed down from one generation to the next. In many cases, the opportunities provided were the legal equivalent of stealing.Not to mention the commissions - good lord, the commissions! - that retail and institutional clients alike were forced to pay in the old days. Add it all up, and I don't think the pennies and nickels hoovered up by the HFT shops, mitigated by the incredibly low-cost commissions available via technology today, amount to such a bad deal.A bit of wildness that made me laugh out loud was the notion that computers are going to take over the markets one day, as in, putting directional investors and traders out of a job. Seriously? Puhleeze. These bots may be great at nano-scalping, playing for blips on a mass scale, but true directional market involvement is another matter entirely.If you're truly worried about thinking machines eating your lunch in a multi-day or multi-week time frame, don't be. Marvin Minsky, a noted forefather in the Artificial Intelligence field - a very confident AI optimist 15 to 20 years ago - recently admitted the following:"The bottom line is that we really haven't progressed too far toward a truly intelligent machine. We have collections of dumb specialists in small domains; the true majesty of general intelligence still awaits our attack.""Dumb specialists in small domains" well describes the proliferation of tick-hungry algos. They are good at what they do in a very tight timeframe, but the inputs required to parse incalculable variables across extended time horizons are another matter entirely. As I wrote some time ago, the most powerful supercomputer on the planet is not smart enough to figure out the turbulence in a glass of water - and yet we expect it to crack the self-referential human feedback loop that is markets?The book closes with a glimpse of the supposed future in "Star," the self-learning, self-teaching virtual machine assigned to make investment decisions for a tiny hedge fund, Rebellion Research. To the extent that Star is supposed to be a threat to humans, color me skeptical. (Those who are just mediocre at their jobs - rather than very good - have much to feel threatened by, of course... but such has always been thus.)Patterson makes brief allusion to computer-assisted chess, the powerful combination of hardware and wetware (software programs plus human guidance). For cream of the crop money managers, I think this is closer to the true way forward - using technology to enhance human capability, not replace it.In terms of thinking games, the best metaphor / AI-intelligence test is perhaps not the fixed Western game, Chess, but the fluid Chinese game, Go, which contains far too many variables within the scope of possible movements for any computer, even Deep Blue, to brute-force calculate the optimal strategy path. (For this reason, no Go champion has ever fallen to a machine.)At the end of the day, trading and investing strategies will continue to evolve. High frequency traders, and various forms of new technology, will continue to influence markets in unexpected and interesting ways.But I believe the following words from "Reminiscences of a Stock Operator," which were true in 1923, will remain just as true a century hence: "There are men whose gait is far quicker than the mob's. They are bound to lead - no matter how much the mob changes."

Frankly I loved the information and history on high frequency trading, but it did feel a bit empty in the end. The beginning and end of the book jumps around so frequently from person to person it’s difficult to know how he intends to bring it all together… I don’t think he truly did.Section two and three have the most focused information and history that will captivate the reader.A few times in the book the author would drop the word “dark pool,” mentioning large investors moving there to avoid hft’s, but that’s all the info you really get regarding dark pools. He even states that many of the algorithms evolved to function in dark pools.My guess is the author just liked the effect of the word “dark pool” and chose it for the title of his book.In summary, this book has nothing to do with dark pools and instead is mostly about how the exchanges, markets and market makers evolved into using computer based algorithms to provide liquidity to the markets and thus profit more efficiently… Up to 2012.But I did learn a valuable lesson. Don’t judge a book by its cover.

Scott Patterson isn't Michael Lewis. Still, he manages to hold one's attention while providing a selective history and a basic lesson in the mechanics of stock trading circa 2012. The story is choppy -- Patterson had only a limited number of sources and he pieced together much from publicly available information. Still, he provides an enlightening lesson on the dramatic changes in the functioning of the equity markets over the past decade. Investors of course can't help but be aware that stock trading these days is different than in past decades. To place a stock trade one uses an electronic platform like TD Ameritrade instead of phoning a broker. What investors may not realize is that the fundamental structure of equity markets -- the market making system (market makers on the NYSE were called "specialists") has all but disappeared, having been replaced by high frequency traders (HFTs). These largely unregulated trading firms serve the function of market makers but they have no obligation to stick around when the going gets rough. Patterson explains how the system evolved. It had a haphazard evolution that nobody planned. Amazingly, it sort of works. Except when it doesn't (as in the flash crash or the Knight debacle). The system is still developing and over time it will get better. At the moment, it seems cobbled together and wobbly, a scary mess really. It is (surprise!) designed to benefit the people who built it and not investors. Most importantly, with no limits on speed (the SEC could have easily created a level playing field by imposing a latency of say 10 msec) those trading firms who have their computers closer to the exchange computers ("colocation" is best) have an advantage. This is about as unfair as one can imagine. At the end of the day, there isn't any evidence that the current system serves investors any better than old system. Yes, spreads on average are lower, but increased volatility and difficulty executing larger size trades has probably caused trading costs to increase overall. Nobody knows how much profit HFT firms are extracting from the public markets, but the fact that they are spending hundreds of millions on technology suggests that it is considerable. When creative destruction occurs, some things become better and others don't. Clearly, the regulators have been asleep at the wheel while the cataclysmic changes in the stock market have occurred. Huge regulatory mistakes have been made because regulators have failed to be proactive and have jumped in only when thing got truly out of hand -- too little, too late. Patterson explains the basic details in the context of engaging stories about some of the architects and players of HFT. Okay, finance can be boring so Patterson does his best to spice things up it up with lots of action, that gets tedious after the umpty-umpth time ("hundreds of thousands of orders flying into the market each second through high-speed connections, fighting to be in front of all the others. Just as quickly, as stocks bobbed and weaved, . . ." and ". . . trades fizzing through fiber-optic cables laced around the world" it is time to to want to scream "enough!"). Still, I couldn't put the book down.

By the end of the twentieth century corruption was widespread among the ranks of Wall Street stock brokers, but there was a powerful force of change on the horizon: technology. In Dark Pools, author Scott Patterson shares the story of how computers would come to upend the old guard of the stock exchanges and transform the markets in fantastic, and often unpredictable, ways.Wall Street: The place where our 401ks and pension funds go, which we should all probably understand better than we actually do. Despite what could be called, at best, amateur interests in the world of trading, I couldn't put this book down. While by no means a deep technical work, Dark Pools still proved highly educational, offering a lot of insight into the inner workings of today's stock market. Fortunately, Patterson presents the material in a way that's eminently accessible, even for those without a Wall Street Journal subscription. Discovering not just how the market is, but how it came into its current form, was eye opening.Dark Pools isn't just about the markets, however. It's also a testament to the raw power of computing and its ability to change the way we work in just a few years time. I expected the book to cover some hyper-modern trends like high frequency trading. What I didn't anticipate was a beautiful and inspirational story about a few gifted individuals employing computers to revolutionize the world of finance. As it turns out, one 20-something young man working out of a tiny office near Wall Street actually built large portions of the high tech stock market we see around us today. Ever the idealist, he sought to beat the hordes of greedy and self-serving market makers at their own game, using little more than a closet full of computers and his brain. And in many ways, he succeeded. But the story doesn't end there; this new technology would eventually take on a life of its own, growing and evolving at fantastic pace. Millions and billions would be made and lost in seconds. And it's still going on, all around us. Some say it's out of control.I thoroughly recommend Dark Pools to anyone with even a passing interest in the stock market or the development of technology. It's an eye-opener on the state of our national finances on one hand, and a testament to the power of man and machine on the other.

This book does 2 things:(1) goes through the historical events that led to our modern stock market and(2) tries to explain what is going on in market microstructure.The book does very well on (1) -- it's really an enjoyable book to read. But on (2) there's almost no information, no analytic discussion about microstructure, and some of the comparisons and conclusions are simply wrong. For example, the Patterson compares the immediate liquidity with the days of the past comparing the top of the book orders and noticing that today you have fewer orders at the top of the book. But this is misleading. In the past when the spread was at least 25 cents (due to regulation, ie, in increments on 1/8ths, but also in part of the gentleman's agreement between specialists to keep the spreads large) and now the spread is mostly 1 cent for most stocks. To compare apples to apples you need to sum up the liquidity on 25 levels in the current market, because basically these 25 levels would have been aggregated into 1 level in the past. Once you do this comparison, it's clear that the order book today has much more immediate liquidity.In spite of the title, another thing that's missing in the book is a discussion of dark pools. Obviously, there's a much bigger problem with dark pools today than with the lit market, mainly because the dark pools can legally do prop trading (and they do) on the flow they see, but in the same time they are marketing a hidden market. This is simply wrong and should have been discussed.The author points out that the market is unfair, in the sense knowledge of market micro-structure gives some players an advantage. But the market was never fair. For example, players with knowledge about a stock have an advantage, even if that knowledge is public. The same with micro-structure. The rules of the micro-structure are published. Anybody can read the order types for the exchanges and the information is public. In the past it wasn't the case (specialists had privileges) In today's market the differentiation is more about technical competence and less about specialists born with entitlements. Moreover, the transaction costs for investors are 1-2 orders of magnitude lower than during the time of the specialists. (I've never seen a single study that claims otherwise) In other words, it'd be unfair to draw Kasparov in a chess tournament, but at least you start with the same pieces (which was not the case during the rule of the specialists)The author also doesn't compare the state of the stock market with our other markets: futures (futures is much cleaner), FX (dirty business, where banks have a separate inter market with tighter spreads for themselves, plus the broken will do prop trades against you) and OTC contracts, say corporate bonds (huge 1% spreads, little liquidity, the bank will own you). In other words the stock market is the most fair, except maybe for futures (1 exchange, no fragmentation, no complicated rules, latency less of an issue), and way more fair than FX and OTC. The few big banks that own the OTC business have a lot to gain from HFT bashing, as they really don't want their business on a lit exchange.I only give 5 stars because it's an enjoyable book to read and does a good job at (1). Patterson is an outsider, but he did a good job at (1) that it reads like a novel. There are a few competing books about HFT for the average reader and all of them are worse than Pattersen's book:1. "All About High-Frequency" (Durbin) - written by an insider, does reveal a little about market microstructure, but a very introductory book. Readable by anybody.2. "Speed Traders" (Perez) - a book which was quickly assembled as a series of interviews, and Mr Perez is using the book to make a name for himself as a consultant for HFT. (his name is constantly on PR wire and he organizes all sorts of events)3. "Broken Markets" (Arnuk, Saluzzi) - full of factual lies, written by old school specialists that use smarts to trade, which are bitter for losing business with the modernization of the exchanges, and want the fat 25 cents spreads back.For the technically inclined that want to learn about market microstructure, there are a few more quantitative book, but they are all dry and not much fun to read.

After reading all the way thru this book, I can't really say I know what the author was trying to accomplish......he leaves us at a point in time without telling what he thinks, how he feels. Which bleeds some of the emotional underpinnings out of the stories. Is the book a clarion call to fairer markets? I guess that is the main point, but at some late part of the book the focus shifted from HFT to AI, and that central message got muddled. This book has a lot of information, perhaps too much. I feel it is strongest when the author tells a story with a human interest angle (though some of the 'protagonists' seem pretty inhuman). A few times too much detail fogged things up. If someone wants an in-depth history lesson on this subject I recommend Dark Pools unreservedly. Lastly, to read a chronicle of so many incredibly intelligent individuals doing nothing except try to make money from other people's money is just fundamentally depressing. To read this book is to understand that Finance has become a magnet for many of our best and brightest, and begs the question what does it say about our society that we provide more incentive in this arena that produces very little of value for us, humanity, instead of fields like science, medicine and education?

I read "Dark Pools" andFlash Boys by Michael Lewis. I was going to buy only Flash boys, but bought this book also on the high recommendations. If you are going to read one book on the subject of the impact of HFT, I suggest you pass this one, and read Flash Boys. I really found "Dark Pools" to be much less readable for two reasons. Firstly, his writing style is atrocious, it is full of hyperbole and a type of purple prose more suited to a cheap detective novel than something which requires close intelligent reading. Secondly, I don't trust his specifics, due to the fact that when he discusses things I understand intimately (server and network technology) he often gets things wrong (distributed computing is based on many redundant systems, not many redundant disks). so I wondered what else was inaccurate in areas I did not understand. I much preferred Flash Boys. However, if you are interested in the subject, and planning on reading many books on the topic, I would say that this book gives a broader history of the evolution of modern markets, introduces you to many key players and gives some useful explanations. I just wondered if this book had been fact checked at all.

High frequency trading has been in the financial press a fair amount in the last few years. There was the May 6, 2010 "flash crash" and the demise of Knight Capital as a result of a software error (were Knight lost $440 million in 40 minutes). Scott Patterson's excellent book Dark Pools provides a very engaging background on the evolution of the computer trading networks and the risk of HFT.Even for someone with a background in finance, Dark Pools is a fascinating account. Scott Patterson manages to paint interesting portraits of the early pioneers of computer network trading (the so called SOES bandits) and the evolution of the trading networks. There's a very interesting portrait of Josh Levine, who wrote the software for the Island trading network. I have a book of essays by Haim Bodek on HFT. However, I didn't know his history in HFT until I read about it in Dark Pools.I was also impressed that Patterson got people at Renaissance Technology to talk to him at all, although he doesn't have a much better picture of how Renaissance trades than anyone else.The only fault that I can find with Dark Pools is that Scott Patterson does not understand software. He has an annoying habit of referring to any complex trading software as Artificial Intelligence (AI) software. Back in the 1960s AI got a bad reputation so very few people refer to complex decision making algorithms as AI. And in many cases the software that Patterson refers to would not be classified as under the old definition AI (although some of the learning algorithms might be).But software is not the strong point of this book: computer trading networks are and Patterson does a good job describing these and how HFT evolved.

Livre faisant une description historique, un peu romancée du développement de la digitalisation des places boursières et de l'émergence de nouveaux marchés financiers. Intéressant de voir à quel point des idéologies démocratiques peuvent être détournées pour construire des intruments de rente et d'influence. Inquiétant de voir la complaisance de certaines autorités vis-à-vis de structures complètement opaques et anti-compétitives.

I am engaged in the quantitative trading and fund management business, and can relate to a lot of this stuff. Absolute your de force

Muy buena, buena compra

Really insightful

In "Dark Pools" geht es nur am Rande um Dark Pools im engeren Sinn. Patterson benutzt den Begriff statt dessen als Sinnbild für die elektronischen Börsen als ganzes, die aufgrund der Komplexität und schieren Masse von Daten - inklusive versteckten und gecancelten Orders und dem ständigen Kampf immer neuer Algorithmen untereinander - nicht mehr durchschaubar oder kontrollierbar seien.Ähnlich wie schon in "The Quants" trägt Patterson im Stile des Boulevardjournalismus erstmal dick auf: Aus Lautsprechern tönt HipHop oder Metal während "ultra-competitive", "testosterone-fueled", "elite of the elite", superreiche junge Computergenies mit "dazzlingly complex algorithms" und "high-octane trading robots" die Märkte auf Kosten von "mom and pop investors" manipulieren.Der Kern des Buches besteht jedoch vor allem aus der faszinierenden und spannend erzählten Geschichte der Computerisierung des US-Börsenhandels, angefangen mit den SOES-Bandits nach 1987, wobei die Entstehung des Island ECNs in den 1990ern, und dessen idealistischer Schöpfer, der öffentlichkeitsscheue Programmierer Josh Levine, im Mittelpunkt stehen. Das wurde (so) noch nie erzählt, und dafür verdient das Buch, trotz einiger schwächen 5 Sterne.Am Ende werden dann noch bekanntere Themen, wie etwa der Flash-Crash vom 6. Mai 2010 oder langfristiger orientierte automatisierte Handelssysteme besprochen.