Ghost in the machine
By Jeremy Grant and Michael Mackenzie
Published: February 18 2010 02:00 | Last updated: February 18 2010 02:00
Not long after lunchtime one day on the New York Stock Exchange three years ago, unusual things started to happen. Hundreds of thousands of "buy" and "sell" messages began flooding in, signalling for orders to be made and simultaneously cancelled.
The volume of messages sent in was so large that the traffic coming into the NYSE from thousands of other trading firms slowed, acting as a drag on the trading of 975 shares on the board.
The case was made public only last month when the disciplinary board of the NYSE fined Credit Suisse for failing adequately to supervise an "algorithm" developed and run by its proprietary trading arm - the desk that trades using the bank's own money rather than clients' funds.
Algorithms have become a common feature of trading, not only in shares but in derivatives such as options and futures. Essentially software programs, they decide when, how and where to trade certain financial instruments without the need for any human intervention. But in the Credit Suisse case the NYSE found that the incoming messages referred to orders that, although previously generated by the algorithm, were never actually sent "due to an unforeseen programming issue".
It was a close call for the NYSE. Asked if the exchange could have been shut down as it was bombarded with false trades, an exchange official says: "If you had multiplied this many times you'd have had a problem on your hands."
Erroneous trades are not new to markets. "Fat finger" errors - mistyped orders, such as the instance this week in which a trader mistakenly bought shares in the Japanese recruitment company known as J-Com rather than JCom, the cable television group - are often blamed for losses. Technological innovation is not new either. The pit trader who had a computer while rivals still used telephones had an early advantagein the 1980s. But many blamed computers for exacerbating the stock market crash of 1987.
Advances in technology have been so great in the past five years that markets are now overwhelmingly driven by machines rather than humans punching orders into a keyboard. The nightmare scenario of an exchange being knocked out by algorithms running amok, and thus causing upheaval in the wider financial system, is seen as a real risk by many in the industry.
A decision to trade can be triggered by a news event; there is even separate technology that "scrubs" news articles to give algorithms a sense of where - "directionally", in the jargon - it may be profitable to start trading in a company's shares. Other types of algorithm seek out where, across a range of exchanges and trading platforms, the best price may be found.
The speed at which such trading takes place is causing alarm. The technology is so sophisticated that thousands of orders can be sent to an exchange's "matching engine", where orders to buy and sell are put together, and a match found, all in less than 300 microseconds - 1,000 times faster than the blink of a human eye.
Joe Ratterman, chairman of BATS Global Markets, which operates equity exchanges in the US, says: "The very nature of speed involves new risks, there is no way around it; it is what it is."
At the same time, markets have come to be dominated by "highfrequency traders" who rely on the perfect marriage of technology and speed. They use algorithms to trade at ultra-fast speeds, seeking to profit from fleeting opportunities presented by minute price changes in markets. According to Tabb Group, a consultancy, algorithmic and high-frequency trading accounts for more than 60 per cent of activity in US equity markets.
The transformational extent to which markets are being moved by machines, and the scale of involvement by high-frequency firms, are raising two concerns. First, has technology reached the point where machines pose systemic risks if they go berserk? Second, if business is now dominated by a few participants that have this technology, does this threaten the integrity of the markets, where a broad mixture of traders has long cohabited peacefully?
In the light of the growth of high-frequency trading, the Securities and Exchange Commission, the US market regulator, has launched a comprehensive review of market structures. In a document published last month, it poses questions such as: "[Does] the high speed and enormous message traffic of automated trading systems threaten the integrity of trading centre operations?"
The Federal Reserve Bank of Chicago, part of America's central banking system, in a paper published this month, says: "The high-frequency trading environment has the potential to generate errors and losses at a speed and magnitude far greater than in a floor or screen-based trading environment. Although algorithmic trading errors have occurred, we likely have not yet seen the full breadth, magnitude and speed with which they can be generated."
John Jacobs of Lime Brokerage, which caters to what it says are "even the most complex, automated and high-volume electronic trading strategies", says "algo" errors have certainly occurred. In a letter to the SEC last June, he warned: "Given the growth and nature of new high-frequency trading participants, the potential for trading-induced multiple domino bankruptcies exists."
In his letter, he identified a number of errors, such as Morgan Stanley submitting a $10.8bn order instead of a $10.8m order in September 2004 and a $31bn order placed by UBS in February 2009, which was 100,000 times larger than intended.
Much of the concern centres on an explosion in the amount of message traffic - electronic signals that contain buy and sell orders - and data generated in the markets, not only by high-frequency traders but by others including asset managers and banks, also big users of algorithms.
Stock exchanges have upgraded their capacity to absorb the rise in message traffic but questions remain over whether they have done enough. In November the London Stock Ex-change's trading system was knocked out for three hours after what it said were "connectivity issues" that had affected two of the "gateways" into the exchange's order books that are used by traders. It has not said whether a sudden surge in messages was the cause, but traders in London suspect so. It is in the process of switching to a new technology platform that will be ready by September.
T he problem is not restricted to stock exchanges. Gerald Hanweck is chief executive of Hanweck Associates, a risk management company bought this month by the International Securities Exchange, a US options exchange owned by Deutsche Börse. He says: "As options trading volume and quote traffic continues to surge, conventional market data systems are struggling to keep pace."
NYSE Euronext, parent of the New York exchange, and other exchanges have "throttling" systems in place that detect and prevent order message traffic exceeding acceptable levels.
Mr Ratterman says: "It's absolutely a market centre's obligation to put in place controls that mitigate risks." BATS has audible and visible alerts that tell staff when its "ports" - gateways through which orders arrive - are experiencing traffic of more than 3,000 messages per second. Throttling starts at 5,000 messages per second. Asked how often that is needed, Mr Ratterman says: "Not very often."
But Doug Rivelli, CEO of Pragma Securities, a trading technology company, echoes the view of many market participants who say high-frequency traders have provided much needed liquidity to the markets, which outweighs any concerns over technology going berserk. "The focus needs to be on making sure there are pre-trade checks in place so that [the industry] can follow and catch errors before they happen," he says.
The difficulty is that responsibility for risk controls does not lie entirely with exchanges and trading platforms. Much of it rests instead with brokers, which increasingly provide access to such venues under an arrangement known as "sponsored access" whereby any trading firm that is not a member of an exchange can "piggyback" on a broker's membership to gain direct access to an exchange. Until recently, before the SEC clamped down on the practice, traders were able to use a form of this process - "naked access" - to gain access to exchanges without brokers conducting pre-trade risk checks to ensure their algorithms were functioning properly.
Brokers are also responsible for "controlling the flow" of orders sent in through a sponsored access arrangement and, where needed, choking off trades. But given that each trader may be connected to a plethora of venues under "co-location" arrangements (see box) - each with slightly different trading infrastructures - doing so is not always easy, according to Valerie Bannert-Thurner at FTEN, a company specialising in pre-trade risk management.
"If you have different systems for all of these, you need to tie them all together - and in a set-up where you have multiple co-location systems it's an integration challenge. There's no standardisation: if all the exchanges had one simple mechanism it would be easy," she says.
Regulators are now faced with the question of how to respond. David Wright, dep uty director-general of the European Commission's internal market and services unit, says it is "a dangerous approach to start out saying you regulate technology". He suggests that regulators should instead pay attention to whether a particular technology "threatens market integrity and fair competition".
Jamil Nazarali, head of electronic trading at US-based broker Knight Capital, proposes that the barriers to entry for high-frequency trading should be raised, saying firms need to have minimal capital requirements. "The market needs better protection in a smart way, so we do not lose the benefits of narrower bid-offer spreads and greater liquidity."
Sang Lee, managing partner at Aite Group, a US-based consultancy, says it might be possible to "impose behavioural restrictions", such as a minimum trading speed requirement, or a requirement that market centres "make an order valid for a certain amount of milliseconds".
He adds: "The core argument for regulating the industry is that some players with deep pockets have a technological advantage. The dilemma is: do we slow down the faster guys or require that the rest of the market speeds up?"
Automated in Amsterdam
'Nobody gets a better deal because of a split second'
Trading on the floor of the Amsterdam Stock Exchange stopped in 2002, when the last of the pit traders were replaced by all-electronic trading on NYSE Euronext, which owns the Dutch bourse, writes Jeremy Grant .
The old traders' jackets, in different colours designating the rival options trading firms that used to be part of the bourse, are now displayed in glass frames on the walls.
But visitors to the vast trading hall, built in 1913, might be surprised to find that the floor has been repopulated with traders. All Options - a Dutch options market-making or "high-frequency" trading firm - took up residence last year, after buying a rival Dutch firm that had been based there since 2007.
For All Options, the exchange floor is as good as, if not better than, any commercially available floor space in a high-rise building. For NYSE Euronext, meanwhile, the rent paid by All Options helps defray the costs of maintaining the building, a historic landmark in the city.
Young traders dressed in T-shirts and jeans sit in front of computer screens arranged on C-shaped desks. There is little noise, save the occasional raised voice when fresh market data flicker on to the screen, prompting a flurry of trading activity.
The return of trading to the pits in pure electronic form - bringing high-speed action as conducted by All Options to a floor once given over to the leisurely business of pit trading - is the latest twist in the evolution of the Dutch options markets, the oldest in Europe.
Perhaps unusually for a high-frequency firm, however,All Options believes that ultra-fast trading has become fast enough. Joost van der Laan, manager of corporate affairs, says the company takes advantage of ways to trade at great speed because the technology is there, and because competition demands that it take part just as its rivals do. But he says: "I think there's no added value for the market in split-second timing. Nobody gets a better deal because of a split second."
Mr van der Laan says the important thing is that "everyone has the same possibilities for latency" - the speed at which trades can be completed.
"I would say we now have a level playing field and we don't need an extra millisecond of speed. Of course there is a need for technology improvements but it's not necessary for the exchanges to make new latency investments. We need to stop the race."
In another sign of the rush for speed in trading, ex-changes are building huge data centres where traders, members and non-members alike, can place computers containing their trading algorithms next to an exchange's matching engine, which matches "buy" and "sell" orders. This "co-location" shaves crucial milliseconds from the time it takes to complete a trade. NYSE Euronext is spending $500m to build two centres, one in New Jersey, another in Essex, England, each larger than two soccer pitches. Specialist companies also operate such co-location centres - including Nasdaq-listed Equinix, which this week expanded in Germany.
Copyright The Financial Times Limited 2010. You may share using our article tools. Please don't cut articles from FT.com and redistribute by email or post to the web.