How Renaissance Technologies won

Jim Simons is legendary. Renaissance Technologies, the hedge fund he founded, is widely considered one of the most successful in history. Between 1988 and 2018, the firm’s Medallion Fund earned over $100 billion in gains and had average annual returns of 66%. “No one in the investment world comes close,” Gregory Zuckerman writes in The Man Who Solved the Market. “Warren Buffett, George Soros, Peter Lynch, Steve Cohen, and Ray Dalio all fall short.” RenTec rakes in $7 billion every year in trading gains with just three hundred or so employees; for comparison, Citadel makes about those gains with nearly ten times as many employees.

The Man Who Solved the Market is a competent history of the man and his fund. It digs the little you can about the secretive fund’s methods, and quite a bit about the motivations and personalities of Simons and other top traders in the firm. This post collects some of the ways RenTec was different from its competitors.

Culture

Openness

Jim Simons worked for several years as a code breaker for the National Security Agency, in a wing called the Institute for Defense Analysis. RenTec’s culture of stony secrecy on the outside, and open sharing of knowledge on the inside, is almost identical to the environment at the IDA.

The word the government used to describe how it classified the IDA’s work was, itself, classified. Internally, however, there were no silos. Researchers would wander into one another’s offices to offer assistance or lend an ear. Most of the twenty-five or so employees — all mathematicians and engineers — were given the same title: technical staff member (a name borrowed from Bell Labs and now adopted by AI companies). (In a similarly equalizing move at RenTec, even big-name recruits had to pass a coding test, which sent a message that everyone was expected to program computers and do tasks deemed menial at other firms.) The IDA team routinely shared credit and met for champagne toasts after discovering solutions to particularly thorny problems. When staffers met each day for afternoon tea, they competed at chess and Go, and worked on puzzles.

At RenTec, Simons insisted that Medallion have a single, monolithic trading system. All staffers had full access to each line of the source code underpinning their moneymaking algorithms. The hope was that researchers would swap ideas, rather than embrace private projects. For a while, even the firm’s secretaries had access to the source code — though that ultimately proved unwieldy.

Simons also used compensation to get staffers focused on the firm’s overall success. From how Zuckerman describes it, staffers had a lot of leeway in work. They were rewarded for their work with bonus points, each of which represented a percentage of Renaissance’s profit pool and was based on clear, understood formulas. “You know your formula from the beginning of the year. It’s the same as everyone else’s with just a couple of different coefficients, depending on your position,” said a top manager of Renaissance’s infrastructure. “You want a bigger bonus? Help the fund get higher returns in whatever way you can: discover a predictive source, fix a bug, make the code run faster, get coffee for the woman down the hall with a great idea, whatever … bonuses depend on how well the fund performs, not if your boss liked your tie.”

This culture of openness did leave RenTec open to defectors who could run away with alpha-generating intellectual property. (And indeed, a high profile defection to Millennium did happen in 2003, though RenTec avoided losing any meaningful edge.) To guard against defections, RenTec vested bonuses over several years. And RenTec’s location in middle-of-nowhere East Setauket, Long Island, helped: as Ben Gilbert puts it in the Acquired podcast episode on RenTec, “Since Renaissance doesn’t recruit from finance jobs, it’s unlikely that you know someone else in finance. You came out of a science-related field. You now work in East Setauket, Long Island, which has 10,000 people or something or less that live there. You’re in this little town, you’re not actually going into the city that often, and if you are, it’s (again) not to grab drinks with other finance people. Even if you didn’t have a many-page non-compete and a lifetime NDA, you’re very unlikely to be in the social circles.”

Selecting for raw smarts

RenTec also borrowed a leaf from the IDA when it came to hiring. The IDA’s staff members, most of whom had doctorates, were hired for their brainpower, creativity, and ambition, rather than for any specific expertise or background. The expectation was that researchers would find problems to work on and be clever enough to solve them.

In a similar vein, RenTec’s Nick Patterson filtered for “supersmart” PhD grads with dissertations he deemed strong, ideally in fields lending themselves to the work Renaissance was doing. The interview process Zuckerman describes sounds fairly standard for quant researchers: discuss your achievements, and tackle some challenging problems involving probability theory and other areas. Candidates usually were grilled by a half dozen staffers for forty-five minutes each and then were asked to present lectures about their scientific research to the entire firm.

Patterson steered clear of Wall Streeters. “We can teach you about money,” he said. “We can’t teach you about smart.” David Shaw, another quantitative hedge fund owner, similarly prioritized smarts. He hired math and science PhDs, but English and philosophy majors were also among his favorite hires. Shaw also hired a chess master, stand-up comedians, published writers, an Olympic-level fencer, a trombone player, and a demolitions specialist.

As a fun sidebar, Simons got some practice identifying talent by investing in technology companies. In fact, his hedge fund’s name change in 1982 from Monemetrics to Renaissance Technologies reflected his developing interest in these upstart companies as Simons came to see himself as a venture capitalist as much as a trader.

There is a fun anecdote in the book that speaks to the persuasiveness and creativity of Peter Brown, who moved to Renaissance from IBM and later became the hedge fund’s CEO. It was he who gave the computer that beat Garry Kasparov in 1997 its name, Deep Blue. At IBM, he learned of a team of computer scientists, led by a former Carnegie Mellon classmate, that was programming a computer to play chess. He set out to convince IBM to hire the team.

One winter day, while Brown was in an IBM men’s room, he got to talking with Abe Peled, a senior IBM research executive, about the exorbitant cost of the upcoming Super Bowl’s television commercials. Brown said he had a way to get the company exposure at a much lower cost — hire the Carnegie Mellon team and reap the resulting publicity when their machine beat a world champion in chess. The team members also might be able to assist IBM’s research, Brown argued.

The IBM brass loved the idea and hired the team, which brought its program (called Deep Thought then) along. As the machine won matches and attracted attention, however, complaints emerged. It turned out that the chess machine’s name made people think of a famed 1972 porno called Deep Throat, a movie at the forefront of the Golden Age of Porn. IBM knew it had a real problem the day the wife of a member of the chess team, who taught at a Catholic college, spoke with the college’s president, an elderly nun, and the sister kept referring to the amazing “Deep Throat” program.

IBM ran a contest to rename the chess machine, choosing Brown’s own submission, Deep Blue, a nod to IBM’s longtime nickname, Big Blue.

Love for the game

It is interesting to consider how the world around Simons reacted when he decided to pursue trading.

Simons was head of the math department at Stony Brook University when he decided to leave the world of academia for finance. He was met with near universal discouragement. His father thought he was making a mistake giving up a tenured position. Mathematicians were even more shocked. “Mathematicians generally have a complicated relationship with money,” Zuckerman writes. “They appreciate the value of wealth, but many see the pursuit of lucre as a lowly distraction from their noble calling.” René Carmona, who was teaching at Cornell at the time, said, “We looked down on him, like he had been corrupted and had sold his soul to the devil.”

This history was so significant that it seems to have inspired Simons’ obsession with Lord Jim, the eponymous hero of Joseph Conrad’s novel. Simons named his first hedge fund (an early incarnation of RenTec) Limroy after Lord Jim and the Royal Bank of Bermuda (which handled the new company’s money transfers), and his yacht The Lord Jim. In the novel, Lord Jim, a young sailor, abandons a sinking ship full of passengers, a split-second act of cowardice that haunts him forever. The rest of his life becomes an obsessive quest to prove his essential worthiness despite that defining failure. Simons seems to have considered his ditching of the purity of mathematics a similarly life-defining event.

Other than a few old-school traders who executed transactions, many at RenTec either didn’t care about or were slightly embarrassed by their wealth. When computer scientist Peter Weinberger interviewed for a job in 1996, he stood in the parking lot, sizing up the researchers he was about to meet. He couldn’t help chuckling. “It was a lot of old, crappy cars,” he recalled. “Saturns, Corollas, and Camrys.”

As a group of researchers chatted over lunch in 1997, one asked if any of his colleagues flew first-class. The table turned silent — not a single one did, it seemed. Finally, an embarrassed mathematician spoke up. “I do,” he admitted. “My wife insists on it.”

Many seemed to have been attracted to RenTec for reasons besides money. Some were drawn to the rewards of solving difficult trading problems. Others were attracted to the camaraderie and the fast iteration cycles: academic papers can take years; by contrast, Simons pushed for results within weeks, if not days. One visitor likened the atmosphere to a “perpetual exam week”.

Success in trading floor was also more clear-cut than success in research. Robert Mercer, the RenTec CEO who earlier worked at IBM, had become frustrated with the speech-recognition world, where scientists could pretend to make progress, relying on what he called “parlor tricks”. In trading, Mercer said, “[y]ou have money in the bank or not, at the end of the day. You don’t have to wonder if you succeeded … it’s just a very satisfying thing.”

Data

Says Cliff Asness, head of the hedge fund AQR: “To state the obvious, if we (or I) had a deep knowledge of how Medallion did it we’d do it too! I have always assumed they were quite early to things like stat arb, HFT, factor investing, natural language processing of financial news, etc.”

Natural language processing of news is just one example of RenTec’s precocious data sourcing.

In 1979, Simons emerged from a failed stint of managed investing (explaining his disillusionment, he told a friend: “If you make money, you feel like a genius. If you lose, you’re a dope.”) with a determination to build a trading system guided by algorithms. “I don’t want to have to worry about the market every minute. I want models that will make money while I sleep.” he said. “A pure system without humans interfering.” He suspected he’d need reams of historic data. He bought stacks of books from the World Bank and elsewhere, along with reels of magnetic tape from various commodity exchanges, each packed with commodity, bond, and currency prices going back decades, some to before World War II. “This was ancient stuff that almost no one cared about,” Zuckerman writes. “But Simons had a hunch it might prove valuable.”

To gather additional data, Simons had a staffer travel to lower Manhattan to visit the Federal Reserve office to painstakingly record interest-rate histories and other information not yet available electronically. For more recent pricing data, Simons tasked his office manager with recording the closing prices of major currencies. Each morning, she would go through the Wall Street Journal and then climb on chairs in the firm’s library room to update various figures on graph paper taped to the walls. (The arrangement worked until she toppled from her perch, pinching a nerve and suffering permanent injury, after which Simons enlisted a younger woman to scale the couches and update the numbers.)

In 1989, when powerful MIPS (million instructions per second) computers arrived, Simons’ hedge fund already had years of tick data featuring intraday volume and pricing information for various futures, even as most investors ignored such granular information and stuck to opening and closing data. “It wasn’t super clean, and it wasn’t all the tick data,” said Sandor Straus, the hedge fund’s data guru, but it was more reliable and plentiful than what the competitors were using.

By 2001, the team was collecting every trade order, including those that hadn’t been completed, along with annual and quarterly earnings reports, records of stock trades by corporate executives, government reports, and economic predictions and papers. Soon, they began tracking newspaper and newswire stories, internet posts, and offshore insurance claims. “The Medallion Fund became something of a data sponge, soaking up a terabyte […] of information annually, buying expensive disk drives and processors to digest, store, and analyze it all, looking for reliable patterns,” Zuckerman writes.

“There’s no data like more data,” Mercer told a colleague, an expression that Zuckerman writes “became the firm’s hokey mantra”. Since Renaissance’s goal was to predict the price of a stock or other investment “at every point in the future” — in “three seconds, three days, three weeks, and three months” — if a newspaper article appeared about a shortage of bread in Serbia, Renaissance’s computers could sift through past examples of bread shortages and rising wheat prices to see how various investments reacted.

Algorithms

Simons’ first automated trading systems focused on momentum (“if a currency went down three days in a row, what were the odds of it going down a fourth day?”) and correlations between commodities (“might wheat prices predict gold and other commodity prices?”).

Initially, Medallion’s researchers tried to make sense of the patterns their systems were finding. They had a three-stage process: identify anomalous patterns in historic pricing data; make sure the anomalies were statistically significant, consistent over time, and nonrandom; and see if the identified pricing behavior could be explained in a reasonable way.

For example, Monday’s price action followed Friday’s, while Tuesday saw reversions to earlier trends. Renaissance’s researchers, like Elwyn Berlekamp, conjectured that maybe locals liked to go home at the end of a trading week holding few or no futures contracts, just in case bad news arose over the weekend that might saddle them with losses. Similarly, commodities brokers seemed to trim futures positions ahead of the economic reports to avoid the possibility that unexpected news might cripple their holdings. These traders got right back into their positions after the weekend, or after the news release, helping prices rebound. Medallion’s system would buy when these brokers sold, and sell the investments back to them as they became more comfortable with the risk — effectively acting as insurance.

This general strategy of betting on retracements had enduring success at Renaissance: about 60 percent of investments that experienced big, sudden price rises or drops would snap back, at least partially. This played out especially strongly in currency markets: the correlation of price moves in deutsche marks, for example, was as much as 20 percent between any two time periods. The team found even stronger correlations in the yen. By comparison, the team found a correlation between consecutive periods of 10 percent or so for other currencies, 7 percent for gold, 4 percent for hogs and other commodities, and just 1 percent for stocks. “The time scale doesn’t seem to matter,” Berlekamp told one colleague. “We get the same statistical anomaly.”

This anomaly flew so strongly in the face of the efficient market hypothesis, which indicated the impossibility of beating the market consistently by taking advantage of price irregularities, that Straus felt the need to understand what may be happening. He found academic papers arguing that central banks have a distaste for abrupt currency moves, which can disrupt economies, so they step in to slow sharp moves in either direction, thereby extending those trends over longer periods of time. To Berlekamp, the slow pace at which big companies like Eastman Kodak made business decisions suggested that the economic forces behind currency shifts likely played out over many months. “People persist in their habits longer than they should,” he said.

By 1997, however, over half the trading signals Simons’ team was discovering were nonintuitive. Most quant firms at the time ignored signals if they couldn’t develop a reasonable hypothesis to explain them. But Simons and his team spent little time searching for the causes of market phenomena as long as their signals met various measures of statistical strength. “I don’t know why planets orbit the sun,” Simons told a colleague. “That doesn’t mean I can’t predict them.” The traders only steered clear of the most preposterous ideas: “Volume divided by price change three days earlier, yes, we’d include that,” said one executive. “But not something nonsensical, like the outperformance of stock tickers starting with the letter A.” Peter Brown, Renaissance’s CEO, said: “If there were signals that made a lot of sense that were very strong, they would have long-ago been traded out. There are signals that you can’t understand, but they’re there, and they can be relatively strong.”

Renaissance also capitalized on trading volume. Many traders ignored signals that worked barely more than 50 percent of the time. They’d move on to search for “juicier opportunities, like fishermen ignoring the guppies in their nets, hoping for bigger catch”, Zuckerman writes. But by trading frequently, the Medallion team figured it’d be worthwhile to hold on to all the guppies they were collecting — signals so faint their team started calling them ghosts. By 1989, Medallion’s average holding time had dropped from a week and a half to a just a day and half.

Error correction seems to have been baked into Medallion’s system. If a strategy wasn’t working, or when market volatility surged, the system tended to automatically reduce positions and risk. For example, Medallion cut its futures trading by 25 percent in the fall of 1998. By contrast, when Long Term Capital Management (which, in another departure from Renaissance, employed economists, not computer scientists and mathematicians) saw its strategies flounder, the firm often grew their size, rather than pull back.

For a while, Simons experimented with a journal club. Each week, he decided, Brown, Mercer, and other senior executives would be assigned three papers to read, digest, and present. But after reading several hundred papers, Simons and his colleagues gave up. The tactics sounded tantalizing, but when Medallion’s researchers tested the efficacy of the strategies proposed by the academics, the trade recommendations usually failed to pan out. Reading so many disappointing papers reinforced a certain cynicism within the firm about the ability to predict financial moves. “Any time you hear financial experts talking about how the market went up because of such and such — remember it’s all nonsense,” Brown would later say.

There is a similar lesson in Iain Dunning’s blog post for HRT titled “In Trading, Machine Learning Benchmarks Don’t Track What You Care About”:

One principle we apply to achieve [robustness and maintainability] is always using the simplest possible approach that achieves the desired outcome. For example, if a linear model is as good as a random forest model, we’d prefer the linear model. It is interesting to contrast this principle with the incentives in academic machine learning research. An empirically-driven paper is more likely to be published if it demonstrates novelty — but often when one optimizes for novelty, the results can be complex, which may make it less appealing in an applied setting.