Date Archives → February 2012

Algorithm wars and ultrafast machine ecologies

In an amazing 2011 TED talk, Kevin Slavin explained how “algorithms shape our world”.

Its well worth a watch to prepare you for the the main part of this post, an analysis of Black Swan events  in financial markets caused by exactly the sort of algorithms Slavin describes.

From “Financial black swans driven by ultrafast machine ecology”:

Society’s drive toward ever faster socio-technical systems, means that there is an urgent need to understand the threat from ‘black swan’ extreme events that might emerge. On 6 May 2010, it took just five minutes for a spontaneous mix of human and machine interactions in the global trading cyberspace to generate an unprecedented system-wide Flash Crash. However, little is known about what lies ahead in the crucial sub-second regime where humans become unable to respond or intervene sufficiently quickly. Here we analyze a set of 18,520 ultrafast black swan events that we have uncovered in stock-price movements between 2006 and 2011. We provide empirical evidence for, and an accompanying theory of, an abrupt system-wide transition from a mixed human-machine phase to a new all-machine phase characterized by frequent black swan events with ultrafast durations (<650ms for crashes, <950ms for spikes). Our theory quantifies the systemic fluctuations in these two distinct phases in terms of the diversity of the system’s internal ecology and the amount of global information being processed. Our finding that the ten most susceptible entities are major international banks, hints at a hidden relationship between these ultrafast ‘fractures’ and the slow ‘breaking’ of the global financial system post-2006. More generally, our work provides tools to help predict and mitigate the systemic risk developing in any complex socio-technical system that attempts to operate at, or beyond, the limits of human response times.

As a great friend of mine, who happens to be a banker who is very senior in the markets, once commented to me by mail, talking about how trading was accelerating into millisecond cascades as described by this paper:

slow the fck down everyone

Brainstorming Doesn’t Really Work

“The first empirical test of Osborn’s brainstorming technique was performed at Yale University, in 1958. Forty-eight male undergraduates were divided into twelve groups and given a series of creative puzzles. The groups were instructed to follow Osborn’s guidelines. As a control sample, the scientists gave the same puzzles to forty-eight students working by themselves. The results were a sobering refutation of Osborn. The solo students came up with roughly twice as many solutions as the brainstorming groups, and a panel of judges deemed their solutions more “feasible” and “effective.” Brainstorming didn’t unleash the potential of the group, but rather made each individual less creative. Although the findings did nothing to hurt brainstorming’s popularity, numerous follow-up studies have come to the same conclusion. Keith Sawyer, a psychologist at Washington University, has summarized the science: “Decades of research have consistently shown that brainstorming groups think of far fewer ideas than the same number of people who work alone and later pool their ideas.”

Brainstorming Doesn’t Really Work : The New Yorker. via Simoleonsense