This is what Wall Street’s terrifying robot invasion looks like
August 8, 2012
This animated GIF chronicles the rise of the HFT Algo Machines from January 2007 through January 2012 (credit: Nanex Research, hosted by imgur.com)
Given the the endless mind-whirling acronyms, derivatives and structures of the financial markets, we’re rarely served with a visualization that so elegantly illustrates the arrival of Wall Street’s latest innovation.
This is what high frequency trading looks like, when specially programmed computers make massive bets at lightning speed, Mother Board reports.
Created by Nanex, the GIF charts the rise of HFT trading volumes across all U.S. stock exchanges between 2007 and 2012. The initial murmur, the brewing storm, the final detonation: Not just unsettling, it’s terrifying.
As Mother Board notes, we don’t know is what the long term consequences are of all this hyper-volume as depicted by the Nanex GIF and the kind of systemic risks created from the market’s ongoing evolution from human traders to rapidfire AI. Sometimes things go wrong, a software glitch, an algorithm gone rogue and the music stops, like last week when Knight Capital lost $10 million a minute when it’s trading platform went haywire or during the infamous Flash Crash when the Dow dropped 1000 points in mere minutes.
Read the excellent full Mother Board article here.
However, according to Nanex Research, “It’s not high frequency trading (HFT) that concerns us. It’s high frequency quoting, and it should concern everyone. The two images below tell the story. The chart on the left shows the growth of high frequency quoting. The chart on the right shows the (lack of) growth of high frequency trading.
Quote data is fromCQS, trade data is from CTA, both which cover listed stocks on NYSE, AMEX, and NYSE-Arca between 2008 and 2012. Quote spam has exploded with no signs of stopping, while trade frequency has stalled and is actually lower than it was years ago.
Each day is plotted in a separate color over the course of a trading day (9:30 to 16:00 Eastern): older data uses colors towards the violet end of the spectrum, recent data towards the red end of the spectrum. The gaps you see between color groups on the quote chart (left-side) is when system capacity was upgraded to handle the increase in traffic, and quote spam jumped to fill the new capacity that very same day.
(Credit: Nanex Research)
Comments (37)
by Ed Baxter
While technology is helping us tremendously in easing up our lives. One must scrutinize the factors contributing to the data. I would say based on the graph how the poor ones are getting poorer and the other side becomes richer.
by communitymotive
Live by technology, die by technology. The tools have no ethics, so they can be used to strip assets and gain wealth. Since wealth is adored regardless of how gained, and it can buy off those nominally in charge of regulations, we get the market whose sole purpose is to enable manipulation and take wealth from others as micro-crimes. The singularity? The 1% of the 1%.
by Robert
Thank goodness the author of the post told me that the increased activity was terrifying, otherwise I would have thought it was just like all the other Wall Street fads that were supposed to kill us all and our children, but didn’t. Making the graph’s colors an angrier and angrier red was also a nice touch. I’ll have to go to Fox News to tone down the hyperbole and alarmism.
by Bri
So what your saying is, all this alarmist talk is unsettling to you, and you want to take a sedative. Like George bush telling you that the fundamentals of our economy are sound and not to worry about the crash(2008), and just go out and spend money. I’d love to talk reason with you buy I think your addicted to those pain killers. Look I’m waving my fairy wond and saying. Don’t worry it will be all right, there’s nothing wrong, all you have to do is get ride of those darn liberals. One of my clients, a hedge fund manager, told me, the housing problem was Jimmy Carters fault. Mr Carter has spent many years, hammer in hand, trying to help the poor. One of my Hispanic works in 2007 tried to buy a house. He was shown a 500,000 dollar house, and wanted to buy it. They would have given him a no questions mortagage. He would have lost it, one way or another, no matter how many illegal immigrants he put in there. I somehow don’t think that is what Jimmy Carter had in mind with habitat for humanity. They actually believe the hype and spin of those very slanted news outlets. Like gush limbo. I know it’s hard, but drink a cup of coffee, snap out of it, and think critically. Somebody the other day was saying that there is no way that man’s tiny amount of carbon release could cause global warming. That the biota of the planet produces so much more. Just one problem with that argument. The biota is a closed loop. We introduce carbon that was in the ground, thus adding to the loop. Critical thinking is not easy. Letting Fox News think for you won’t help. Oh and by the way. Those color choices are pretty standard. They are used in many graphic representations. The differences make it easier to see separate bits of information. Nothing intrinsically emotional about it.
by Orcanator
Your brain is overstimulated but also very weak. You are clearly ignorant of the Jimmy Carter legacy: “The Community Reinvestment Act” which forced banks to lend money to people who couldn’t afford to pay it back. Your pals Clinton/Frank upped the ante in the ’90′s setting the stage for mortgage backed securities and other Wall Street nonsense. Get a clue.
by Matthew Thurn
Hahaha! Some people are f#*king crazy I think. Geometric growth in any direction on a scale (x,y) such as this is hysterical!!! Thanks for the laugh!
by Erik
what a douche-nozzle comment
by Ed
all of this could be solved if they simply instituted an across the board millicent tax on all trades – and then increased this tax, incrementally, on a year to year basis.
I’m not against AI per se doing trades, but we are optimizing for the wrong version of AI. We need AI that does its stock picking against long-term indicators, analysis, and so forth – that has TRUE merit, in my opinion as opposed to this technology, which only makes the roulette wheels spin a little more efficiently.
by Eric Ansley
Yes, many of us are aware of this potential problem. Some observers have proposed eliminating futures and similar products. Others suggest that high frequency trading should be prohibited. Gold and silver holdings provide a personal hedge, but won’t provide us with a sustainable future. We recognize that we have a challenge, and there is still time to respond. This is a fine opportunity to post solutions, not fears.
The solution isn’t to kill trading liquidity, or change the way human investors participate, through algorithm or not. We all have different kinds of personal assistants… from human to voicemail… some have trading formulas. The outcome we would like to avoid has been from, and will come from institutional programs operating virtually within the exchanges themselves.
The relationship between faster quote data and faster trading is clear. I question an implication that quote data speed or depth would minimize future economic trauma from emergent behavior. Humans will still not be able to participate in market decision-making simply by slowing the feed. HFT systems will respond to delayed data too, before any human can register a bar change or respond to a significant event. It is not possible to release information to humans as a priority. We are one.
A possible solution? Introduce a minimum time for any instrument to be held. If a program buys in, it should not be able to sell within microseconds, but should be required to hold its purchase for a specified period of time–whether that is a bar minute or ten. I’d recommend a minimum of three minutes to level the field and avoid a resonant mess. The same rule must apply to human traders–or workarounds will be found. Stop loss triggers might still apply to protect traders from unusual change during an obligation period. From a microsecond to a minute is huge, and we would see a substantial improvement in the robot trading patterns… until the next challenge emerges.
by 4science
I thought about this some before posting as to avoid sounding “knee jerkish”. I for one have lost significant confidence in the stock market. With AI that trades virtually at the speed of light(light through fiber that is) much of the financial system is losing value(my perception obviously). Service value and wealth are getting distilled down further and further, faster and faster. You use to be able to find a good financial advisor but how can these people compete with current and future AI? The AI reacts so fast and you never know what is going to happen. I am mostly conservative and very pro technology. It’s not that I’m anti AI(I’m not). It’s just that there are so many disruptive technologies coming online. I think about this daily but for the life of me, I can’t figure out(yet) how to employ people in the near and far future. Through my voting life I’ve supported conservative and independent candidates but I see AI as a huge game changer. I believe that the conservative movement needs to think long and hard about convincing people they know how to create and maintain jobs in this new world. I have quick note to finish with since I mentioned conservatism. The coolest guy I ever met was a diehard liberal. Less hate, more reaching out. No one candidate or party can fix this but if people work together and look for solutions within themselves I believe they will find them. Communication is the foundry of creation. That has served my friendships and marriage very well.
by ChrisF
Hi 4science, I think you’re exactly right. What we’re beginning to see in the financial markets is technology rendering highly paid human experts obsolete…. a process that will ultimately lead to widespread white-collar unemployment. I’m also optimistic about the ‘end state’ for humanity, but I think it’s going to be an extremely turbulent decade or two.
I can recommend a book which addresses this issue in great depth : Lights in the Tunnel by Martin Ford. A rather grim read, but it does discuss some interesting solutions.
by melajara
4science, you are raising very important points.
We are witnessing the prodrome of massive unemployment!!!
U.S.A. built itself on the assumption of full employment. This is still possible but not for material goods creation. Services too will be fulfilled more and more by increasingly intelligent machines. Then what humans are good at? Just at entertaining other humans! (Remember the concise Aristotle definition of a human being as a “zoon politikon”).
Quite unfortunately, the all too myopic democracies (myopic because they are, by definition, suffering from a debilitating illness: electoralism backed by promises for willful thinking fulfillment in the electors mind) are ignoring this because it’ s still over the event horizon of the typical electoral mandate (up to 5 years).
Massive job transfer to robots should occur IMHO from circa 2020 onwards. It’s about time to prepare for this!
Alas no political figure would address this central subject because it should not impact his/her immediate next mandate and, being fearsome, the topic would be a potent repellent to vote for such a candidate, barring general uneasiness for people to confront changes in their habits.
The topic could still be avoided by the next presidential candidates running for 2016 and in 2020 it will be too late!
Failing to plan is planning to fail. In times of accelerated changes toward the singularity, this endemic weakness from the democratic political regime will only be more severe year after year and could prove fatal to so called “advanced” western societies in teaming up for the advent of an “optimal” (read as human as possible) singularity.
by Bri
Yes, exactly Melajara! The only point I would call into question( as in examine further), is the 2020 time line. It’s already begun, and will only pick up speed and momentum. The politician will only become aware, when the people raise the issue. In the long run, it’s a change that we all want, but emotions will run high. It will ignite a wave of Ludite anger. We need something like an Arab spring coelecence of like thought. No particular leaders, just unity of thought. Being a leader is like standing on a roof during a lightning storm and waving a golf club in the air.
by Carl Brooks
Cant really comment much about this story due to a massive lack of knowledge on the subject. However, i have a thought that i feel i need to post.
What happens to money (it being a technology), when it becomes an information technology? When it was just a technology it had a “linear progression”, but now its starting to become an information technology, what is the accelerating return? where is the exponential?
is this where the problem lies?
Anyone?
by melajara
Money is still backed by tangible assets and natural ones (e.g. gold). This is what gives ultimately its legitimacy to money (Remember Fort Knox, the Bretton Woods system and USD to gold convertibility).
Historically our very concept of money is in coextension with the concept of a finite world or a world of finite resources. When IT will make the creation of “natural resources” almost unlimited (given enough energy supply) then money will have to be backed by something else.
I think it will/should be backed by what makes “natural resources” possible, so in a post singularity world, money should be backed by energy!
by Carl Brooks
i defiantly agree that the tangible resource that backs money will have to change but i’m not to sure that its going to be energy that fills that gap. Energy capture too will become abundant and when something becomes abundant it inherently looses its “value”. This “resource” would have to be something that is constant and has “value” and yet can not be something that can be created/engineered/captured in a way that makes it abundant. So what do you use when everything becomes abundant?
by AlexLoures
How do you know what companies will be around and profitable? I read about strong AI,disruptive technologies, in this forum but when it comes to business is more of the same.Gold and silver?What value do they really hold? The algorithms will get better ,profits will soar ,but what is been created?
by JohnAlbers
The companies that will be around and profitable will be those that are not on the stock market.
by melajara
Indeed, I would love to be able to invest in Zyvex ;-)
by ThisIsIt
They are systematically destroying the public’s confidence in the markets and investing, and ultimately money itself.
The last refuge will be, as it always has been in the past – gold and sliver.
Where do you go when you don’t trust ANY institution – Bank, Broker, Government.
Look at the poor investors in Madoff, Stanford, Washington Mutual, Lehman, MF Global, PFG, etc…
They fall one by one, their investors and many time their clients are out of luck.
When the average 401k investor/ Pension Fund pulls out who will be left to fleece?
by BabyMachine
I would like to point out the promise of Bitcoin. Digital currency could eventually replace physical money.
by Faul Sname
What about Aluminum? As in the physical metal. It’s useful for a huge number of practical applications, and is one of the hardest ones to obtain. Also, in the likely event that society doesn’t collapse, it will still be valuable with less fluctuation in value than Gold (more than silver though).
Alternatively, Platinum group metals (which are probably a good investment regardless).
by alex
Quotes affect bid and ask prices. You can change a price simply by quote spam. The profit is then made a millisecond later when an algorithm takes advantage of the spread that the other created.
by John Wentworth
Does anybody know WHY growth in quote volume is supposedly such a problem? It increases network congestion, but how does it create a serious risk of financial collapse?
by Bri
Many companies don’t go public. They don’t want the games. Remember when you invested in a company because you believed in them? If your a young company, you need investment. When I talk to my shock broker, he talks about making our money and going, ” like a herd of locusts”(I don’t have the heart to tell him they swarm). If the investment capital is so volatile, it makes it that much harder on the company. They loose the wind in thier sails. Then somebody comes in and shorts it, greed is the driving force. They even promote it to me. Those high speed trading programs are optimized for greed, not good. All about the individual, not the group good. If every individual of every company, didn’t charge a salary, but instead drew thier income from the some of all companies. Thier would be plenty around for all. Like I said, if you spent one thousand dollars a day, since Jesus was around, you’d still have hundreds of years to go. The 99% are getting pretty scabby, a fleeced sheep go. They will eventually revolt, if moderating forces aren’t put into place. Organize, discuss, but take my advice, don’t polarize. The one percent are a part of the whole too. I could speak on thier behalf just as easily. They rationalize that they are doing good, and that they are better at it.(standard republican thought is that the masses can’t govern. That they have the skills from experience. At this point in time, we don’t really have a democracy. We have an oligarchy. I have said this since high school. What’s so funny about peace love and understanding. Hey, if you really want to see how bleak thier gilded world is, watch Born Rich. It’s a Johnson and Johnson heirs, college film class documentary of his disillusionment with being super wealthy. A real eye opener. Try to feel compassion along with the anger , they are lost in the wilderness.
by ChrisF
There’s nothing stopping you from investing in companies you believe in, Bri. Just pick a solid stock that pays a good dividend yield and forget about it for a few years. Trying to time the market is a fool’s game, because you’re increasingly competing against HTF algorithms. But for those with a long-term view, nothing has changed.
by Bri
That’s my point. I do believe in venture capital. I do invest. It’s just that wall street has lost that purpose. People think it’s a personal gold mine, that they can horde for themselves. All the pensions are long term. The bank that’s drawn from is the portfolio. In theory that draw shouldn’t be disruptive to the functioning of those companies. In reality it’s like a pickpocket convention. If a company looks weak, short it. If enough people( at this point machines) perceive that trend, they all dive in, and destabilize the company. If a company is doing well, day trading activities draw of much of those positive sentiment and the companies hard work gets eaten up by lightning trades, playing small percentages, in high volumes. The market has been subverted into a parasitic relationship, with the companies they were supposed to be lending to, to help.
by Boristabby
Most astute. Meanwhile, what do we in the 99% do? How is FOREX affected?
by de Broglie
Own equities in the companies that are going to profit. That is the beauty of our late capitalism. Bri doesn’t seem to understand how ownership works in the world equity markets, especially the United States.
by ChrisF
Exactly the comment I was going to post, de Broglie, thank you. All – If you’re truly convinced that Goldman Sachs (or any other large corporation) is able to manipulate markets and hoover up the world’s wealth so easily, then simply buy their stock and take your share. Alternatively, invest money in one of these algo hedge funds that are supposedly generating such easy returns.
by Bri
I do invest. I have a broker. I invest in companies that I believe in. My broker tries to tell me to play it only for the money. Companies need investment capital. If I follow my brokers advise, as soon as the company starts to make money, I would withdraw that and then some profit. This only hurts the company I was supposed to be helping. Shorting companies has a similar effect. Any sign of weakness gets exploited. If enough machines get a sense of that inflated weakness, it further weakens what might be a good company, going through some hard times. It increases the likelihood that the company will struggle and fail. I’ve seen this with good companies. To have therebest efforts thwarted, as nesscesary capital gets taken away, when they need it most. Fair weather investors fleeing because they can’t make a quick buck. Those high frequency machines drain a lot of the new investments, as they play small differences in high volume. It saps the strenghth of the company for a quick buck. I believe in the market. It’s our best hope for financial security, once robots do all jobs. I know quite a few brokers. If you don’t understand what I’m saying, you don’t know how they play the system. I’ve known them to buy a big block to generate interest. As people move in to ride the wave, they take thier initial money and then some. That’s a parasitic relationship. Not a beneficial helping a company by giving needed venture capital. If you don’t understand, you don’t know the street look at the Hunt brothers and silver. A classic example. It’s more of a poker game to them.
by Gorden Russell
This is fantastic, Melajara. How did you find all this out?
by melajara
I’m indulging myself in algo trading ;-)
by Bri
Nice post melajara. A lot of my clientele is from wall street. High frequency trading is taking thier jobs away. Everything is getting consolidated(hands of the ice skater pulling in closer). Fewer people are making money, just the company and it’s principal owners, eventually. The underlining greed still is the driving force( the ice skater spins faster)). More companies get gobbled up by larger ones, as they can’t compete( more arms being pulled in)) to introduce the idea, I use the ice skater analogy, but in reality it’s a tornado, and it will suck up everything. Still early in terms of the storm. As robots and AI increase thier effects, the consolidation(arms pulling in) will increase in speed. Eventually, very few will own it all, briefly. Then total collapse of the old system and a new system will evolve. Every stop gap measure will fail, because they aren’t addressing the main issues. Each of these machines are a stone axe that someone wields. When bill gates owned Micro soft directly, it was an excellent stone axe for him. He built up a tremendous charge. If we all owned micro soft, we would all have access to finance to consume goods. Draw from it like a bank. All business will eventually be one, and we will all draw from it. How much, when where and why still needs to be worked out, but we will. We have no choice. Don’t worry about stock market gyrations. They will put bandaids on it. To successful, too much money to be made. Just fewer people will be able to make it. We can only watch in horror as AI does what would have taken days, months years to unfold, in a matter of seconds. Like watching the war simulations in War games, as they speed up until collapse. The days of things being adversarial are coming to a close. The days of things being symbiotic( of the many, one) will be ushered in by strong AI. Competition was essential for evolution, but we have changed the game with the advent of our mighty stone axes if wall street is to remain competitive, thier can only be one winner. Each axe will fight till there is only one left. Each one winning and consolidating, acquiring, till only one remains.
by Gorden Russell
Hey Bri, did you know that in Egyptian hieroglypics, the pictograph of a stone axe is the word for God?
by melajara
Indeed KCG stock went wild after the “glitch” they acknowledged, costing them $440M. For a few days, the stock went mad, going from $10.38 (normal price before the issue), to below $7 the first day, resuming @$3 the following day, going past $4 intraday, but down to $1.86 AH on imminent BK speculation and from $3 to $4.25 again the following day. All that on a sustained daily 100M+ volume.
Once again, there is a bank behind those mad fluctuations, Goldman Sachs who paid for KCG misstrades a whoopy $440M in order to be able to play cat and mouse with KCG now (btw making already substantial profits with intraday trading of KCG stock those days).
GS is the same infamous bank dictating to the FED its policy and even almost directly governing several EU countries (Greece, Italy, soon enough Spain and even the EU Central Bank).
Add to this U.S. general landscape for the equity market. 70 separate venues for trading (a byproduct from 911), very opaque “dark pools” of money, heavens for High Frequency trading over laundered money, and a staggering corruption across WS.
In comparison, reread Chap II from “The Singularity is near”, Singularity as an economic imperative. So optimistic, painting a so rosy economical future of sustained growth benefiting everybody.
Alas, on economics, Ray has been way too much optimistic, underrating the destructive power of greed and corruption over the preservation of our common interests.
by pt
Old, inefficient, and wasteful systems are being underscored with this sort of behavior and they are coming closer and closer to collapse every day. A new economic system will ultimately evolve out of this, and much of the reason for that is because so many more people are aware of the flaws these days due to the spread of information, us being here discussing it being a perfect example of that. How long it will take to evolve is up to how people react to it all, but it’s inevitable that the system will collapse and dissolve. Too optimistic? Perhaps, but chaos and death are always the precursor to rebirth, it’s a necessary mechanism of change. The internet, and mindsets that it’s cultivating are forcing a change to happen, and the stubborn kings of old hierarchies are clinging to their power as it flows through their fingers.