One of the most-watched companies these days is a hedge fund in Silicon Valley called Sentient Technologies.  Sentient is led by a scientist called Babak Hodjat, and Hodjat has created a hedge fund run entirely by artificial intelligence.

Warren Buffet is famous for saying that nobody really knows what the stock market is going to do. Babak Hodjat and his his team are betting that a machine learning system, trained on the right datasets, can figure it out.

This is brilliant.

Sentient and Babak Hodjat are not an anomaly: they’re the tip of an iceberg in a sea that’s been heading this way for years. Neural networks and genetic algorithms have been used for predicting stocks for quite a while now.

I Know First does it. Bridgewater Associates, the world’s largest hedge fund (they sound very stiff, don’t they?) are building systems to automate the running of the entire company; they say they’ve already got machine prediction systems doing their thing for fund strategy.

In fact, stock markets are perfect for machine learning (to use the real term). Large amounts of data at high velocities – it’s actually surprising that it’s taken us this long to come to the point where the hard math and computing is being applied to Wall Street. What gives?


One: the efficient market hypothesis – the idea that we operate in markets so efficient that the moment some information becomes available about a stock, the sellers and buyers do their thing, and thus the price of a stock accurately reflects all known information.

It’s a very elegant theory, but we don’t seem to have efficient markets. Information asymmetry is a huge problem. Even if we take out insider trading and front-running and all that monkey business, the simple fact is that no human can look at all of the millions of transactions happening every single second and make the decisions that need to be made. And if those decisions can’t be made, then no, the stock prices aren’t the sum of all known information about the stock.

Two: the Random Walk. Behold:

Burton Malkiel, in his influential 1973 work A Random Walk Down Wall Street, claimed that stock prices could therefore not be accurately predicted by looking at price history. As a result, Malkiel argued, stock prices are best described by a statistical process called a “random walk” meaning each day’s deviations from the central value are random and unpredictable.

– Wikipedia

Basically, a poor mathematician back in the 70’s confused stock markets and quantum theory. He didn’t realize what we could do with big data and the right set of ML algorithms


Where does this take us?

Well, for one, a lot of Wall Street are going to lose their jobs. Stockbrokers – the kind we saw in the Wolf of Wall Street – will be dead soon. Instead of Leo DeCaprio, companies will be run by enormously complex prediction systems.

And people will entrust their money to these companies based on the kind of systems they’re running. The research and discoveries in this field are likely to be closely guarded commercial secrets, so we’ll see people picking Hedge Fund A over B because they believe A has better algorithms and better data scientists. Someone will set up a niche fund that does bizarrely well in trading energy interests. Some will generalize. Vanguard will probably have its own Ai running the show.

The average stock broke firm will no longer be a bunch of MBAs, but a handful of computer scientists like Babak Hodjat, sipping decaf lattes in Silicon Valley.

(Interestingly, this brings us back to that efficient market hypothesis. If everyone was running a massive machine intelligence system capable of looking at all of those billions of data points and making reasoned predictions, then the stock would be a better indicator of available information.)