Ahead of their sessions at QuantMinds International, we interviewed Alexander Lipton, and Marcos Lopez de Prado, both Global Head – Quantitative Research & Development, Abu Dhabi Investment Authority (ADIA)
Much of the empirical evidence presented in finance’s academic journals is anecdotal, in the form of associations, and does not hold up to scientific standards. The Abu Dhabi Investment Authority (ADIA), a global investment institution that invests funds on behalf of the Government of Abu Dhabi, has developed a process to assess the scientific validity of empirical claims, allowing it to concentrate on the most promising investment ideas. We sat with two of members of the team responsible for this work at ADIA, Alexander Lipton and Marcos López de Prado, Buy-Side Quants of the Year 2021, to understand better the nature of this challenge.
A theory is an explanation of how X causes Y through a mechanism M. To be scientific, the mechanism M must be falsifiable, that is, it must be able to admit empirical evidence to the contrary, in the form of experiments designed to prove that its postulates are false. For example, Ohm’s law is a postulate, derived from electromagnetic theory, that tells us that the current through a conductor between two points is directly proportional to the voltage across the two points. We can falsify this postulate (and with it, the theory) by designing an experiment that yields measurements inconsistent with the postulate’s predictions.
In the context of investments, an example of a scientific claim is the square root law, which states that market impact is proportional to the square-root of the executed volume. This claim can be deduced from first principles of market microstructure theory. One can design controlled experiments to falsify this claim (and the theories it is derived from) under various market conditions. Sophisticated asset managers routinely conduct controlled experiments to test the validity of their market microstructural theories, even though the results from those experiments are not published in academic journals.
Marcos López de Prado
Global Head, Quantitative R&D at Abu Dhabi Investment Authority (ADIA)
Alexander Lipton
Consider the large losses experienced by value funds between late 2017 and early 2022. Investors never received a straight answer to the question “why did value funds perform so poorly?” The reason is, absent of a causal theory, nobody knows why value funds should have performed well in the first place, or what turned the sign of value’s risk premium. Answering the “why” question requires a falsifiable causal mechanism, which to this day remains unknown for value investments. The factor investing industry has reached a size of over USD 3 trillion based on associational claims, without scientific standards being applied to develop experimental evidence.
Non-scientific claims abound in the factor investing literature, efficient market hypothesis framework, and dynamic stochastic general equilibrium model, to mention but a few. Take, for example, Fama and French’s factor investing model. These authors claim that markets reward investors who hold certain risk characteristics, which they denote by the names of beta, value, size, and quality. This claim may or may not be true, however it cannot be considered a scientific claim, since it is not derived from a falsifiable theory that explains how investors holding those risks are rewarded. The authors do not propose a causal mechanism that can be subjected to falsification. The empirical evidence presented by these two authors is associational but, as we all know, association does not imply causation, and without causation there is no falsifiable theory.
Almost all quantitative investment firms use backtests to research and select strategies. However, a backtest does not tell us what the source of the simulated performance is. A simulation is not a controlled experiment. A backtest is associational, non-causal evidence. The simulated performance can be the result of random variation (coincidence, a statistical fluke), or unknown preconditions that will not hold going forward. Just because something appears to have worked in the past, it does not mean that it will continue to work going forward. As a result, most quantitative investment strategies are not scientific. Yes, they may rely on complex math, but you would be surprised to learn how much mathematics astrologers and alchemists have used over history.
The problem is, associations are not stable, even if the causal mechanism remains the same. When a firm develops a product based on an observed association, it is implicitly assuming that the association will persist, even though the firm does not understand the causal mechanism responsible for the association. A small change in one of the parameters of the causal mechanism can reverse the sign of the association, exposing the investor to a stream of systematic losses. Association-based investment strategies are primed to suffer losses in the face of black-swan events. Only causal theories can answer counterfactual questions, and anticipate the possibility of black swans.
As a 600 year-old Spanish poem says, “we cannot tell our song, but to those who travel with us.” For confidentiality reasons, I cannot more specific on our approach. What I can tell you is that our Quantitative Research & Development team at ADIA formulates falsifiable theories that explain why associations exist as observed, and then subjects that theory to rigorous scrutiny by applying recent advances in causal inference and causal discovery. We have engaged with some of the leading minds in those fields, and have developed strict protocols for controlling the false discovery probability. Backtest overfitting is the deadly sin of quantitative investing.
Every year, new alternative datasets become available at an increasing rate, allowing researchers to conduct natural experiments and other causal analyses that were not possible in the 20th century. I understand why Fama and French proposed these associational models 30 years ago. At that time, it was the best that they could do. Three decades later, we can do much better. It is time to bring greater scientific rigor to investing, which, in due time, will produce replicable profitable outcomes.
The adoption of causal inference methods has the potential to transform investing into a truly scientific discipline. However, investment research can often be influenced by extraneous factors, particularly when commercial interests are at play. Researchers wanting to fully explore the causal mechanism responsible for financial phenomena, should seek opportunities at organisations that rigorously apply the scientific method in their work.
Answering the “why” question is more than an academic pursuit. Causal theories benefit investors, for several reasons: Interpretability, transparency, reproducibility, adaptability, etc. This is an exciting time to do investment research, particularly if you find a firm that allows you to work as a scientist.
Don't miss Marcos and Alex at QuantMinds International. Secure your spot now.
