The Proof Is In The Pudding: How portfolio crash testing worked for me.

In the last post I discussed my history using Portfolio Crash Testing’s (PCT) big brother, Riskostat. Riskostat shares the same DNA with PCT, but Riskostat contains more features like risk factor decompositions, marginal contributions to risk, etc. All the things that quant geeks require in a system. But PCT makes the information more accessible and easier to incorporate in a conversation with a client. It also contains the most important feature of Riskostat: Crash testing (aka scenario analysis). In this post I will dive into one of the key events that convinced me that crash testing really works.

In my role as the Chief Risk Officer of an OCIO, I had modeled all the managers on our platform using Riskostat. There was one platform hedge fund in particular that was problematic. They never lost any money for our client, but they never made money either. During 2011-2015, this was a real issue. The historic returns of the fund were about 1.75%-2.5% p.a., and at a 2% management and 20% performance fee, the manager made more off of a $100M investment than our client did. Further, using the Riskostat software, I didn’t see any way that this manager could make money in the future. The portfolio was so risk controlled that they were impervious to all but the most idiosyncratic of events. With this information, I decided to pay the manager a visit to find out how they expected to make money for my client.

So I flew to New York and met with client, PC and Riskostat software in tow. We met in a conference room where I had projected the modeled version of their portfolio for all to see, and I asked them how they expected this portfolio to make money.

They replied that the portfolio was positioned to make significant gains when Japanese credit spreads widened. I asked “by how much”. They replied “600 basis points”. So I clacked in a 600bps widening of Japanese credit spreads and Riskostat showed that they would make about 1.25%. They were stunned at the speed of the software, and asked if I really did anything, then they replied that my results were impossible. In particular, they argued that Europe and North America would remain unaffected by such an event.

So I keyed in what they thought the market effects would be in such a scenario. And we kept modifying and modifying, taking their ideal scenario further and further away from market conditions that would be expected in such an event. The best result we got, even with unrealistic assumptions, was that the strategy would make about 2% in their perfect storm. We were doing modeling that they were unable to perform themselves.

Further, the conversation led me to realize that the manager really didn’t care about Japanese corporate spreads widening, they cared about spreads widening in 3 Japanese credits in particular. Of course, their response was that we modeled the scenarios and assets improperly and they would make “well over 10% and closer to 20% in such a credit widening event”.

The conclusion I came to, was that the manager didn’t have a good handle on how they were positioned or how they would make money for my client. They didn’t understand how their European positioning was offsetting their Japanese positions. An event would have to play out perfectly for them to make money and we didn’t think such a scenario was realistic. Against the backdrop of 3 consecutive years of disappointing returns, my firm fired them. But we still had 90 days before the client would get their money back.

During this time, their event occurred (or at least a version of it). While the Japanese credit market in general widened slightly, 2 of their 3 names did widen considerably. And during the 3 months that we waited to redeem, the manager did make 2% for our client. And because they didn’t take us off their distribution list, I know that over the rest of the year, they resumed making almost nothing for their clients. Exactly the way I expected it thanks to the Riskostat software, which in turn increased my confidence in the analytics.

In my next post I will discuss how advisors can utilize the scenarios in PCT to have meaningful conversations about portfolio positioning with their clients.