The problem with scenario analysis

The three methods described above are all types of scenario analysis, i.e. they involve
applying a particular scenario to a portfolio and quantifying its impact. The main
problem with scenario testing, however it is performed, is that it only reveals a very
small part of the whole picture of potential market disturbances. As shown below, in
the discussion on systematic testing, there are an extremely large number of possible
scenarios. Scenario analysis will only identify a tiny number of the scenarios that
would cause significant losses.

A second problem with scenario analysis is that the next crisis will be different.
Market stress scenarios rarely if ever repeat themselves in the same way. For any
individual asset over a period of time there will be several significant market shocks,
which in terms of a one-day price move will look fairly similar to each other. What is
unlikely to be the same is the way in which a price shock for one asset combines
with price shocks for other assets. A quick examination of Table 8.2 above shows
this to be true.

‘All right, so next time will be different. We will use our economists to predict the
next economic shock for us.’ Wrong! Although this may be an interesting exercise it
is unlikely to identify how price shocks will combine during the next shock. This is
simply because the world’s financial system is extremely complex and trying to
predict what will happen next is a bit like trying to predict the weather. The only
model complex enough to guarantee a forecast is the weather system itself. An
examination of fund management performance shows that human beings are not
very good at predicting market trends, let alone sudden moves. Very few fund
managers beat the stock indices on a consistent basis and if they do, then only by a
small percentage. What is needed is a more thorough way of examining all price
shock combinations.