Stress test the model

Before using a model, one should perform a set of scenario analysis to investigate
the effect of extreme market conditions on a case-by-case analysis. The G30 states
that dealers ‘regularly perform simulations to determine how their portfolio would
perform under stress conditions’. Stress tests should ‘measure the impact of market
conditions, however improbable, that might cause market gaps, volatility swings, or
disruption of major relationships, or might reduce liquidity in the face of unfavourable
market linkages, concentrated market making, or credit exhaustion’.
Scenario analysis is appealing for its simplicity and wide applicability. Its major
drawback is the strong dependence on the ability to select the appropriate extreme
scenarios. These are often based on historical events (Gulf War, 1987 crash, European
currency crisis, Mexican peso devaluation, Russian default, etc.) during an
arbitrarily chosen time period. Unfortunately, on the one hand, history may not
repeat itself in the future; on the other, for complex derivatives, extreme scenarios
may be difficult to identify.13 Of course, increasing the number of scenarios to
capture more possible market conditions is always a solution, but at the expense of
computational time.
In addition, stress testing should not only focus on extreme market events. It
should also test the impact of violations of the model hypothesis, and how sensitive
are the model’s answers to its assumptions. What happens if prices jump, correlations
behave differently, or liquidity evaporates? Model stress testing is as important as
market stress testing. If the effects are unacceptable, the model needs to be revised.
Unfortunately, the danger is that one does not really suspect a model until
something dramatic happens. Furthermore, there is no standard way of carrying out
stress model risk testing, and no standard set of scenarios to be considered. Rather,
the process depends crucially on the qualitative judgement and experience of the
model builder.