Implementation issues

The implementation issues are the final sources of model risk in the model-building
process. Let us provide some examples.

First, data problems. Which data source should be used? Datastream, Reuters,
Telekurs, Bloomberg, Telerate, Internet and others often provide different quotations
for closing prices, depending on their providers. At what time of the day should the
data be pulled into the model? And in the presence of multiple currencies or multiple
time zones, other problems arise. For instance, should the CHF/USD exchange rate
be taken at New York or at Zurich closing time? In a global book, at what time should
the mark-to-market be performed? Can the data be verified on an independent basis?
Second, continuous time problems. Despite its limitations, the concept of delta
hedging is a cornerstone of modern financial risk management. In theory, the
constructed hedge requires continuous rebalancing to reflect the changing market
conditions. In practice, only discrete rebalancing is possible. The painful experiences
of the 1987 and 1998 crashes or the European currency crisis of 1992 have shown
the problems and risks of implementing delta-hedging strategies in discrete time.
Third, oversimplifying assumptions. For instance, an important model risk source
is risk mapping. When calculating the Value-at-Risk for a portfolio using the variance–
covariance method, it is common practice to take the actual instruments and
map them into a set of simpler standardized positions or instruments. This step is
necessary because the covariance matrix of changes in the value of the standardized
positions can be computed from the covariance matrix of changes in the basic risk
factors. Typically, an option on a stock will be mapped into its delta equivalent
position, which is itself related to the volatility and correlation of a given stock market
index. This suffers from important drawbacks. First, the dependence of the option
on other risk factors – such as interest rates – will often be omitted, because the
delta equivalent position does not depend on them. Second, the computation of the
delta requires a pricing model, which may differ from the general risk model. Third,
the resulting mapping becomes rapidly unrealistic if the market moves, as the delta
of the position changes very rapidly.
While the concept of Value-at-Risk is straightforward, its implementation is not.
For the same portfolio, as evidenced by Beder (1995) or Hendricks (1996), there is a
variety of models that will produce very different estimates of risk. Surprisingly, for
the same model, there are also a large number of implementations that produce also
very different estimates of risk. For instance, Marshal and Siegel (1997) assess the
variance in Value-at-Risk estimates produced by different implementations9 of the
RiskMetricsTM parametric model. For a linear portfolio, the Value-at-Risk estimates
range from US$2.9 million to US$5.4 million around a mean of US$4.2 million. For
a portfolio with options, they obtain a range from US$0.75 to US$2.1 million around
a mean of US$1.1 million. With the increasing optionality and complexity of the new
products, the divergence of answers produced by the various models should increase
in the future.