Model risk: towards a definition

Most economic and financial models fall into one of the following categories:
Ω Macro-economic models typically attempt to forecast macro-economic data such
as interest rates, inflation, unemployment rate, etc. Their complexity ranges from
naive single-equation models to models with several hundred equations and
thousands of variables.

Ω Micro-economic models attempt to explain relationships in a given market, such
as the link between savings and past returns, or between interest rates and
consumption. They are generally simple, not necessarily true, but provide a useful
picture and an understanding of reality.

Ω Valuation models are typically used by investors to select their portfolios, by
traders to price financial instruments and by banks to determine their investment
strategy and asset-allocation mix. They range from very simple models based on
discounting and compounding cashflows to more elaborate ones, typically those
used for exotic options or complex swaps pricing.

Ω Risk models attempt to estimate how the value of a given position will change due
to a particular change in its environment. This change can find its source in a
general market change (interest rates, stock market, commodity prices, etc.) or
from a position-specific characteristic (credit risk, operational risk, etc.). Typical
examples of risk models are hedging strategies and Value-at-Risk calculations.
In the following, we will focus mainly on models belonging to the third and fourth
categories, but we could easily extend our framework to include macro- and microeconomic
models.

Model risk results from the use of an inappropriately specified model, or the use
of an appropriate model but in an inadequate framework or for the wrong purpose.
Uncertainty in a financial model arises from four sources. First, there is an inherent
uncertainty due to the stochastic nature of a given model. By definition, market
moves are not totally predictable, which means that any financial model is subject
to this uncertainty. Next, there is the uncertainty about the applying a model in a
specific situation, given the model structure and its parameter estimation. Can we
use the model extensively? Is it restricted to specific situations, financial assets or
markets? One should carefully examine these points before the use of any model.
Third, there is an uncertainty in the values of the parameters in a given model.
Statistical estimates (particularly for non-observable quantities such as volatility or
correlation) are subject to errors, and can therefore lead to errors when they serve
as an input. Finally, there is the specification error, that is, the uncertainty of the
model structure itself. Did we really specify the right model? Even after the most
diligent model selection process, we cannot be sure that we have adopted the true
model, if any.

Generally, only the first source of uncertainty receives rigorous attention. The
others are simply ignored. The values of the parameters and the underlying model
are generally assumed to be the true ones. At best, a limited sensitivity analysis is
performed with respect to the parameters.