Fees and commissions

When a trade is carried out, a fee may be payable to a broker, or a spread may be
paid relative to the mid-market price of the security or contract in question. Typically,
in a market making operation, fees will be received, and spreads will result in a
profit. For a proprietary trading desk, in contrast, fees would usually be paid, and
spreads would be a cost. In some cases, fees and commissions are explicitly stated
on trade tickets. This makes it possible to separate them from other sources of profit
or loss. Spreads, however, are more difficult to deal with. If an instrument is bought
at a spread over the mid-price, this is not generally obvious. The price paid and the
time of the trade are recorded, but the current mid-price at the time of the trade is
not usually available. The P&L from the spread would become part of intraday P&L,
which would not impact clean P&L. To calculate the spread P&L separately, the midprice
would have to be recorded with the trade, or it would have to be calculated
afterwards from tick-by-tick security price data. Either option may be too onerous to
be practical.
Fluctuations in fee income relate to changes in the volume of trading, rather than
to changes in market prices. Market risk measures give no information about risk
from changes in fee income, therefore fees and commissions should be excluded
from P&L figures used for backtesting.

Dirty or raw P&L

As noted above, P&L calculated daily by the business unit control or accounting
department usually includes a number of separate contributions.

Profit and loss calculation for backtesting

When market risk is calculated, it gives the loss in value of a portfolio over a given
holding period with a given confidence level. This calculation assumes that the
composition of the portfolio does not change during the holding period. In practice,
in a trading portfolio, new trades will be carried out. Fees will be paid and received,
securities bought and sold at spreads below or above the mid-price, and provisions
may be made against possible losses. This means that P&L figures may include
several different contributions other than those related to market risk measurement.
To compare P&L with market risk in a meaningful way, there are two possibilities.
Actual P&L can be broken down so that (as near as possible) only contributions from
holding a position from one day to the next remain. This is known as cleaning the
P&L. Alternatively, the trading positions from one day can be revalued using prices
from the following day. This produces synthetic or hypothetical P&L. Regulators
recognize both these methods. If the P&L cleaning is effective, the clean figure should
be almost the same as the synthetic figure. The components of typical P&L figures,
and how to clean them, or calculate synthetic P&L are now discussed.

Comparison process

Risk reports are based on end-of-day positions. This means that the risk figures give
the loss at the chosen confidence interval over the holding period for the portfolio that
is held at the end of that business day. With a 1-day holding period, the risk figure
should be compared with the P&L from the following business day. The P&L, if
unwanted components are removed, gives the change in value from market movements
of the portfolio the risk was measured for. Therefore, the risk figures and P&L figures
used for comparison must be skewed by 1 business day for meaningful backtesting.

Comparing risk measurements and P&L

Holding period
For regulatory purposes, the maximum loss over a 10-business-day period at the
99% confidence level must be calculated. This measurement assumes a static
portfolio over the holding period. In a realistic trading environment, however, portfolios
usually change significantly over 10 days, so a comparison of 10-day P&L with
market risk would be of questionable value. A confidence level of 99% and a holding
period of 10 days means that one exception would be expected in 1000 business
days (about 4 years). If exceptions are so infrequent, a very long run of data has to be
observed to obtain a statistically significant conclusion about the risk measurement
model. Because of this, regulators require a holding period of one day to be used for
backtesting. This gives an expected 2.5 events per year where actual loss exceeds
the market risk figure. Figure 9.2 shows simulated backtesting results. Even with
this number of expected events, the simple number of exceptions in one year has
only limited power to distinguish between an accurate risk measurement model and
an inaccurate one.
As noted above, risk figures are often calculated for a holding period of 10 days.
For backtesting, risks should ideally be recalculated using a 1-day holding period.

For the most accurate possible calculation, this would use extreme moves of risk
factors and correlations based on 1-day historical moves rather than 10-day moves.
Then the risk figures would be recalculated. The simplest possible approach is simply
to scale risk figures by the square root of 10. The effectiveness of a simple scaling
approach depends on whether the values of the portfolios in question depend almost
linearly on the underlying risk factors. For instance, portfolios of bonds or equities
depend almost linearly on interest rates or equity prices respectively. If the portfolio
has a significant non-linear component (significant gamma risk), the scaling would
be inaccurate. For example, the value of a portfolio of equity index options would
typically not depend linearly on the value of the underlying equity index. Also, if the
underlying risk factors are strongly mean reverting (e.g. spreads between prices of
two grades of crude oil, or natural gas prices), 10-day moves and 1-day moves would
not be related by the square root of time. In practice, the simple scaling approach is
often used. At the whole bank level, this is likely to be reasonably accurate, as
typically the majority of the risk of a whole bank is not in options portfolios. Clearly,
this would not be so for specialist businesses such as derivative product subsidiaries,
or banks with extensive derivative portfolios.

Backtesting

MARK DEANS
The aim of backtesting is to test the effectiveness of market risk measurement by
comparing market risk figures with the volatility of actual trading results. Banks
must carry out backtesting if they are to meet the requirements laid down by the
Basel Committee on Banking Supervision in the Amendment to the Capital Accord to
incorporate market risks (1996a). If the results of the backtesting exercise are
unsatisfactory, the local regulator may impose higher capital requirements on a
bank. Further, when performed at a business line or trading desk level, backtesting
is a useful tool to evaluate risk measurement methods.

Backtesting is a requirement for banks that want to use internal models to calculate
their regulatory capital requirements for market risk. The process consists of comparing
daily profit and loss (P&L) figures with corresponding market risk figures over a
period of time. Depending on the confidence interval used for the market risk
measurement, a certain proportion of the P&L figures are expected to show a loss
greater than the market risk amount. The result of the backtest is the number of
losses greater than their corresponding market risk figures: the ‘number of exceptions’.
According to this number, the regulators will decide on the multiplier used for
determining the regulatory capital requirement.
Regulations require that backtesting is done at the whole bank level. Regulators
may also require testing to be broken down by trading desk (Figure 9.1). When there
is an exception, this breakdown allows the source of the loss to be analysed in more
detail. For instance, the loss might come from one trading desk, or from the sum of
losses across a number of different business areas.
In addition to the regulatory requirements, backtesting is a useful tool for evaluating
market risk measurement and aggregation methods within a bank. At the whole
bank level, the comparison between risk and P&L gives only a broad overall picture of
the effectiveness of the chosen risk measurement methods. Satisfactory backtesting
results at the aggregate level could hide poor risk measurement methods at a lower
level. For instance, risks may be overestimated for equity trading, but underestimated
for fixed income trading. Coincidentally, the total risk measured could be approximately
correct. Alternatively, risks could be underestimated for each broad risk
category (interest rate, equity, FX, and commodity risk), but this fact could be hidden
by a very conservative simple sum aggregation method.
Backtesting at the portfolio level, rather than just for the whole bank, allows individual market risk measurement models to be tested in practice. The lower the
level at which backtesting is applied, the more information becomes available about
the risk measurement methods used. This allows areas to be identified where market
risk is not measured accurately enough, or where risks are being taken that are not
detected by the risk measurement system.
Backtesting is usually carried out within the risk management department of a
bank where risk data is relatively easily obtained. However, P&L figures, often
calculated by a business unit control or accounting department, are equally important
for backtesting. The requirements of these departments when calculating P&L
are different from those of the risk management department. The accounting principle
of prudence means that it is important not to overstate the value of the portfolio, so
where there is uncertainty about the value of positions, a conservative valuation will
be taken. When backtesting, the volatility of the P&L is most important, so capturing
daily changes in value of the portfolio is more important than having a conservative
or prudent valuation. This difference in aims means that P&L as usually calculated
for accounting purposes is often not ideal for backtesting. It may include unwanted
contributions from provisions or intraday trading. Also, the bank’s breakdown of P&L
by business line may not be the same as the breakdown used for risk management.
To achieve effective backtesting, the risk and P&L data must be brought together
in a single system. This system should be able to identify exceptions, and produce
suitable reports. The data must be processed in a timely manner, as some regulators
(e.g. the FSA) require an exception to be reported to them not more than one business
day after it occurs.
In the last few years, investment banks have been providing an increasing amount
of information about their risk management activities in their annual reports. The
final part of this chapter reviews the backtesting information given in the annual
reports of some major banks.

Acknowledgements

Certain sections of this chapter were drawn from Implementing Value at Risk, by
Philip Best, John Wiley, 1998. John Wiley’s permission to reproduce these sections
is kindly acknowledged.
The author would also like to thank Con Keating for his invaluable assistance in
reviewing this chapter and for writing the appendix on Extreme Value Theory. This
chapter also benefited from the comments of Gurpreet Dehal and Patricia Ladkin.

Notes
1 Note that observing other market parameters, such as the volatility of short-term interest
rates, might have warned the risk manager that a currency devaluation was possible.
Observed by a central risk management function in a different country, however, the chances
of spotting the danger are much reduced.
2 That is, twenty times the return volatility prior to the crisis.
3 Z score of binomial distribution of exceptions: 1.072, i.e. the VaR model would not be rejected
by a Type I error test.
4 Extreme price changes that have an almost infinitesimally small probability in a normal
distribution but which we know occur with far greater regularity in financial markets.
5 For a more comprehensive coverage of EVT see Embrechs et al. (1997).
6 This is the number of ways of selecting n assets from a set of 10, all multiplied by the
number of scenarios – 69 for this example.
7 Counterparty A’s liquidators would expect the bank to perform on the contracts, thus their
value at the time of default would have to be written off. Once written off, of course, there is
no potential for future beneficial market moves to improve the situation.
8 Bond price curvature – the slight non-linearity of bond prices for a given change in yield.
9 Note that this is not the same as the group of countries who have chosen to ‘peg’ their
currencies to the US dollar.
10 For a more formal and complete introduction to EVT see Embrechs et al. (1997), Reiss and
Thomas (1997) and Beirlant et al. (1996). Readers interested in either the rapidly developing
multivariate theory or available software should contact the author. 261