Growing Pains
Monte Carlo, that Old Faithful of financial analytics,
is more widely available today than ever before. But here's the catch: its
longstanding limitations are more worrisome in today's maturing markets.
By Karen Spinner
Monte Carlo analysis has been a reliable standby technique for both dealers and users of complex derivatives. Today it has two critical uses at most
financial institutions. First, it is an important valuation tool for financial
instruments that are either highly structured or include very complex prepayment
assumptions. As a rule, if an instrument is too complex to price with closed
formulae such as the BlackScholes model and if it is resistant to lattice
techniques, then it may be appropriate to turn to Monte Carlo for a price.
Indeed, Monte Carlo, which simply refers to a sophisticated random number
generator, has been around for years as the pricing methodology of last
resort.
Monte Carlo analysis also plays an important role as a risk management
tool. Typically it is used to generate scenarios that can then be used in
a valueatrisk (VAR) calculation to determine the maximum variance of the
portfolio, to a certain level of confidence, over a specific time horizon.
Monte Carlo, along with historical and parametric approaches, is one of
three popular ways to generate VAR scenarios.
But ironically, Monte Carlo's nearubiquity on trading desks and in risk management departments has coincided with a growing awareness of its limitations.
These include less than perfect accuracy, the vast number of trial runs
required to obtain a reasonably accurate result and the length of time it
can take, particularly in the case of large portfolios, to process.
1001 uses for Monte Carlo
Monte Carlo is, of course, an extremely flexible and useful technique.
It is the pricing court of last resort, and to date it remains the pricing
methodology of choice for CMOs and related instruments. Although most experts
recommend that Monte Carlo analysis be used only when no other, more precise
methodology is available, it can provide a quickanddirty price estimate
for most instruments. Bob Geske, vice president at C*ATS, says that Monte
Carlo is particularly useful in cases where more than one deterministic
solution exists and there is disagreement over which solution is more correct.
It has also caught on as a method for determining probable portfolio variance
without resorting to either pure historical data or a parametric assumption
that market factors are normally distributed.
And today Monte Carlo analysisthe derivatives valuation jackofalltradesis easier and less expensive to obtain than ever before. It is available as
a standard feature in many sophisticated spreadsheet addins. Says George
Williams, a principal at Kalotay and Associates, a consultancy which specializes
in the development of sophisticated pricing models for interest ratebased
instruments: "The temptation of Monte Carlo is that it is so powerful,
yet so easy to program. You can whip up a Monte Carlo generator in a single
afternoon that can give you a quick and very reasonable price estimate of
some very complex financial instruments."
Finally, in a way Monte Carlo simulation is the mother of all models.
Russel Caflisch, professor of mathematics at the University of California
in Los Angeles, explains that Monte Carlo is an extremely useful tool for
validating new pricing models under a multitude of "random" market
conditions.
The drawbacks
But just as Monte Carlo has definite strengths, the technique also possesses some considerable weaknesses which, perhaps, have become more significant
now that the technique is so widely used. Here are some of its wellknown
limitations:
General inaccuracy. As a valuation tool, Monte Carlo typically
works as follows. Start by generating random market factorssuch as interest
rates and currenciesbased on their historical distributions. Feed each
set of randomly generated data into a model which is designed to predict
the behavior of the instrument in question based on changes in these market
factors, and this model spits out a price. This cycle continues over and
over again, until a statistical distribution of prices is created. The mean
of this distribution is then taken as the instrument's approximate price.
The accuracy of the results will depend on the number of times the simulation is run, but there is no perfectly linear relationship between the number
of Monte Carlo sequences performed and the degree of pricing accuracy achieved.
Instead, the degree of error in any Monte Carlo simulation is inversely
proportional to the square root of the iterations used. This means that
by increasing the number of iterations from 100 to 1,000, there won't be
a tenfold improvement in accuracy. And importantly, the larger the number
of iterations already run, the larger the number of additional runs needed
to obtain substantial improvements in accuracy.
And Williams points out that there is no such thing as a perfectly accurate Monte Carlobased price. He says: "Monte Carlo cannot be perfectly
calibrated to today's markets. Consider the example of three optionfree
cash securities: a floating rate note, a swap and a fixed rate bond. From
a structural perspective, the note and the swap can be combined to create
the bond and, if you have available market prices for the note and the bond,
you can solve the following for the price of the swap:
FLOATING RATE NOTE + SWAP = FIXED RATE BOND
"Suppose," says Williams, "that the market tells me that
the bond is a par bond. Suppose further that the floating rate note resets
at the fair shortterm rate so that it is also priced at par. In this case
the value of the swap should be zero, but Monte Carlo will provide a slighly
different answer. Of course, Monte Carlo analysis will misprice the bond
and the note as well so that the equation cited above will work out under
Monte Carlo, although the individual terms will be wrong."
The greatest danger, concludes Williams, would arise from combining another technique with Monte Carlo.
C*ATS's Geske, however, explains that this calibration problem is not
limited to Monte Carlo analysis. Many instruments rely on "implied"
market factors. For example, in the case of options, volatilities may be
calculated by obtaining current market prices and then solving Black
Scholes or a similar model for vega. If the equation was solved for the
"implied price" of the swap above and then a deterministic equation
was used to solve for the price of the swap, the answers would be slightly
different, as in the case of Monte Carlo.
Calibration errors can be more severe in the case of Monte Carlo. If,
for instance, the closedform valuation formula included an implied forward
rate, then the valuation may be off to the extent that the implied forward
rate is off, says Geske. Likewise, if a binomial or trinomial model is used,
where implied forward rates are assigned to a finite number of nodes, the
cumulative discrepancies in those forward rates will have a slightly larger
impact on the final price. But under Monte Carlo analysis, the imperfect
forward rate is used thousands of times with a result, depending on the
extent to which the discrepancies cancel themselves out, that can have an
even more significant effect on the final price.
Processing speed. In order to get extremely accuratethough by
no means perfectresults from Monte Carlo, it is necessary to run a lot
of simulations. Consider the findings of Jonathan Berk and Richard Roll
of Goldman Sachs, who required 10,000 iterations in order to value one adjustable
rate mortgage to a single standard deviation. Faced with a whole book full
of adjustable rate mortgages, it doesn't take a systems expert to see that
this process will take a very, very long time.
The same basic principle holds true when using Monte Carlo analysis to
determine the likely variance of a portfolio. According to Dan Rosen, a
senior financial engineer for software firm Algorithmics, one of the most
critical drawbacks to Monte Carlo as used on a portfolio basis is processing
speed. Consider running a valueatrisk calculation on a portfolio comprised
of 20,000 swaps using 5,000 randomly selected sets of market factors. This
kind of task, says, Rosen, becomes very difficult to manage in real time.
Of course, the amount of time such a problem would take to process depends on the sort of market factors chosen. For example, if on every Monte Carlo
pass, multiple, manyfactored yield curves are generated in different currencies,
then the problem will be more complex than if a concerted attempt is made
to simplify these market factors, thus reducing the dimensionality. And
C*ATS's Geske stresses that instruments within a portfolio being analyzed
for risk management purposes would necessarily have to be valued by a deterministic
approach. "Monte CarlowithinMonte Carlo" is not viable.
"Specific" inaccuracy. While Monte Carlo simulation
is in a sense "always" slightly inaccurate, there are some instruments
for which Monte Carlo cannot produce a meaningful result. Consider the classic
case of the Americanstyle option.
An Americanstyle option can be exercised at any time during its life,
and so valuation must include some sort of earlyexercise model to predict
when, exactly, the investor is most likely to exercise the option. Says
Williams, "To decide whether to exercise an option, one must know two
things: the benefit realized and the value forfeited by exercise. For European
options, this is trivial. But American options have an incremental time
value beyond their intrinsic exercise value, and an option should only be
exercised when its time value is dominated by the exercise valuesay, at
least ten to one."
The problem here, according to Williams, is that Monte Carlo can only
approximate the incremental time value left on the option at exercise. He
says, "Monte Carlo works okay for discounting known cash flows, as
long as it is used consistently; however, it should not be used to make
the decisions required in managing options." Emmanuel Fruchard, head
of financial engineering for Summit Software, concurs: "Monte Carlo
has problems with American options because these methods are highly dependent
on users' assumptions relating to under what circumstances the option will
be exercised, and thus provide a less than optimal tool for decisionmaking."
Of course Monte Carlo would never be the valuation model of choice for
Americanstyle options. C*ATS's Geske says, "In most cases, everyone
agrees on the earlyexercise function and, generally speaking, it is not
all that significant." He stresses that Monte Carlo is typically used
in cases where it is impossible to accurately predict a prepayment function,
as in the case of CMOs, where the forces of "QFRD" (Quit, Fire,
Retire and Die) make such a prediction too complex to be represented by
simple deterministic formulae.
And they're getting worse
The classic drawbacks to Monte Carlo analysis described above are certainly no wellkept secret. Financial engineers have known about these issues for
years. The reasons these drawbacks have become more significant today is
that the derivatives markets are maturing and users are becoming more technologically
proficient. Consider the following:
Tiny spreads. Over the past five or so years, the derivatives
markets have to a large extent become commoditized. This means that swaps,
for example, which used to yield 80 or 100 basis points in profits to the
typical dealer, may now only yield, say, five basis points. And this drastic
reduction in profit margins has also affected more complex instruments.
Says Williams: "Monte Carlo pricing is no longer an adequate approximation
now that the markets have tightened. Even small variations in price are
much more critical now."
In other words, the calibration errors described earlier, which can be
more severe in Monte Carlo analysis, become more critical when there is
little margin for error. And although it is possible to eventually reduce
the error in Monte Carlo simulations to a virtually negligible level, the
amount of work required to do this can be prohibitive.
Operational risk. The widespread availability of Monte Carlocapable software has increased the probability that it may be used improperly. A
considerable danger exists that systems may not provide a full, obvious
disclosure of Monte Carlo's limitations. Dan van Deventer, president of
Japanbased Kamakura, a risk management consulting firm, provides the following
scenario.
Consider an asset/liability manager who regularly marks his balance sheet to market using Monte Carlo analysis. Let's say he runs 200 iterations and
his balance sheet is valued at 100. Any competently written software will
inform him of his sampling error, which in this case can be expressed by
stating that there is a two standard deviation variance around the portfolio's
"mean" of 100; in other words, between 98 and 102. However, if
the software does not display this fact, then the asset manager could blithely
assume his valuation is more accurate than it really is.
This kind of basic misconception can cause problems with hedging decisions. For example, many risk managers use the concept of delta in order to determine
appropriate hedges for their exposures. "Delta" is the projected
change in a security's price in response to a change in a market factor.
Using Monte Carlo to arrive at the delta for a portfolio can be particularly
dangerous because of sampling error, says van Deventer. It is very easy
to get a "long" delta where, in reality, the position is "short"
delta, and vice versa. On the basis of this faulty information, a risk manager
can easily enter into a hedge that exacerbates the problem rather than ameliorating
it.
Solutions, alternatives for valuation...
Given the potential pitfalls of Monte Carlo, coupled with a market characterized by constantly narrowing margins, it is not surprising that many researchers
are working on alternatives to Monte Carlo in the pricing arena. Consider
the following popular alternatives:
Closedform equations. According to van Deventer, one promising
avenue for alternatives to Monte Carlo is to be found in "finite difference"
methods, particularly for products that incorporate Americanstyle options.
This process is very flexible and is consistent with a wide variety of statistical,
market factor distributions, says van Deventer. Thus, using finite difference
methods, a risk manager does not have to assume that market factors are
normally or lognormally distributed.
Multifactor models. From simple, onefactor binomial models to
highorder polynomial approximations, multifactor models are considered
the secondbest way to value derivative instruments. Typically, these methods
involve valuing the instrument in question at a finite number of nodes to
which predetermined market factors are assigned.
Of these techniques, the simplestand therefore the most efficientis
the onefactor model. Sometimes known as "recursive," "lattice"
or "bushy tree" techniques, these models track one critical market
factor from the present to the instrument's maturity and back again.
Onefactor models rely on either binomial or trinomial trees. A binomial tree is a discrete time model that describes the movement of a single random
variable whose movement at each "node" in the tree can be expressed
as an upward or downward movement with a known probability, explains van
Deventer. Since each node has two branches, it is simple enough to predict
the values of all the nodes on the tree. To price a relatively simple instrument,
whose price is largely derived from a single factor, a manager needs to
travel "up" the tree, into the future, to generate market prices
for each node, and back "down" the tree to arrive at values for
the derivative instrument at each node. Probabilities are then used to arrive
at a final price.
Trinomial trees, unsurprisingly, have three branches: one representing
"up," one representing "down" and one representing "stays
the same." In the case of interest rates, three parameters are typically
used: shortterm rate bias, known as "drift"; volatility; and
mean reversion, which represents the probability that shortterm interest
rates will be pulled back to a historical mean value.
These onefactor models often, however, have substantial drawbacks. For
example, while a trinomial tree might do well in pricing a relatively simple
swap or bond, what about a convertible bond or a currency swap? Many shops
are at work on effective twofactor, bushytreetype models that can handle
multiple market factors, but these tend to be quite complex and arduous
to program.
Beyond the one and twofactor models are highorder polynomial or "finite difference" techniques, which although they are more complex than the
lattice techniques, still avoid the randomness associated with Monte Carlo.
These methods use grids from which solutions to the partial differential
equation that describes a particular instrument's price can be obtained.
This process is very flexible, says van Deventer, and is consistent with
a wide variety of statistical market factor distributions. Thus, using finite
difference methods, it isn't necessary to assume that market factors are
normally or lognormally distributed.
These methods do require that a distribution pattern is chosen for the
market factors. Says Geske, "Your price will be correct only if you
have chosen the optimal statistical distribution."
Trades which are exposed to more than two risk factors cannot be effectively priced using a latticebuilding technique, counters Fruchard of Summit Software.
For example, a European swaption on a currency swap would involve exposure
to multiple interest rates, yield curves and currencies. With this many
dimensions, any latticetype framework would have too many nodes to evaluate
efficiently.
Still, Williams explains that recursive and finitedifference techniques can replace Monte Carlo as the pricing methodology of choice for many sorts
of path dependent securities, including index amortizing swaps and adjustablerate
mortgages. The one sticky point, he explains, comes with the path dependent
nature of CMOs. The need to model the interactions among the multiple tranches
of the underlying pool of mortgages, coupled with prepayment models driven
in part by some moving average of rates, leads to computational demands
that seem to render recursive valuation unsuitable to the task. "But,"
he says, "We're working on it."
"Deterministic," "quasirandom," "low discrepancy" sequences. Another techniquewhich is essentially a "tweaked"
version of Monte Carlo analysisis known variously as "deterministic
simulation," "quasirandom selection" and "lowdiscrepancy
sequences." Basically, this technique takes the "random"
out of Monte Carlo by providing rules which guide how each market factor
is generated. Quasirandom selection can correct for the bunching of market
factors which frequently happens with classic Monte Carlo, says UCLA's Caflisch.
(See chart.)
This tendency for randomly generated numbers to clump means that vast
numbers of iterations may be required to correct for this clumping. Quasirandom
sequencing provides a smoother distribution much earlier on, thus reducing
the number of iterations required for a reasonably accurate measure. According
to Caflisch, the error in a quasirandom simulation is inversely proportional
to the number of sequences run. Relative to Monte Carlowhere error is inversely
related to the square root of the total number of iterationsquasirandom
simulation can provide an acceptable answer much faster than Monte Carlo.
And, in fact, many firms are looking into the potential applications
of deterministic simulation. Torontobased Algorithmics, which offers sophisticated
models and risk management technology, is exploring potential uses for low
discrepancy sequences. Algorithmics is conducting a joint project with University
of Toronto's RiskLab to identify where the use of nonrandom sequences will
likely be most productive.
But while quasirandom simulation sounds great in theory, it has a lot
of drawbacks. Remember IBM's announcement last year that a new breakthrough
methodology would revolutionize how banks priced their path dependent securities?
(See Derivatives Strategy, December 1995.) Despite the fanfare, Big Blue
is still bogged down in the testing phase. Although quasirandom methods
can dramatically speed the pricing of certain relatively simple CMOs, the
technique tends to break down when used for more complex instruments.
This type of deterministic simulation has two key drawbacks. First, the
dramatic improvements in processing speed that quasirandom methods can
deliver on relatively simple problems do not translate into the same sorts
of improvements on complex problems because complex problems tend to have
a high "dimensionality," says Caflisch. Dimensionality refers
to the number of variables which much be independently simulated during
each "pass." Caflisch explains that, in a quasirandom sequence,
it is necessary to create a separate set of rules for each dimension. The
rules specified for each dimension must be independent of the rules specified
for the other dimensions and, for problems with hundreds of dimensions,
this can be quite challenging. The rule of thumb, then, is that the more
complex the problem, the less improvement in speed quasirandom methods
will provide over standard Monte Carlo.
Furthermore, it is very difficult to pin down, exactly, the degree of
error present in quasirandomgenerated pricing results. Caflisch explains
that while the exact standard deviation associated with a Monte Carlo sequence
can be determined, that is not possible with quasirandom. This point alone
is enough to make most bankers a little edgy.
"Nevertheless," says Caflisch, "it is important to keep
in mind that while quasirandom sequencing is not appropriate for all problems,
they have a great deal of potential for pricing both simple and moderately
complex instruments. The trick will be to isolate the most critical variables
in any problem and then to minimize the total number of variables that must
be processed."
Solutions and alternatives for risk management
Monte Carlo analysis is just one of three techniques that can be used
to generate scenarios in the context of calculating valueatrisk. Of the
two alternative methods, Geske says that historical simulation is used more
frequently as a supplement to other forms of analysis because it is widely
recognized that history is not necessarily the best predictor of future
market behavior. Furthermore, utilizing large quantities of historical data
does not necessarily yield great advantages in terms of processing time
over Monte Carlo methods. The other alternative, parametric simulation,
assumes that market factors are normally distributed and uses correlation
and volatility to predict market prices. The biggest advantage of this method
is the speed with which a value can be obtained. Data for this simulation
is easily available from several sources, including the RiskMetrics web
site.
Monte Carlo, because it neither assumes a particular distribution of
market forces nor relies on historical data, is considered by many to be
a more rigorous test of a portfolio's likely future behavior, given uncertain
future market conditions. But the biggest drawback remains the time it can
take to process a large portfolio under Monte Carlo.
The goal of many researchers then is not to replace Monte Carlo as a
technique for generating risk scenarios, but rather to find new ways of
representing the portfolio and other market factors so they can be easily
processed through Monte Carlo.
There are two relatively accessible techniques that could be used to
make Monte Carlo more efficient as an engine for VAR and other portfoliowide
analytics. The first is portfolio compression, wherein an index is created
which imitates the behavior of a very large portfolio. This portfolio replication
can be accomplished using, among other tools, Algorithmics' patented scenario
optimization technique. Rosen explains that the replicated portfolio need
not behave like the large original portfolio forever and under all conditions.
Instead, it should imitate the portfolio under a small, representative set
of likely states over the time horizon over which VAR is being calculated.
Algorithmics already supports a limited use of this technique. Offline,
the larger portfolio is compressed over a small set of scenarios. Then the
compressed portfolio, which consists of a small number of simple instruments,
can be used for intraday calculations, even if some market conditions change.
This way the replicated portfolio captures the behavior of complex instruments
without incorporating their pricing complexities. For example, the changes
in the value of exotic and pathdependent options may be captured by an
optimal combination of simpler plain vanilla instruments.
The second method is to simplify or compress the horizon of risk factors over which Monte Carlo sampling is performed prior to running the simulation.
This can be accomplished by applying a transformation process that directly
incorporates information on the target portfolio. The result is a more focused
sampling process and a faster valuation of the portfolio.
This technique is similar to traditional principal component analysis
(PCA) in some respects. But while standard PCA applied to scenario generation
would focus on capturing the variability of the risk factors, the components
obtained through the compression technique are optimally chosen to reflect
the portfolio's behavior. Moreover, the resulting functional relationship
between the portfolio and the new risk factors is much simpler. Although
this method requires some analytical work, the combination of a smaller
simulation space and a simpler valuation can result in a considerable speed
enhancement over standard Monte Carlo.
Pulling it all together...
So, despite the growing pains, Monte Carlo analysis is a basic building
block of financial analytics. And as pricing technology becomes increasingly
sophisticated, Monte Carlo's most important function may be as a risk management
tool and a mechanism for validating new, more selfcontained models.
Dos and Don'ts
Do...
 Use Monte Carlo to price instruments for which no other method is readily
available
 Make sure you know how accurate your Monte Carlo analysis really is
 Select software that provides clear, easytounderstand information
on accuracy
 Use Monte Carlo to validate pricing models
 Use Monte Carlo to obtain portfoliowide value at risk
Don't...
 Price instruments with Monte Carlo if other more accurate methods are
available
 Price instruments with Monte Carlo "because it comes with my software
package"
 Cut down on the number of iterations to save time
 Use Monte Carlo to price instruments with complex optionality without
considering the dangers
