Comparing Addin Accuracy
Our intrepid modeler puts eight packages through their paces—and finds some significant pricing differences.
By William Margrabe
My 10day odyssey through an often mysterious and sometimes frustrating world of computer hardware, Windows 95, Excel and packages of addins is over. While we wait for a modernday Homer to chronicle my exploits in verse, this article summarizes how I navigated the perils on that trip, and how you might want to deal with the same issues.
My original plan was to design a diverse portfolio of equity, currency, commodity and fixedincome derivative products; load all the addins simultaneously; price each product with the relevant function from each package; and compare the results. That plan was too ambitious—like planning to climb Mount Everest, solo, one day after lunch, without oxygen.
In the time I had allotted for this project, I managed to price 10 deals with up to eight of the packages and my proprietary software. I will report in detail on only three relatively simple trades: a European put option, an American put option and a European put on a spread with a nonzero strike. For reasons that I hope will become obvious, fixedincome derivatives must wait for another day.
Coaxing the addins to give the same value for a price or Greek wasn’t nearly as easy as I had anticipated—even when they would eventually agree to several decimal places. I kept on investigating, however, until all the packages sang the same tune or until I knew why they didn’t.
The differences came from three main sources: inputs, algorithms and outputs.
The eight packages interpreted a given input rate of interest or dividend yield three different ways. One package wouldn’t accept dividend yield as input.
The eight packages had at least three different algorithms for converting input dates into years to expiration. While all eight packages appeared to use the BlackScholesMerton model for pricing European puts, they differed visibly on some of the Greeks. They used four algorithms for pricing American puts, and eight algorithms to price the spread option. Sometimes the algorithms weren’t what the documentation said. Sometimes the documentation didn’t say.
There were also some significant differences in the methods of reporting sensitivities to interest rates. Some vendors reported more sensitivities—sometimes many more—than others. Some provided sensitivity to a move in an annual percentage rate, while others assumed a continuously compounded rate was moving. Sometimes the sensitivities weren’t as documented.
The European put
We would all expect a fair amount of agreement on the pricing of European options for three reasons: European options have been around for a long time; the standard “closedform” model for pricing them has been available for more than 25 years; and numerical algorithms that the BlackScholesMerton model requires have been around for decades.
My initial results for premium and Greeks appear in Table 1. The different packages—anonymous at this point—did not all produce the same values. Five packages agree that the premium should be 5.5913. Two agree on 5.6049. One says 5.5886. That raised questions: How can we quantify the disagreement? Why don’t all the addins agree to four significant figures on the premium and all the Greeks? Do “right answers” exist? If so, what are they? How much disagreement should we tolerate? Answering all these questions completely took several days.
Standard deviation—slightly more than one part in a thousand for premium—is a way to measure variability of results across vendors. The variability of sensitivity to interest rate and dividend yield is much larger. The apparent frequent lack of a number describing sensitivity to a change in dividend yield is startling.
To answer the remaining questions, I studied the documentation—and spoke with experts, where required—for clues about models, numerical algorithms and definitions of inputs and outputs.
Inputs
Table 1 
European Put Option Value and Greeks 
Model 
Premium 
Delta 
Gamma 
Kappa 
Rho_Div 
Rho_Int 
Theta 
FEA 
5.5913 
0.4523 
0.0255 
51.0503 
90.4540 
101.6367 
0.9967 
FinCAD 
5.5913 
0.4523 
0.0255 
51.0500 

96.6840 
0.9967 
Intermark 
5.5913 
0.4523 
0.0255 
51.0500 

101.6404 
0.9967 
Marvin 
5.5886 
0.4577 
0.0255 
50.8849 

102.8093 
1.0468 
MBRM 
5.5913 
0.4523 
0.0255 
51.0500 

101.6404 
0.9967 
Monis 
5.6049 
0.4534 
0.0256 
51.1737 
86.3596 
97.0355 
1.0059 
Savvysoft 
5.6049 
0.4534 
0.0256 
51.1737 
86.3596 
97.0355 
1.0059 
Tech Hackers 
5.5913 
0.4523 
0.0255 
51.0500 
90.4584 
101.6410 
0.9967 
Mean 
5.5944 
0.4532 
0.0255 
51.0603 
88.4079 
100.0154 
1.0053 
Std.Dev. 
0.0065 
0.0019 
0.0000 
0.0902 
2.3652 
2.5972 
0.0173 
Different vendors define and handle inputs differently. Sometimes, this is inconsequential. For example, some systems refer to an option’s “maturity,” rather than its “expiration.” Some systems accept time to expiration in years, others accept a value date and an expiration date, and some want or allow other inputs.More significant were differences in handling interest rate and dividend yield. The FEA, Intermark, Marvin and MBRM systems take their inputs the way I happen to be used to inputting them—as continuously compounded annual rates (such as 0.05). FinancialCAD and Savvysoft, however, want to receive dividend yield and interest rate as annual percentage rates (such as 0.05127 = exp(0.05)—1). The Monis system requires APRs to be expressed as percents (such as 5.127, rather than 0.05127). The Tech Hackers system asks for a continuously compounded interest rate, but expects a continuously compounded “holding cost”—the cost of carry—that equals the interest rate minus the dividend yield. After correcting for my misunderstanding about conventions for dividend yield and interest rate, the premium values for all but one package agreed to four decimal places. Two days later, I learned how to input time properly for the remaining package, and all premium values agreed to four decimal places, as in Table 2.
Table 2 
European Put Option Value and Greeks—Revised 
Model 
Premium 
Delta 
Gamma 
Kappa 
Rho_Div 
Rho_Int 
Theta 
FEA 
5.5913 
0.4523 
0.0255 
51.0503 
90.4540 
101.6367 
0.9967 
FinCAD 
5.5913 
0.4523 
0.0255 
51.0500 
91.3630 
101.6411 
0.9967 
Intermark 
5.5913 
0.4523 
0.0255 
51.0500 

101.6404 
0.9967 
Marvin 
5.5913 
0.4523 
0.0255 
51.0503 

101.6411 
0.9967 
MBRM 
5.5913 
0.4523 
0.0255 
51.0500 
90.4513 
101.6404 
0.9967 
Monis 
5.5913 
0.4523 
0.0255 
51.0500 
90.4584 
101.6411 
0.9967 
Savvysoft 
5.5913 
0.4523 
0.0255 
51.0500 
90.4584 
101.6410 
0.9967 
Tech Hackers 
5.5913 
0.4523 
0.0255 
51.0500 
90.4584 
101.6410 
0.9967 
Mean 
5.5913 
0.4523 
0.0255 
51.0501 
90.6370 
101.6402 
0.9967 
Std.Dev. 
0.0000 
0.0000 
0.0000 
0.0002 
0.4058 
0.0016 
0.0000 
Different algorithms
Based on the apparent inputs, I guessed that each vendor used a variation on the BlackScholesMerton model, and probably the same algorithm. The identity of that algorithm wasn’t obvious, however. To discover it, I computed premium and Greeks with eight of the algorithms that I use when reviewing models for clients. The premium and Greek results for my “closedform” or “analytic” BlackScholesMerton algorithm agreed to four decimal places with those of all packages, so I conclude that they all use that algorithm, or a similar one.
Table 3 
A Translation Table of Names For Greeks 
Input 
Underlying 
Vol. 
Dividend Yield, Foreign Rate 
Interest Rate, Domestic Rate 
Correlation 
Theta 
FEA 
Delta, Gamma 
Vega 
Lambda 
Rho 
Eta 
ðC/ðt 
FinCAD 
Delta, Gamma 
Vega 
Rho of foreign rate 
Rho of domestic rate 

ðC/ðt/365 
Intermark 
Delta, Gamma 
Kappa, Vega 

Rho 

ðC/ðT /365 
Marvin 
Delta, Gamma 
Vega 

Rho 

ðC/ðt /365 
MBRM 
Delta, Gamma 
Kappa, Vega 
Phi 
Rho 

ðC/ðt /252 
Monis 
Delta, Gamma 
Lambda 
Rho (Div) 
Rho 
Epsilon 
ðC/ðt 
Savvysoft 
Delta, Gamma 
Vega 
Foreign Rho 
Rho 
corr sens 
ðC/ðT 
Tech Hackers 
Delta, Gamma 
Kappa, Vega 
Phi 
Rho 

ðC/ðT 
WM 
Delta, Gamma 
Kappa 
Rho_Div 
Rho, Rho_Int 
Kappa_corr 
ðC/ðT 
My other seven algorithms produced similar results, which they should, since they are merely different ways to solve the same BlackScholesMerton initial value problem. In the limit, as the number of time periods, price intervals and Monte Carlo sample size go to infinity, the algorithms would produce identical results—at least, if the computer made no tiny mistakes such as rounding errors. Of course, in the limit we are all dead.
The different algorithms are not equally fast and accurate, however. The standard BlackScholesMerton “analytic” solution is quickest and most accurate. Numerical quadrature can be about as accurate, but tends to be slightly slower. Monte Carlo is slowest and among the least accurate. In a shootout between the pricing algorithms for European calls and puts, the BlackScholesMerton algorithm is like a gunfighter who’s downed plenty of coffee, but the Monte Carlo algorithm is like one who hit the whiskey bottle too hard. The two binomial and three finite difference algorithms that I used are ordinarily less accurate than the analytic algorithm and more accurate than Monte Carlo.
Different ways To report outputs
The various ways the packages report and interpret Greeks illustrate the Tower of Babel aspect of the derivatives world. Table 3 lists names for the various sensitivities. The key features of this table are:
 Nearly all vendors recognize vega as a name for sensitivity to a change in volatility. Three recognize kappa. Monis calls it lambda.
 Most vendors don’t use lambda at all, but FEA uses it for sensitivity to moves in the foreign rate (dividend yield), and Monis uses it for sensitivity to moves in volatility. So don’t compare the lambdas for FEA and Monis!
 Tech Hackers and MBRM use phi to indicate the sensitivity to changes in the dividend yield.
 FEA defines eta, Monis defines epsilon and Savvysoft defines “corr sens” as the sensitivity of premium to correlation.
 After adjusting for the different definitions for theta, and the fact that FinCad, Monis and Savvysoft report sensitivity to the change in the interest rate or dividend yield, expressed as an APR, the Greeks for the eight packages fell pretty much into line. The results—premium and all Greeks—from my analytic BlackScholesMerton algorithm agreed to four decimal places with the results for all vendors. The missing Greeks from some vendors surprised me, however.
The American put
We might anticipate near unanimity in pricing American puts because listed American options and the pricing algorithms have been around for decades. Even after we carefully give each package the inputs it wants and carefully interpret the outputs, however, we find that the pricing for the American put isn’t as unanimous as for the European put. (See Table 4.)
The premium values fell into four groups: FEA and FinCAD; Intermark and MBRM; Marvin; and Monis, Savvysoft and Tech Hackers. This suggests that the eight vendors used essentially four different algorithms for computing premium value. The agreement for the Greeks—after all the usual adjustments for the differences in conventions—was also less than with the European put. In particular, the rho figures are diverse.
In an effort to identify those algorithms, I computed premium and Greeks with eight algorithms. The results appear in Table 5. CRR and JR are the CoxRossRubinstein and JarrowRudd binomial algorithms. EFD, CN, and IFD are the explicit finite difference, CrankNicholson, and implicit finite differences. FEA and FinCAD appear to use CoxRossRubinstein. Monis, Savvysoft and Tech Hackers appear to use JarrowRudd. Not so obviously, Intermark and MBRM use one variant of CRR, and Marvin uses another.
Table 4 
American Put Option Value and Greeks For Eight Commercial Packages 
Model 
Premium 
Delta 
Gamma 
Kappa 
Rho_Div 
Rho_Int 
Theta 
BS PDE 
FEA 
5.7585 
0.4738 
0.0280 
52.4169 
72.3531 
79.9731 
1.1169 
0.0054 
FinCAD 
5.7585 
0.4523 
0.0282 
52.4072 
80.4539 
71.9449 
1.1248 

Intermark 
5.7614 
0.4742 
0.0284 
52.5461 

80.0527 
1.1227 
0.0225 
Marvin 
5.8717 
0.4995 
0.0298 
53.6944 

92.7516 
1.1946 
0.0000 
MBRM 
5.5913 
0.4743 
0.0284 
52.5461 
72.3702 
80.0527 
1.1229 
0.0007 
Monis 
5.7560 
0.4738 
0.0283 



1.1059 
0.0135 
Savvysoft 
5.7560 
0.4738 
0.0283 
52.4800 
66.2966 
73.5402 
1.1253 
0.0059 
Tech Hackers 
5.7560 
0.4738 
0.0283 
52.4804 
69.7095 
77.3243 
1.1261 
0.0003 
Mean 
5.7724 
0.4770 
0.0285 
52.6530 
72.2367 
79.3771 
1.1299 

Std.Dev. 
0.0402 
0.0091 
0.0005 
0.4625 
5.2255 
6.7536 
0.0270 

Whether you worry about the discrepancies depends on the use to which you want to put the numbers. Every vendor’s premium and Greek numbers are close enough for government work, which includes valueatrisk and credit risk measures for regulatory purposes. An accountant might not think 1.5 percent of the value of a portfolio was material for financial statements, or even that the overall impact would be that large, after pricing many positions.
A trader might or might not care about model error or algorithmic error that was 10 basis points of notional, depending on his market. Shelly Natenberg, an experienced floor trader, puts the issue into the context of IBM longterm equity anticipation securities (LEAPS): “Because the bidask spread in LEAPS can be a point or more wide, whether a marketmaker is off by 1/8 or even 1/4 in his quote is probably not significant. What’s at least as important is how he handles the position’s risk once he has it.”
In commenting on currency options, Nassim Taleb, another experienced trader, says that a twoyear expiration was “too long,” and that the bidask spread for one year might be about “twotenths of a volatility point—say 16.1 bid, 16.3 offered.” Making all the appropriate changes of variables, I conclude that that amounts to ticking our volatility up and down by just over 7/100 of a percent, which gives us a bidask spread in premium of some 7–8 basis points—5.7189 to 5.7931. Compared with this, a model error of 11 or 12 basis points appears to me to be significant.
Call me a worrier, but I believe that the differences are worth exploring, regardless of the market. The spread in premium in Table 5 from the lowest to highest is 0.1125, more than 11 basis points, nearly 1/8 of a percent of notional. If the underlying notional amount is $100 million, the buyer has the trade on his books based on 5.8684, and the seller carries the transaction based on 5.7560, then their net profit and loss is about $112,500 more than it should be.
Practical Suggestions For Buying a Package of Addins 
I can’t tell you what package to buy, and don’t have the space here to discuss all considerations fully. Here are some key points:
 Demand that the vendor gives or sells you training—or, at least, sufficient access to a help desk.
 Speak the language of the package and interpret inputs and outputs properly.
 Understand your addin’s algorithm, so you can evaluate its strengths and weaknesses.
 Make sure the package you buy does what you want and does it right, before you buy it. Merely assuming that it does is not safe, particularly for exotic options.
 Ask for a demo period of at least two weeks.
 Make sure you know how to use the package properly, before you start using it in earnest. Most packages include worked examples. Rework them.
 Check accuracy by comparing results with another package or system, preferably with a user who has more experience than you have.
 If you can’t reproduce results that you know are correct, and can’t explain the discrepancy—such as different algorithms that differ in a predictable way—don’t buy the product.
—W.M. 
What are the correct values for premium and Greeks? The bottom line of Table 5, “Extrapolation,” contains my best estimate.
The European put on a spread
The European put on a spread is a relatively simple and common rainbow option involving more than one risk factor. It has more Greeks than the first two options. It has the usual rho and theta. But with two underlying assets, we have two deltas, kappas and phis, and a sensitivity to the changes in the correlation. The option has three or four gammas, depending on how you look at it.
The dispersion across eight vendors in results for the European put on a spread is greater than it was for the American put—the range is more than 1 percent of underlying notional. This explains part of the large bidask spread in the market for exotic options. I attribute this dispersion largely to the fact that the seven vendors attempting to price this product used at least eight different algorithms.
I used three different algorithms to check the premium values and Greeks. The packages produce numbers that are in the ballpark. All the models and algorithms might be correct in the sense that they would approach the correct limiting value if the number of binomial periods or Monte Carlo paths approached infinity, or they might contain some flaws. If a customer wanted my opinion, I would build my own versions of the various algorithms for comparison.
Table 5 
American Put Option Value and Greeks For Eight Methods 
Model 
Premium 
Delta 
Gamma 
Kappa 
Rho_Div 
Rho_Int 
Theta 
CRR 
5.7585 
0.4737 
0.0283 
52.4162 
73.8277 
82.5022 
1.1249 
JR 
5.7560 
0.4743 
0.0283 
52.5732 
74.6595 
82.5621 
1.1264 
EFD (VB) 
5.7532 
0.4730 
0.0281 
52.1109 
72.5560 
80.1660 
1.1155 
CN(VB) 
5.7429 
0.4729 
0.0282 
52.4014 
72.7874 
80.3896 
1.1207 
IDF(VB) 
5.7330 
0.4728 
0.0282 
52.6873 
73.1506 
80.7632 
1.1258 
EFD (C) 
5.7532 
0.4734 
0.0282 
52.4554 
73.9629 
82.6173 
1.1223 
CN (C) 
5.7429 
0.4734 
0.0283 
52.3505 
74.1795 
82.8455 
1.1276 
IFD (C) 
5.7330 
0.4733 
0.0284 
52.2537 
74.3549 
83.0320 
1.1327 
Mean 
5.7466 
0.4733 
0.0282 
52.4061 
73.6848 
81.8597 
1.1245 
Std. Dev. 
0.0101 
0.0005 
0.0001 
0.1785 
0.7659 
1.1987 
0.0051 
Extrapolation 
5.7474 
0.4733 
0.0282 
52.3758 
72.4851 
80.0658 
1.1208 
This glimpse into the world of exotic options tells us that coverage of the Greeks is thinning out. Hardly anybody had crossgamma, an important measure. Various other Greeks were missing. In addition, everybody has their own way of doing things, and some people have two ways. Consequently, we see larger discrepancies in results.
Summary
Different packages sometimes require their inputs and express their outputs in different forms. Everybody used practically the same model and algorithm to price European puts. Even so, it wasn’t easy to coax the same premium and Greeks out of all the addins.
Everybody used a binomial algorithm to price American puts. Six used a variant of the CRR algorithm, and two used the JR algorithm. One addin had a noticeable bug which the vendor syas is now fixed. Not surprisingly, outputs differed.
Seven out of eight packages could price a European put option on a spread, and three packages had two ways apiece to do it. Five of the packages provided both deltas. Two provided two of three correct gammas and one transposed them. None provided all three gammas. After that, Greeks were sparse.
Learning to use any of these packages properly requires a significant amount of time. I’m not talking about simply loading the package, delivering the inputs, pushing the [F9] key and looking at the results. I’m talking about getting meaningful results that you can use correctly. None of the packages was intuitive for me. Several of them fooled me at first, and one of them took a lot of work to learn to use properly.
The pricing algorithms in these eight packages are like snowflakes. No two are exactly alike, even if they purport to do the same thing—including Intermark and MBRM, which separated only a couple of years ago. This is consistent with my 25 years of experience with derivatives pricing models.
You can’t conclude that one addin is wrong simply because it produces numbers that don’t agree perfectly with another addin. They may use different algorithms that agree only in the limit. This problem appears to grow more common with more complex derivatives. I would anticipate a serious problem of this sort with exotic fixedincome derivatives.
Models and Assumptions 
The Put Options
I chose European and American puts because they make up a large percentage of open option contracts. I assumed that a system that contains an accurate put option pricing function would contain an accurate pricing function for the corresponding call.
My assumptions about the market conditions of both put options are:
 The underlying asset is a cash instrument—a share or a foreign currency.
 The underlying price is 100.
 Volatility equals 10 percent.
 Dividend yield (if the underlying is a share) or the foreign interest rate (if the underlying is a currency) equals 5 percent per annum, continuously compounded.
 The (domestic) market rate of interest is, likewise, 5 percent.
 The put expires in two years.
 Its strike is 101.
 Number of time steps equals 100 for binomial models of American put option premium and Greeks.
The Put On a Spread
I chose the put on the spread of asset two’s price over asset one’s price, struck at 10, because (a) it involves the significant step into a second dimension of risk, and (b) most of the packages claimed to price it. Of course, if the strike were zero one could change “numeraires” and price the option with the BlackScholesMerton model.
My assumptions are:
 The underlying assets are cash instruments—two different shares or currencies.
 The underlying prices are 101 and 102.
 The volatilities are 10 percent and 20 percent.
 The dividend yields are 1 percent and 2 percent.
 The correlation between changes in logs of the two prices is 0.12.
 The domestic rate of interest is 5 percent.
 The European put expires in two years.
 Its strike is 10.
 Payoff is Max[0, K (S2—S1)] = Max [0, K + S1—S2]—a put on the spread of price 2 over price 1.
 Number of time steps equals 100 for binomial models.
 Sample size equals 10,000 for Monte Carlo models.
—W.M. 
I noticed a tendency for some of these packages to behave like the bird that hatches first from its egg, then tries to destroy its unhatched rivals. The philosophy seems to be “If you capture the Excel menu bar and RAM, then the user’s heart and soul will follow peacefully.” Also, the last one loaded sometimes has an advantage over its predecessors. For example, if two addins have the same name, then the last one loaded replaces its predecessor.
Was this information valuable?
Subscribe to Derivatives Strategy by clicking here!


