281 lines
No EOL
18 KiB
HTML
281 lines
No EOL
18 KiB
HTML
<!DOCTYPE html>
|
|
<html>
|
|
<head>
|
|
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
|
|
<title>CLiki: Performance Benchmarks2</title>
|
|
<link rel="alternate" type="application/atom+xml" title="ATOM feed of edits to current article"
|
|
href="https://www.cliki.net/site/feed/article.atom?title=Performance%20Benchmarks2">
|
|
<link rel="stylesheet" href="static/css/style.css">
|
|
<link rel="stylesheet" href="static/css/colorize.css">
|
|
</head>
|
|
|
|
<body>
|
|
<span class="hidden">CLiki - Performance Benchmarks2</span>
|
|
<div id="content"><div id="content-area"><div id="article-title">Performance Benchmarks2</div><div id="article">Here are the results of several performance <a href="benchmark.html" class="category">benchmark</a>ing tests run on seven different <a href="Common Lisp implementation.html" class="internal">Common Lisp implementation</a>s running on a P3 1.1 GHz, an UltraSPARCIIe 500 MHz and a dual G5 2.5 GHz. The tests run include<p><ul>
|
|
<li> the well-known Gabriel benchmarks
|
|
</li>
|
|
<li> hashtable exercising
|
|
</li>
|
|
<li> <a href="https://www.cliki.net/site/HyperSpec/Body/fun_read-line.html" class="hyperspec">READ-LINE</a> exercising
|
|
</li>
|
|
<li> mathematical functions: factorial, fibonacci, ackermann's number
|
|
</li>
|
|
<li> some bignum intensive code from Bruno Haible
|
|
</li>
|
|
<li> various aspects of CLOS: time taken to compile and execute
|
|
<a href="https://www.cliki.net/site/HyperSpec/Body/mac_defclass.html" class="hyperspec">DEFCLASS</a> forms which create a class hierarchy, instance creation
|
|
time, time taken to compile and execute <a href="https://www.cliki.net/site/HyperSpec/Body/mac_defmethod.html" class="hyperspec">DEFMETHOD</a> forms, execution
|
|
of method combinations, both with and without :after methods.
|
|
</li>
|
|
<li> various operations on arrays, bitvectors and strings.
|
|
</li>
|
|
<li> <a href="Odd Benchmarks.html" class="internal">Odd Benchmarks</a> that test various features of CL against each other
|
|
</li>
|
|
</ul><p><em>This page is very similar to <a href="performance benchmarks.html" class="internal">performance benchmarks</a>. Should they be merged?</em> -- <a href="Roland Kaufmann.html" class="internal">Roland Kaufmann</a><p>Except for the CLOS code, timings do not include compilation time. The
|
|
garbage collector is run before each test to try to make the timings
|
|
more repeatable. Remember that the only real benchmark is your
|
|
application: this code is only representative of real-life programs to
|
|
a limited extent. For certain targets, we assume that the times
|
|
reported by <a href="https://www.cliki.net/site/HyperSpec/Body/fun_get-internal-run-time.html" class="hyperspec">GET-INTERNAL-RUN-TIME</a> and <a href="https://www.cliki.net/site/HyperSpec/Body/fun_get-internal-real-time.html" class="hyperspec">GET-INTERNAL-REAL-TIME</a> are
|
|
accurate.<p>The code for these performance benchmarks can be downloaded from
|
|
<a href="http://www.chez.com/emarsden/downloads/cl-bench.tar.gz">http://www.chez.com/emarsden/downloads/cl-bench.tar.gz</a>. (For some reason
|
|
this seems to error out as a direct link but not when accessed from
|
|
the following page: <a href="http://www.chez.com/emarsden/downloads/">http://www.chez.com/emarsden/downloads/</a>.)
|
|
|
|
Here is an example of the type of results you can obtain, for x86 and
|
|
SPARC platforms. To interpret these numbers: the number in the Reference column is a number of seconds taken to execute that test. The other numbers are the relative performance of that implementation, so for example a number of 1.3 means 30% slower than the reference implementation. A number of -1 means that the corresponding test was not run on that implementation, either because it does not support the given feature (certain types of method combination for CLISP, for instance), or because of implementation limits (heap size limit on LispWorks Personal, for example). <p> <pre>
|
|
,---- Pentium III@1.1GHz, 512MB RAM ---
|
|
|
|
|
| Benchmark Reference CMUCL CMUCL CLISP SBCL Poplog
|
|
| CMUCL 18c 18d 18dsm 2.27.2 0.7.1 2.0
|
|
| -----------------------------------------------------------------------
|
|
| BOYER 1.50 0.73 0.74 5.61 1.01 3.03
|
|
| BROWSE 0.61 0.74 0.74 2.54 1.26 2.85
|
|
| DDerviv 0.66 0.52 0.53 2.14 1.15 1.62
|
|
| Deriv 1.29 0.50 0.48 2.05 1.10 1.26
|
|
| DESTRUCTIVE 1.05 0.91 0.95 3.17 1.13 3.38
|
|
| DIV2-TEST-1 1.09 0.51 0.51 1.66 1.07 1.29
|
|
| DIV2-TEST-2 0.54 0.52 0.61 1.69 1.09 1.31
|
|
| FFT 0.19 1.00 1.00 37.00 1.21 91.21
|
|
| FRPOLY/FIXNUM 0.54 0.87 0.89 6.85 1.09 3.00
|
|
| FRPOLY/BIGNUM 1.30 0.73 0.74 2.44 1.08 1.39
|
|
| FRPOLY/FLOAT 0.54 0.76 0.78 4.41 1.06 2.11
|
|
| PUZZLE 0.68 0.99 0.99 25.09 9.35 27.93
|
|
| TAK 0.79 0.97 0.97 9.08 1.25 3.10
|
|
| CTAK 0.73 1.00 0.97 6.49 1.05 4.84
|
|
| TRTAK 0.79 0.97 0.97 9.13 1.25 3.11
|
|
| TAKL 1.37 1.01 1.00 7.70 1.27 4.02
|
|
| STAK 0.96 1.02 1.02 7.07 1.08 1.50
|
|
| FPRINT 1.21 0.92 0.82 1.06 2.28 4.17
|
|
| TRAVERSE 2.00 0.94 0.99 11.45 1.50 5.42
|
|
| TRIANGLE 1.70 1.01 1.01 17.58 1.28 23.19
|
|
| RICHARDS 0.53 1.00 0.98 21.26 1.00 6.47
|
|
| FACTORIAL 0.37 0.54 0.57 0.81 1.14 0.62
|
|
| FIB 0.33 1.00 1.00 5.30 1.06 1.18
|
|
| RATIOS 2.24 0.96 0.96 0.12 0.96 1.81
|
|
| BIGNUM/ELEM-100-1000 1.06 0.92 0.97 0.08 0.97 2.27
|
|
| BIGNUM/ELEM-1000-100 4.99 0.99 0.99 0.05 0.93 2.14
|
|
| BIGNUM/ELEM-10000-1 5.19 1.01 1.00 0.05 0.92 12.90
|
|
| BIGNUM/PARI-100-10 1.21 0.99 1.02 0.02 0.95 0.04
|
|
| BIGNUM/PARI-200-5 14.30 0.99 1.00 0.01 0.92 0.01
|
|
| HASH-STRINGS 1.15 0.84 0.69 3.25 0.99 30.89
|
|
| HASH-INTEGERS 0.44 0.80 0.66 2.75 1.27 6.95
|
|
| BOEHM-GC 6.41 0.47 0.46 4.10 1.12 1.53
|
|
| DEFLATE 1.16 1.00 1.02 6.34 1.52 5.53
|
|
| CLOS/defclass 1.75 0.90 0.92 0.53 2.53 0.06
|
|
| CLOS/defmethod 8.73 0.77 0.71 0.02 2.17 0.00
|
|
| CLOS/instantiate 6.42 0.57 0.54 0.96 0.99 3.57
|
|
| CLOS/methodcalls 8.17 0.42 0.44 1.01 0.90 3.93
|
|
| CLOS/method+after 10.76 0.84 0.77 0.47 2.42 2.01
|
|
| CLOS/complex-methods 0.29 1.07 1.14 -1.00 6.17 -1.00
|
|
| 1D-ARRAYS 2.44 0.81 0.79 5.68 1.39 15.75
|
|
| 2D-ARRAYS 11.28 0.99 0.89 3.45 4.39 6.04
|
|
| BITVECTORS 2.01 0.82 0.81 0.60 0.98 -1.00
|
|
| FILL-STRINGS 6.83 0.88 0.82 1.16 3.77 27.38
|
|
| fill-strings/adjustable 14.18 0.99 0.87 1.36 1.73 118.49
|
|
| BENCH-STRING-CONCAT 5.79 1.88 1.62 -1.00 2.75 -1.00
|
|
|
|
|
| Reference implementation: CMU Common Lisp 18c
|
|
| Impl 18d: CMU Common Lisp 18d-pre, level-1 built 2002-04-08 on melbourne
|
|
| Impl 18dsm: CMU Common Lisp 18d-pre-small, level-1 built 2002-04-08 on sydney
|
|
| Impl CLISP: CLISP 2.27.2 (released 2001-10-05) (built 3222523548) (memory 3222523674)
|
|
| Impl SBCL : SBCL 0.7.1.22 (from debian testing)
|
|
| Impl Poplog: Sussex Poplog Common Lisp 2.0
|
|
| Linux melbourne 2.4.17 #2 SMP Wed Feb 6 11:35:03 CET 2002 i686 unknown
|
|
| (declaim (optimize (speed 3) (space 0) (safety 1) (debug 0) (compilation-speed 0)))
|
|
`----
|
|
|
|
|
|
,---- UltraSPARCIIe at 500MHz, 640MB RAM, SunOS 5.8 ---
|
|
|
|
|
| Benchmark Reference CMU C CLISP
|
|
| -----------------------------------------------------
|
|
| BOYER 3.98 0.91 8.03
|
|
| BROWSE 1.72 0.91 2.85
|
|
| DDerviv 2.02 0.75 3.21
|
|
| Deriv 3.63 0.81 3.13
|
|
| DESTRUCTIVE 3.11 1.01 4.18
|
|
| DIV2-TEST-1 2.19 0.83 3.92
|
|
| DIV2-TEST-2 1.12 0.82 3.85
|
|
| FFT 0.74 1.03 28.86
|
|
| FRPOLY/FIXNUM 1.87 1.01 7.89
|
|
| FRPOLY/BIGNUM 4.59 1.29 3.07
|
|
| FRPOLY/FLOAT 1.65 0.96 5.68
|
|
| PUZZLE 2.07 0.95 30.62
|
|
| CTAK 2.74 1.01 9.04
|
|
| TAK 1.84 1.00 14.08
|
|
| RTAK 1.84 1.01 13.95
|
|
| TAKL 3.37 1.01 11.63
|
|
| STAK 2.32 1.01 8.87
|
|
| FPRINT 4.17 1.02 1.12
|
|
| TRAVERSE 5.84 0.99 13.74
|
|
| TRIANGLE 5.53 0.86 15.57
|
|
| CASCOR 10.53 0.73 52.81
|
|
| RICHARDS 2.35 0.94 22.46
|
|
| FACTORIAL 1.46 1.48 2.88
|
|
| FIB 0.94 0.99 6.71
|
|
| BIGNUM/ELEM-100-1000 2.80 1.24 0.28
|
|
| BIGNUM/ELEM-1000-100 10.14 1.19 0.44
|
|
| BIGNUM/ELEM-10000-1 11.38 1.35 0.41
|
|
| BIGNUM/PARI-100-10 2.76 1.15 0.09
|
|
| BIGNUM/PARI-200-5 27.19 1.06 0.05
|
|
| READ-LINE 3.39 1.06 1.19
|
|
| HASH-STRINGS 5.42 1.20 2.19
|
|
| HASH-INTEGERS 1.61 0.76 2.00
|
|
| BOEHM-GC 19.97 0.76 4.14
|
|
| CLOS/defclass 4.78 1.01 0.81
|
|
| CLOS/defmethod 27.61 0.89 0.03
|
|
| CLOS/instantiate 20.93 0.85 1.28
|
|
| CLOS/methodcalls 23.62 1.08 1.94
|
|
| CLOS/method+after 33.70 1.07 0.78
|
|
| CLOS/complex-methods 1.41 0.92 -1.00
|
|
| 1D-ARRAYS 10.77 0.92 3.51
|
|
| 2D-ARRAYS 56.66 1.40 2.61
|
|
| BITVECTORS 5.35 0.86 0.42
|
|
| FILL-STRINGS 18.88 1.07 0.97
|
|
| fill-strings/adjustable 45.09 1.46 1.41
|
|
| BENCH-STRING-CONCAT 48.10 0.90 -1.00
|
|
|
|
|
| Reference implementation: CMU Common Lisp 18c, Built 2000-11-27
|
|
| Impl CMU C: CMU Common Lisp 18d-pre, level-1 built 2001-12-12 on liszt
|
|
| Impl CLISP: CLISP 2.27.2 (released 2001-10-05) (built on moustacho)
|
|
| SunOS eagles 5.8 Generic_108528-10 sun4u sparc SUNW,Sun-Blade-100
|
|
`----
|
|
</pre><p><i>Note that results prior to 2002-01 were a little bogus, since the optimization settings were not being set correctly across implementations, so default optimization levels were used. </i><p>If anyone can run these against ACL (not the trial version, whose heap size limits prevent most of these tests from running) I would appreciate obtaining the results. -- Eric Marsden<p>----<p>I reran the cl-bench benchmarks, using newer versions of SBCL and OpenMCL.
|
|
I aimed for ~10 seconds of run time per individual benchmark. Compiler optimization
|
|
made this impossible on some tests.<p>I would like to include Lispworks, but the personal edition has a heap size limitation
|
|
that prevents many of the tests from completing. No bias is intended!<p>I did not bother with the Allegro alisp8 runs this time.<p>SBCL no longer crashes on the CLOS tests.<p>A quick hand count shows:<p>Allegro 47/65
|
|
SBCL 14/65
|
|
OpenMCL 4/65<p>The test machine is a dual 2.5 G5, running OS X 10.4.2<p><pre>
|
|
Benchmark Reference OpenMCL SBCL
|
|
ACL 7.0 0.14.3 0.9.4
|
|
-------------------------------------------------------------------------------------
|
|
COMPILER [ 10.80 ] 0.99* 3.33
|
|
LOAD-FASL [ 9.77*] 2.83 3.51
|
|
SUM-PERMUTATIONS [ 10.68*] 3.05 1.86
|
|
WALK-LIST/SEQ [ 0.01*] 5.50 2.60
|
|
WALK-LIST/MESS [ 0.02*] 2.00 2.85
|
|
BOYER [ 14.94 ] 1.83 0.65*
|
|
BROWSE [ 10.89*] 2.62 4.17
|
|
DDERIV [ 10.21*] 4.65 8.26
|
|
DERIV [ 10.90*] 4.60 8.95
|
|
DESTRUCTIVE [ 10.32*] 3.47 5.47
|
|
DIV2-TEST-1 [ 11.01*] 12.54 22.10
|
|
DIV2-TEST-2 [ 9.80*] 10.11 18.23
|
|
FFT [ 10.76*] 116.68 1.07
|
|
FRPOLY/FIXNUM [ 12.02*] 1.84 2.47
|
|
FRPOLY/BIGNUM [ 11.91*] 2.60 3.12
|
|
FRPOLY/FLOAT [ 10.84*] 2.89 2.59
|
|
PUZZLE [ 10.91 ] 4.89 0.50*
|
|
TAK [ 11.76*] 1.39 1.70
|
|
CTAK [ 11.31 ] 0.93 0.59*
|
|
TRTAK [ 10.58*] 1.39 1.79
|
|
TAKL [ 11.74 ] 1.71 0.82*
|
|
STAK [ 11.53 ] 0.37 0.15*
|
|
FPRINT/UGLY [ 10.41*] 10.81 4.50
|
|
FPRINT/PRETTY [ 9.87*] 5.78 4.98
|
|
TRAVERSE [ 11.05*] 1.77 1.04
|
|
TRIANGLE [ 10.72 ] 1.84 0.75*
|
|
RICHARDS [ 10.66 ] 0.75 0.64*
|
|
FACTORIAL [ 10.72*] 7.25 9.42
|
|
FIB [ 11.87*] 1.49 2.51
|
|
FIB-RATIO [ 10.92*] 4.40 8.20
|
|
ACKERMANN [ 13.47 ] 0.52* 0.81
|
|
MANDELBROT/COMPLEX [ 11.02*] 2.61 8.44
|
|
MANDELBROT/DFLOAT [ 11.79*] 56.63 4.89
|
|
MRG32K3A [ 11.92 ] 16.34 0.57*
|
|
CRC40 [ 13.95*] 3.31 2.82
|
|
BIGNUM/ELEM-100-1000 [ 11.64*] 3.92 2.03
|
|
BIGNUM/ELEM-1000-100 [ 11.98 ] 5.38 0.87*
|
|
BIGNUM/ELEM-10000-1 [ 11.94 ] 5.00 0.40*
|
|
BIGNUM/PARI-100-10 [ 11.67*] 25.08 3.81
|
|
BIGNUM/PARI-200-5 [ 11.98*] 68.45 2.92
|
|
PI-DECIMAL/SMALL [ 12.09*] 12.20 5.51
|
|
PI-DECIMAL/BIG [ 10.29*] 35.78 3.33
|
|
PI-ATAN [ 11.35*] 3.41 5.75
|
|
PI-RATIOS [ 11.44*] 2.96 2.84
|
|
HASH-STRINGS [ 11.70 ] 0.53 0.31*
|
|
HASH-INTEGERS [ 11.84*] 15.67 4.92
|
|
SLURP-LINES [ 10.39*] 4.80 26.57
|
|
BOEHM-GC [ 13.29*] 4.84 2.78
|
|
DEFLATE-FILE [ 11.27*] 1.56 1.42
|
|
1D-ARRAYS [ 11.01 ] 0.92* 1.19
|
|
2D-ARRAYS [ 11.13*] 18.09 2.23
|
|
3D-ARRAYS [ 12.00*] 17.90 1.88
|
|
BITVECTORS [ 10.55*] 62.95 15.56
|
|
BENCH-STRINGS [ 11.88*] 4.20 1.10
|
|
fill-strings/adjustable [ 15.82*] 7.70 7.37
|
|
SEARCH-SEQUENCE [ 11.92 ] 1.35 0.10*
|
|
CLOS/defclass [ 10.70*] 102.03 1323.65
|
|
CLOS/defmethod [ 11.71 ] 0.68* 197.61
|
|
CLOS/instantiate [ 10.30*] 7.38 126.80
|
|
CLOS/simple-instantiate [ 9.46*] 40.98 157.12
|
|
CLOS/methodcalls [ 10.47*] 3.59 32.44
|
|
CLOS/method+after [ 11.32*] 3.11 79.45
|
|
CLOS/complex-methods [ 11.23 ] 3.19 0.61*
|
|
EQL-SPECIALIZED-FIB [ 12.10 ] 1.48 0.60*
|
|
Reference time in first column is in seconds; other columns are relative
|
|
Reference implementation: International Allegro CL Professional Edition 7.0 [Mac OS X] (Sep 29, 2005 18:24)
|
|
Impl OpenM: OpenMCL Version (Beta: Darwin) 0.14.3
|
|
Impl SBCL : SBCL 0.9.4
|
|
=== Test machine ===
|
|
Machine-type: PowerPC
|
|
Machine-version: Power Macintosh
|
|
</pre></div></div>
|
|
<div id="footer" class="buttonbar"><ul><li><a href="Performance Benchmarks2.html">Current version</a></li>
|
|
<li><a href="https://www.cliki.net/site/history?article=Performance%20Benchmarks2">History</a></li>
|
|
<li><a href="https://www.cliki.net/site/backlinks?article=Performance%20Benchmarks2">Backlinks</a></li><li><a href="https://www.cliki.net/site/edit-article?title=Performance%20Benchmarks2&from-revision=3638977252">Edit</a></li><li><a href="https://www.cliki.net/site/edit-article?create=t">Create</a></li></ul></div>
|
|
</div>
|
|
<div id="header-buttons" class="buttonbar">
|
|
<ul>
|
|
<li><a href="https://www.cliki.net/">Home</a></li>
|
|
<li><a href="https://www.cliki.net/site/recent-changes">Recent Changes</a></li>
|
|
<li><a href="CLiki.html">About</a></li>
|
|
<li><a href="Text Formatting.html">Text Formatting</a></li>
|
|
<li><a href="https://www.cliki.net/site/tools">Tools</a></li>
|
|
</ul>
|
|
<div id="search">
|
|
<form action="https://www.cliki.net/site/search">
|
|
<label for="search_query" class="hidden">Search CLiki</label>
|
|
<input type="text" name="query" id="search_query" value="" />
|
|
<input type="submit" value="search" />
|
|
</form>
|
|
</div>
|
|
</div>
|
|
<div id="pageheader">
|
|
<div id="header">
|
|
<span id="logo">CLiki</span>
|
|
<span id="slogan">the common lisp wiki</span>
|
|
<div id="login"><form method="post" action="https://www.cliki.net/site/login">
|
|
<label for="login_name" class="hidden">Account name</label>
|
|
<input type="text" name="name" id="login_name" class="login_input" />
|
|
<label for= "login_password" class="hidden">Password</label>
|
|
<input type="password" name="password" id="login_password" class="login_input" />
|
|
<input type="submit" name="login" value="login" id="login_submit" /><br />
|
|
<div id="register"><a href="https://www.cliki.net/site/register">register</a></div>
|
|
<input type="submit" name="reset-pw" value="reset password" id="reset_pw" />
|
|
</form>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
</body></html> |