« Univac Document Archive: 1107 COBOL Programmer's Guide Added | Main | Reading List: Artemis »

Monday, January 22, 2018

Floating Point Benchmark: PHP Language Added

I have posted an update to my trigonometry-intense floating point benchmark which adds PHP to the list of languages in which the benchmark is implemented. A new release of the benchmark collection including PHP is now available for downloading.

PHP is a brutal hack of a language created by bashing together HTML and a bastardised flavour of Perl which is just enough like Perl to trick you into thinking you know it, only to pull the rug out from under you whenever you become complacent. PHP is the foundation of vast snowdrifts of incomprehensible spaghetti code which adorns many Web sites, and has been the wellspring of innumerable security holes in those sites. PHP was originally created in 1994 and to this day has no formal specification: the language is defined by the implementation of the interpreter.

I wouldn't let PHP anywhere near my own www.fourmilab.ch Web site, but I have to deal with it in mantaining a WordPress-based site I help to administer. Having been forced to wallow with the pig, I decided to add PHP to the list of FBENCH languages.

The implementation is a relatively straightforward port of the Perl version of the program, modified to be embedded within a Web page and adapted to the differences between Perl and PHP in handling things such as variable scope in functions, declaration (or non-declaration) of variables, and the structure of arrays.

As the reference, I started with the C version, compiled with GCC 5.4.0, and run for 166,051,660 iterations. Run times in seconds were (296.89, 296.37, 296.29, 296.76, 296.37) for a mean time of 296.536, or 1.7858 microseconds per iteration.

I then made five timing runs on the same machine, using the PHP command line interpreter version 7.0.22. I ran the benchmark for 32,952,075 iterations, having calculated that number from a previous timing run to yield a run time of around five minutes. The timings in seconds were (295.320, 295.135, 297.174, 296.037, 297.098) for a mean of 296.153 seconds or 8.9874 microseconds per iteration.

Comparing the two measurements of time per iteration, PHP is thus 5.033 times slower than C. Interestingly, this is much faster than Perl, which usually benchmarks more than twenty times slower than C.

I then verified that the same Web page can be run on a Web server with PHP. Since the machine on which I ran the test had a different type of CPU than the development machine on which I made the timing tests above, I did not compare execution time.

Notes on Running the Benchmark

The benchmark is supplied as the fbench.php file. This is a Web page, compliant with the XHTML 1.0 Strict standard, with PHP embedded to implement the benchmark. It can be run either from the command line using the PHP interpreter, or by installing it on a Web site which supports PHP. The iteration count defaults to 100000. You can specify the iteration count when running from the command line as the first argument, for example:

php fbench.php 250000 >fbench.html

to run 250000 iterations and write the resulting HTML in the file fbench.html, which may then be viewed with a Web browser using a "file:" URL.

If you install fbench.php on a Web server with PHP, you can run it and specify the iteration count with a URL like:


In this case your Web browser will directly display the results of the benchmark.

The benchmark program displays the result of the last iteration of the computation, reports any discrepancies from the reference results, and reports the timing, both in seconds elapsed and microseconds per iterations. Timing is done using the PHP microtime(true) function, which reports wall clock time in seconds and microseconds. Archival runs should be done on an idle system with the iteration count adjusted to produce a run time around five minutes. (When running from a Web server, you may have to reduce the run time to avoid having the request time out.)

The relative performance of the various language implementations (with C taken as 1) is as follows. All language implementations of the benchmark listed below produced identical results to the last (11th) decimal place.

Language Relative
C 1 GCC 3.2.3 -O3, Linux
JavaScript 0.372
Mozilla Firefox 55.0.2, Linux
Safari 11.0, MacOS X
Brave 0.18.36, Linux
Google Chrome 61.0.3163.91, Linux
Chromium 60.0.3112.113, Linux
Node.js v6.11.3, Linux
Chapel 0.528
Chapel 1.16.0, -fast, Linux
Parallel, 64 threads
Visual Basic .NET 0.866 All optimisations, Windows XP
C++ 0.939
G++ 5.4.0, -O3, Linux, double
long double (80 bit)
__float128 (128 bit)
MPFR (128 bit)
MPFR (512 bit)
Modula-2 0.941 GNU Modula-2 gm2-1.6.4 -O3, Linux
FORTRAN 1.008 GNU Fortran (g77) 3.2.3 -O3, Linux
Pascal 1.027
Free Pascal 2.2.0 -O3, Linux
GNU Pascal 2.1 (GCC 2.95.2) -O3, Linux
Swift 1.054 Swift 3.0.1, -O, Linux
Rust 1.077 Rust 0.13.0, --release, Linux
Java 1.121 Sun JDK 1.5.0_04-b05, Linux
Visual Basic 6 1.132 All optimisations, Windows XP
Haskell 1.223 GHC 7.4.1-O2 -funbox-strict-fields, Linux
Scala 1.263 Scala 2.12.3, OpenJDK 9, Linux
FreeBASIC 1.306 FreeBASIC 1.05.0, Linux
Ada 1.401 GNAT/GCC 3.4.4 -O3, Linux
Go 1.481 Go version go1.1.1 linux/amd64, Linux
Julia 1.501 Julia version 0.6.1 64-bit -O2 --check-bounds=no, Linux
Simula 2.099 GNU Cim 5.1, GCC 4.8.1 -O2, Linux
Lua 2.515
LuaJIT 2.0.3, Linux
Lua 5.2.3, Linux
Python 2.633
PyPy 2.2.1 (Python 2.7.3), Linux
Python 2.7.6, Linux
Erlang 3.663
Erlang/OTP 17, emulator 6.0, HiPE [native, {hipe, [o3]}]
Byte code (BEAM), Linux
ALGOL 60 3.951 MARST 2.7, GCC 4.8.1 -O3, Linux
PHP 5.033 PHP (cli) 7.0.22, Linux
PL/I 5.667 Iron Spring PL/I 0.9.9b beta, Linux
Lisp 7.41
GNU Common Lisp 2.6.7, Compiled, Linux
GNU Common Lisp 2.6.7, Interpreted
Smalltalk 7.59 GNU Smalltalk 2.3.5, Linux
Ruby 7.832 Ruby 2.4.2p198, Linux
Forth 9.92 Gforth 0.7.0, Linux
Prolog 11.72
SWI-Prolog 7.6.0-rc2, Linux
GNU Prolog 1.4.4, Linux, (limited iterations)
COBOL 12.5
Micro Focus Visual COBOL 2010, Windows 7
Fixed decimal instead of computational-2
Algol 68 15.2 Algol 68 Genie 2.4.1 -O3, Linux
Perl 23.6 Perl v5.8.0, Linux
BASICA/GW-BASIC 53.42 Bas 2.4, Linux
QBasic 148.3 MS-DOS QBasic 1.1, Windows XP Console
Mathematica 391.6 Mathematica, Raspberry Pi 3, Raspbian

Download floating point benchmark collection

Posted at January 22, 2018 15:59