|
JDBCBench
In looking to categorize the performance of different databases and JDBC drivers, I found that there were no benchmarks available. I decided to create my own simple benchmark, based on standards from other benchmarks. I call it JDBCBench, and the source is available here on the main page. JDBCBench uses a simple update-intensive transaction to load the system to be benchmarked. This workload reflects the database aspects of applications such as servlets and intranet applications, but does not reflect the entire range of transaction processing requirements characterized by multiple transaction types of varying complexities encountered in OLTP and data-mining applications. It is derived from the TPC-B benchmark available from the TPC (www.tpc.org). The single transaction type provides a simple, repeatable unit of work, and is designed to exercise the basic components of a database system.
Preliminary Results
In my first round of testing I created two workloads, one heavy in transactions, and one
heavy in the number of simultaneous clients (threads) that were accessing the database.
The results can be seen below in both graphical and tabular formats.
It is interesting to note that only one product (mm.mysql/mysql) completed
the benchmark without failing any transactions. Both mysql drivers scaled better than
the others during heavy loads, but mm scaled the best, providing anywhere from 6 to 12
times the transaction processing rate under heavy client load, and 2 to 4 times the
transaction processing rate under heavy transaction load.
|
Graphical Depictions
Raw Data for Heavy Transaction Load
| Driver/DB | Clients | TPC | Transactions/Sec | Max Mem | Failed Transactions |
| mm/mysql | 1 | 100 | 35.36 | 835808 | 0 |
| twz/mysql | 1 | 100 | 62.15 | 901120 | 0 |
| jdbc-odbc:jet | 1 | 100 | 13.73 | 884800 | 0 |
| Driver/DB | Clients | TPC | Transactions/Sec | Max Mem | Failed Transactions |
| mm/mysql | 5 | 100 | 87.9 | 933816 | 0 |
| twz/mysql | 5 | 100 | 102.98 | 987952 | 14 |
| jdbc-odbc:jet | 5 | 100 | 7.54 | 968912 | 0 |
| Driver/DB | Clients | TPC | Transactions/Sec | Max Mem | Failed Transactions |
| mm/mysql | 10 | 100 | 75.12 | 945464 | 0 |
| twz/mysql | 10 | 100 | 96.94 | 994240 | 73 |
| jdbc-odbc:jet | 10 | 100 | 10.69 | 894800 | 1000 |
| Driver/DB | Clients | TPC | Transactions/Sec | Max Mem | Failed Transactions |
| mm/mysql | 50 | 100 | 79.4 | 925896 | 0 |
| twz/mysql | 50 | 100 | 52.3 | 985904 | 1901 |
| jdbc-odbc:jet | 50 | 100 | DNF | DNF | 5000 |
| Driver/DB | Clients | TPC | Transactions/Sec | Max Mem | Failed Transactions |
| mm/mysql | 100 | 100 | 79.53 | 949120 | 0 |
| twz/mysql | 100 | 100 | 30.19 | 1008672 | 1868 |
| jdbc-odbc:jet | 100 | 100 | 10000 |
Raw Data for Heavy Client Load
| Driver/DB | Clients | TPC | Transactions/Sec | Max Mem | Failed Transactions |
| mm/mysql | 1 | 4 | 9.48 | 167192 | 0 |
| twz/mysql | 1 | 4 | 8.26 | 218608 | 0 |
| jdbc-odbc:jet | 1 | 4 | 4.66 | 192296 | 0 |
| Driver/DB | Clients | TPC | Transactions/Sec | Max Mem | Failed Transactions |
| mm/mysql | 10 | 4 | 36.56 | 699792 | 0 |
| twz/mysql | 10 | 4 | 12.17 | 484624 | 9 |
| jdbc-odbc:jet | 10 | 4 | 12.92 | 461614 | 0 |
| Driver/DB | Clients | TPC | Transactions/Sec | Max Mem | Failed Transactions |
| mm/mysql | 20 | 4 | 58.19 | 961912 | 0 |
| twz/mysql | 20 | 4 | 38.76 | 633304 | 40 |
| jdbc-odbc:jet | 20 | 4 | 12.22 | 753264 | 0 |
| Driver/DB | Clients | TPC | Transactions/Sec | Max Mem | Failed Transactions |
| mm/mysql | 50 | 4 | 70.32 | 919856 | 0 |
| twz/mysql | 50 | 4 | 57.72 | 947000 | 16 |
| jdbc-odbc:jet | 50 | 4 | 10.22 | 847784 | 0 |
| Driver/DB | Clients | TPC | Transactions/Sec | Max Mem | Failed Transactions |
| mm/mysql | 100 | 4 | 36.99 | 901816 | 0 |
| twz/mysql | 100 | 4 | 33.16 | 1005816 | 85 |
| jdbc-odbc:jet | 100 | 4 | 7.42 | 973768 | 0 |
| Driver/DB | Clients | TPC | Transactions/Sec | Max Mem | Failed Transactions |
| mm/mysql | 200 | 4 | 58.45 | 932080 | 0 |
| twz/mysql | 200 | 4 | 14.02 | 1020168 | 180 |
| jdbc-odbc:jet | 200 | 4 | 3.74 | 1002280 | 391 |
| Driver/DB | Clients | TPC | Transactions/Sec | Max Mem | Failed Transactions |
| mm/mysql | 300 | 4 | 63 | 932080 | 0 |
| twz/mysql | 300 | 4 | 10.12 | 1025028 | 261 |
| jdbc-odbc:jet | 300 | 4 | 2.55 | 986696 | 904 |