Back when the first version of the MariaDB Java Client was released, someone asked in the comments about the performance characteristics of the driver compared to ConnectorJ. I answered with hand-waving, saying that nobody does anything stupid, the performance of the drivers would be roughly the same, but I promised to measure it and tell the world one day. And now that day has come. The day where three MySQL JDBC drivers (ConnectorJ, MariaDB JDBC, and Drizzle JDBC) are compared against each other. Unlike the server, which gets benchmarking attention all the time, there is no standard benchmark for connectors, so I needed to improvise, while trying to keep the overhead of the server minimal. So I did something very primitive to start. I used my two favorite queries:

  • DO 1 — this one does not retrieve a result set, and thus can be seen as a small “update”.
  • SELECT 1 — the minimal SELECT query.

The test program runs a query N times, and if the query was a select, it retrieves all values from the result set, using ResultSet.getObject(i), and calculates the queries-per-second value. (The best thing is that the test program is single-threaded, and how often does one get to run single-threaded tests? :)  the test was run on my own workstation, which runs Windows Server 2008 R2, and I have useConfigs=maxPerformance in the URL for ConnectorJ.

Results (Queries per second,  unprepared)

ConnectorJ-5.1.24 MariaDB-JDBC-1.1.2 Drizzle-JDBC-1.3-SNAPSHOT
DO 1 19543 22104 15288
SELECT 1 17004 19305 13410

jdbc_fast_queries

 

MariaDB JDBC appears to be a little faster (~10%) than  ConnectorJ, and much faster (~30%) than Drizzle JDBC.

Can ConnectorJ do better? I bet it can. Looking into profiler output – CPU profiling, instrumentation mode in NetBeans – for  a test that executes “SELECT 1″ in a loop,  shows com.mysql.jdbc.StatementImpl.findStartOfStatement() taking 7.5% of runtime. Ok, instrumentation results should be taken with a grain of salt, however the single reason string search is used, is because - if an update (DML) statement is executed inside ResultSet.executeQuery(), it is rejected with an exception. This can be done differenty, I believe. If absolutely necessary, throwing an exception can be delayed, until the client finds out that the server sent an OK packet instead of a result set.

Even more interesting is the case with Drizzle JDBC. In theory, since the MariaDB driver has a Drizzle JDBC heritage, the performance characteristics should be similar, but they are not, so there must be a bug somewhere. It appears very easy to find, as according to profiler, 50.2% CPU time (take that number with a big grain of salt) is spent in a function that constructs a hexdump from a byte buffer. Looking at the source code, we find following line that is unconditionally executed:

log.finest("Sending : " + MySQLProtocol.hexdump(byteHeader, 0));

While the result of the hexdump is never used (unless logging level is FINEST), the dump string is still created, using relatively expensive Formatter routines, concatenated with the String “Sending:”, and then thrown away… In Markus’ defense, hexdump() is not his fault, it was contributed 3 years ago. But it remained undetected for 3 years. This bug is now filed  https://github.com/krummas/DrizzleJDBC/issues/21 [UPDATE: this bug was resolved within hours  after reporting]

So, let’s check how much we can gain by putting the offending code into an if (log.getLevel() == java.util.logging.Level.FINEST) condition.
The QPS from “DO 1″ raises from 15288 to 19968 (30%), and for “SELECT 1″ we have increase from 13410 to respectable 16824 (25%). Not bad for a single line fix.
jdbc_fast_queries_drizzle_fix

While the one-liner makes the Drizzle JDBC faster, with slightly better numbers than ConnectorJ, it is still not as fast as MariaDB.

In the MariaDB JDBC connector, there were a couple of improvements to performance which were made since forking. One of the early improvements was to avoid copying data unnecessarily when sending, and to decrease the number of byte buffers.  Another improvement came recently, after profiling and finding that parsing Field packets is expensive (mostly due to the construction of Strings for column name, aliases, and etc…). The improvement was lazy parsing,  delaying string construction, and avoiding it entirely in most cases. For example, if column names are not used, and rows are accessed using integer indexes in ResultSet.getXXX(int i), the metadata won’t be fully parsed. Also, perhaps there were some other fixes that I do not remember anymore. :)

Can we further increase the QPS?

We can try. First, statements can be prepared. MariaDB and Drizzle so far only provide client-side prepared statements (ConnectorJ can do both client and server-side prepared statements) but using them saves having to convert the query to bytes, and JDBC escapes preprocessing. From now on I’ll stay just with “DO 1″ which proved to be the fastest query. Trying it on MariaDB driver shows some minimal QPS increase 22104 (not prepared) vs 22183 (prepared), or 0.3%. Slightly more on ConnectorJ (19543 vs 20096, or 2.9%). Nothing revolutionary so far.

But, We still have not used all of the options in this (admittedly silly) quest for maximizing the performance of “DO 1″. Recall that ConnectorJ can support named pipes on Windows, which are allegedly much faster than TCP connections. Restart server with named pipe, set JDBC URL to “jdbc:mysql:///?socketFactory=com.mysql.jdbc.NamedPipeSocketFactory&namedPipePath=\\\\.\\Pipe\\MySQL&user=root&useConfigs=maxPerformance”, and rerun the test with 1000000 prepared queries. Now the QPS grew to 29542! That is strong, and is a 33% improvement compared to the best result seen so far. Yet, unfortunately, still no cigar, since JVM dumps a stack trace when the named pipe connection is closed. This is a “Won’t fix” (chalked off as a JVM problem) MySQL bug Bug#62518, which renders named pipe support almost useless – though maybe there is a trick to shut up th JVM somehow in this case, but I do not know of such a trick.

How fast is C client library in comparison?

Out of curiosity, I also tested how the native client compares to JDBC. With the TCP protocol, it does slightly better than the fastest JDBC (MariaDB, prepared), but it is not a huge margin – 24063 QPS vs 22183 (8.5% difference), and I believe Java drivers could improve further.
With named pipe, QPS is 33122, which is ~12% better than what ConnectorJ could do, if pipes worked properly there.

 

Accessing benchmark program

I put the benchmark program on Launchpad, together with the drivers. If you’re on Windows, and if you have a server running on port 3306, and the ‘root’ user doesn’t have a password, you can just branch the repository and run bench_all.bat. Those of you who are using other operating systems, I trust you to be able to quickly rewrite the batch files as shell scripts.

The MariaDB project is pleased to announce the immediate availability of the MariaDB Java Client 1.1.2. This is a Stable (GA) release. See the Release Notes and Changelog for detailed information on this release and the About the MariaDB Java Client page in the AskMonty Knowledgebase for general information about the client.

Download MariaDB Java Client 1.1.2

Release Notes Changelog About the MariaDB Java Client

Bugs fixed in this release

MariaDB Java Client 1.1.2 is a bug fix release. Some of the bugs fixed include the following:

  • PreparedStatement.getMetaData() will now return correct ResultSet metadata, also prior to statement execution. In this case, to retrieve metadata, statement will be prepared on the server side. (CONJ-21)
  • Performance enhancement : Connection.getAutoCommit() will not more issue not issue “select @@autocommit” query anymore. This information is also available in on status flags sent with OK/EOF protocol packets. Also, Connection.setAutoCommit() will be a no-op, if autocommit status is already the same as desired one (CONJ-30)
  • Fixed several issues with CLOB datatype and ResultSet.setCharacterStream() -non-ASCII character could get lost after setCharacterStream(). CLOB was errneously sent to server as binary (using _BINARY introducer). (CONJ-31)
  • CHAR BINARY and VARCHAR BINARY were errnously handled as binary type. They are now correctly treated as CHAR/VARCHAR with binary collation. (CONJ-28)
  • Blob.getBinaryStream() could return blob with incorrect size (CONJ-31)
  • Fix server version string for MariaDB Server 10.0.2 (CONJ-32)

The MariaDB Java Client 1.1.1 has been released. You can download it here.

See the Release Notes and Changelog for detailed information on this release and the About the MariaDB Java Client page in the AskMonty Knowledgebase for general information about the client.

New functionality in this release

  • Implement tcpAbortiveClose option, for “hard” socket close (CONJ-27)
    • This option can be used in environments where connections are created and closed in rapid succession. Often, it is not possible to create a socket in such environment after a while, since all local “ephemeral” ports are used up by TCP connections in TCP_WAIT state. Using tcpAbortiveClose works around this problem by resetting TCP connections rather (abortive or hard close) than doing an orderly close. It is accomplished by using socket.setSoLinger(true,0) for abortive close.

Bugs fixed in this release

  • MySQLStatement will now indicate there are no more results by returning -1 from getUpdateCount() and null from getResultSet(),as mandated by the standard (CONJ-14)
  • Introduced nullCatalogMeansCurrent parameter for compatibly with ConnectorJ, and make it default (CONJ-16)
    • Prior to this change, DatabaseMetadata.getTables() or other methods of DatabaseMetaData that accept catalog names and return result sets would treat null as prescribed by the JDBC standard (null means no restriction which catalog is used). This behavior is now changed for the sake of compatibility. Starting with 1.1.1 , null for catalog name will mean current catalog. To get JDBC standard behavior, one needs to set nullCatalogMeansCurrent=false.
  • DatabaseMedataData.getColumns() returned incorrect values in the “COLUMN_SIZE” column for character data.(CONJ-15))
    • Prior to the change, octet size was returned (length in bytes). Depending on the character set used, it could be 3 times bigger than the length in characters that was specified by CREATE or ALTER table. This behavior is corrected in 1.1.1
  • DatabaseMetaData.getColumns() always handled MySQL YEAR datatype as SMALLINT. (CONJ-19)
    • The behavior is now fixed and getColumns returns either DATE or SMALLINT depending on how the ‘yearIsDateType’ parameter is set.
  • ResultSetMetaData.getColumnName() returned an empty string in special cases (CONJ-17)
    • ResultSetMetaData.getColumnName() returned and empty string for “non-columns” in a result set (functions, aggregates like count(*), and so on). The fix is to return the column label to be returned if the column name is empty.
  • Ensure that getObject() returns byte array for CHAR BINARY.(CONJ-20)
    • Also make sure that getColumnType(),getColumnClassName(),getColumnTypeName() return values indicate BINARY for fixed binary type.
  • JVM does not exit if statement timeout is used.(CONJ-23)
    • Constructor for Timer was corrected to ensure that the Timer thread has type background thread. Thus Timer won’t prevent JVM from exiting.
  • Calling first() on “streaming” result set and using the result set afterwards generated NullPointerException (CONJ-24)
    • Now a SQLException is thrown early on, in the first() call, since streaming results sets are not scrollable.
  • Connection.close() hangs if there is an open streaming result set, and next() was not called on this result set (CONJ-25)