The Board of the MariaDB Foundation thought it would be good to provide an update — hopefully the first of a regular quarterly series — on how we’re progressing with the interim activities around constructing governance, identifying a new representative Board and structuring an engineering council.

  • The MariaDB Foundation is now independent of any business interests. With the acquisition of Monty Program Ab by SkySQL Ab, there’s now a clear separation of functions. The Foundation is the home of community activity around MariaDB, dedicated to sustaining and delivering the MariaDB database on behalf of its community independently of the business interests of any member. We are acting as a US 501(c)(6) not-for-profit and intend to follow the advice we have received to formally register with the IRS at the start of 2014.
  • Support from commercial backers is steadily flowing; you can see some of them listed on our new Supporters page, which is still being updated. We have two different kinds of supporters.
    • Commercial Sponsors are businesses within the MariaDB Community who want to team together to keep the infrastructure and administration running. In return, we offer them recognition via our Supporters Page.
    • Commercial Members are businesses within the MariaDB Community who additionally want to participate directly in the governance oversight. They will be helping the Board through our new Advisory List and will also be identifying up to half of the members of the new Board when we switch to permanent governance (the other half will be community contributors to the code). I’m pleased to have three generous supporters in this category already.
    • While we now have a good number of Commercial Sponsors, we still need more and I encourage you to offer sponsorship if your business uses MariaDB. If you have a high strategic dependence on MariaDB, I encourage you to consider Commercial Membership.
  • We have a small staff team in place. I’m acting as CEO in the interim period on a part-time basis, with Monty serving as my CTO and structuring our engineering functions. Working for him are two developers and a part-time administrator, with a current focus on documentation. We also have a General Counsel, Andrew Katz, supported by US attorney Andy Updegrove, and we’re retaining a “virtual CFO” through Virtual Management, Inc. I expect to add a membership director and further developers to the team in the coming months. I hope our overall staff team will peak at about 8 FTE by the end of the year.
  • The interim Board has met regularly. It comprises Sergei Golubchik, Rasmus Johansson, Andrew Katz, myself, Monty Widenius and Jeremy Zawodny. Since the governance we envisage is based on the Eclipse Foundation, we have also invited its executive director Mike Milinkovich to attend all our meetings as an advisor.
  • The Board is keen for MariaDB to play a full part in the wider community of open source communities. Consequently, we have now joined the Open Source Initiative as an Affiliate Member, and we have become a licensee of the Open Invention Network.
  • We’re now ready to start the next key element of the Foundation’s transformation, establishing community-based governance. We expect the permanent governance for the Foundation to reflect both the developer and business needs of the MariaDB Community, with the Board being equally divided between community-recognised technical contributors to MariaDB and Corporate Members. We plan to conduct public discussion of new Bylaws during August 2013.
  • We’re just starting discussion of how to structure an engineering council to act as the focus for consensus around MariaDB itself, very much along the lines of the current MariaDB Captains. We have not discussed any proposal to make any changes to the licensing of MariaDB and I don’t expect any such proposal to be made; we’re sticking with the GPL. Rather, I’m expecting the community around MariaDB to continue to strengthen and grow as an open community.

If you would like to discuss any aspect of the Foundation with me – especially if your company would like to become a Sponsor or Member — please get in touch. My e-mail address is webmink at mariadb org.

The MariaDB project is pleased to announce the immediate availability of MariaDB 10.0.3. This is an alpha release. See the release notes and changelog for details.

Download MariaDB 10.0.3

Release Notes Changelog What is MariaDB 10.0?

APT and YUM Repository Configuration Generator

About this Release

MariaDB 10.0 is the development version of MariaDB. It is built on the MariaDB 5.5 series with backported and reimplemented features from MySQL 5.6 and entirely new features not found anywhere else.

This is the fourth release in the MariaDB 10.0 series. We are releasing it now to get it into the hands of any who might want to test it. Not all features planned for this series are included in this release. Additional features will be pushed in future releases. See the release notes and changelog for details on what is new in this release.

Do not use alpha releases on production systems.

User Feedback plugin

MariaDB includes a User Feedback plugin. This plugin is disabled by default. If enabled, it submits basic, completely anonymous MariaDB usage information. This information is used by the developers to track trends in MariaDB usage to better guide development efforts.

If you would like to help make MariaDB better, please add “feedback=ON” to your my.cnf (my.ini on Windows) file!

See the User Feedback Plugin page for more information.

Quality

The project always strives for quality, but in reality, nothing is
perfect. Please take time to report any issues you encounter at:

http://mariadb.org/jira

We hope you enjoy MariaDB!

Back when the first version of the MariaDB Java Client was released, someone asked in the comments about the performance characteristics of the driver compared to ConnectorJ. I answered with hand-waving, saying that nobody does anything stupid, the performance of the drivers would be roughly the same, but I promised to measure it and tell the world one day. And now that day has come. The day where three MySQL JDBC drivers (ConnectorJ, MariaDB JDBC, and Drizzle JDBC) are compared against each other. Unlike the server, which gets benchmarking attention all the time, there is no standard benchmark for connectors, so I needed to improvise, while trying to keep the overhead of the server minimal. So I did something very primitive to start. I used my two favorite queries:

  • DO 1 — this one does not retrieve a result set, and thus can be seen as a small “update”.
  • SELECT 1 — the minimal SELECT query.

The test program runs a query N times, and if the query was a select, it retrieves all values from the result set, using ResultSet.getObject(i), and calculates the queries-per-second value. (The best thing is that the test program is single-threaded, and how often does one get to run single-threaded tests? :)  the test was run on my own workstation, which runs Windows Server 2008 R2, and I have useConfigs=maxPerformance in the URL for ConnectorJ.

Results (Queries per second,  unprepared)

ConnectorJ-5.1.24 MariaDB-JDBC-1.1.2 Drizzle-JDBC-1.3-SNAPSHOT
DO 1 19543 22104 15288
SELECT 1 17004 19305 13410

jdbc_fast_queries

 

MariaDB JDBC appears to be a little faster (~10%) than  ConnectorJ, and much faster (~30%) than Drizzle JDBC.

Can ConnectorJ do better? I bet it can. Looking into profiler output – CPU profiling, instrumentation mode in NetBeans – for  a test that executes “SELECT 1″ in a loop,  shows com.mysql.jdbc.StatementImpl.findStartOfStatement() taking 7.5% of runtime. Ok, instrumentation results should be taken with a grain of salt, however the single reason string search is used, is because - if an update (DML) statement is executed inside ResultSet.executeQuery(), it is rejected with an exception. This can be done differenty, I believe. If absolutely necessary, throwing an exception can be delayed, until the client finds out that the server sent an OK packet instead of a result set.

Even more interesting is the case with Drizzle JDBC. In theory, since the MariaDB driver has a Drizzle JDBC heritage, the performance characteristics should be similar, but they are not, so there must be a bug somewhere. It appears very easy to find, as according to profiler, 50.2% CPU time (take that number with a big grain of salt) is spent in a function that constructs a hexdump from a byte buffer. Looking at the source code, we find following line that is unconditionally executed:

log.finest("Sending : " + MySQLProtocol.hexdump(byteHeader, 0));

While the result of the hexdump is never used (unless logging level is FINEST), the dump string is still created, using relatively expensive Formatter routines, concatenated with the String “Sending:”, and then thrown away… In Markus’ defense, hexdump() is not his fault, it was contributed 3 years ago. But it remained undetected for 3 years. This bug is now filed  https://github.com/krummas/DrizzleJDBC/issues/21 [UPDATE: this bug was resolved within hours  after reporting]

So, let’s check how much we can gain by putting the offending code into an if (log.getLevel() == java.util.logging.Level.FINEST) condition.
The QPS from “DO 1″ raises from 15288 to 19968 (30%), and for “SELECT 1″ we have increase from 13410 to respectable 16824 (25%). Not bad for a single line fix.
jdbc_fast_queries_drizzle_fix

While the one-liner makes the Drizzle JDBC faster, with slightly better numbers than ConnectorJ, it is still not as fast as MariaDB.

In the MariaDB JDBC connector, there were a couple of improvements to performance which were made since forking. One of the early improvements was to avoid copying data unnecessarily when sending, and to decrease the number of byte buffers.  Another improvement came recently, after profiling and finding that parsing Field packets is expensive (mostly due to the construction of Strings for column name, aliases, and etc…). The improvement was lazy parsing,  delaying string construction, and avoiding it entirely in most cases. For example, if column names are not used, and rows are accessed using integer indexes in ResultSet.getXXX(int i), the metadata won’t be fully parsed. Also, perhaps there were some other fixes that I do not remember anymore. :)

Can we further increase the QPS?

We can try. First, statements can be prepared. MariaDB and Drizzle so far only provide client-side prepared statements (ConnectorJ can do both client and server-side prepared statements) but using them saves having to convert the query to bytes, and JDBC escapes preprocessing. From now on I’ll stay just with “DO 1″ which proved to be the fastest query. Trying it on MariaDB driver shows some minimal QPS increase 22104 (not prepared) vs 22183 (prepared), or 0.3%. Slightly more on ConnectorJ (19543 vs 20096, or 2.9%). Nothing revolutionary so far.

But, We still have not used all of the options in this (admittedly silly) quest for maximizing the performance of “DO 1″. Recall that ConnectorJ can support named pipes on Windows, which are allegedly much faster than TCP connections. Restart server with named pipe, set JDBC URL to “jdbc:mysql:///?socketFactory=com.mysql.jdbc.NamedPipeSocketFactory&namedPipePath=\\\\.\\Pipe\\MySQL&user=root&useConfigs=maxPerformance”, and rerun the test with 1000000 prepared queries. Now the QPS grew to 29542! That is strong, and is a 33% improvement compared to the best result seen so far. Yet, unfortunately, still no cigar, since JVM dumps a stack trace when the named pipe connection is closed. This is a “Won’t fix” (chalked off as a JVM problem) MySQL bug Bug#62518, which renders named pipe support almost useless – though maybe there is a trick to shut up th JVM somehow in this case, but I do not know of such a trick.

How fast is C client library in comparison?

Out of curiosity, I also tested how the native client compares to JDBC. With the TCP protocol, it does slightly better than the fastest JDBC (MariaDB, prepared), but it is not a huge margin – 24063 QPS vs 22183 (8.5% difference), and I believe Java drivers could improve further.
With named pipe, QPS is 33122, which is ~12% better than what ConnectorJ could do, if pipes worked properly there.

 

Accessing benchmark program

I put the benchmark program on Launchpad, together with the drivers. If you’re on Windows, and if you have a server running on port 3306, and the ‘root’ user doesn’t have a password, you can just branch the repository and run bench_all.bat. Those of you who are using other operating systems, I trust you to be able to quickly rewrite the batch files as shell scripts.