Last weekend Vadim from Percona published his MariaDB 5.3.4 benchmark results. As the new benchmark guy at Monty Program I take this oportunity to add some more results of my own.

One question in the comments to Vadim was if it is fair to compare MariaDB-5.3 with MySQL-5.5. Or if this comparison should be done with MySQL-5.1. The answer is: it does not matter much. MySQL-5.5 and MySQL-5.1 show very similar results in the Sysbench OLTP benchmark.

So I created a Sysbench environment pretty much like Vadims and tested the following versions of the MySQL Server:

  • MariaDB-5.3.4 – the Monty Program release candidate, both with XtraDB and the InnoDB plugin
  • Percona-Server 5.1.61 because it is based on the same XtraDB version as MariaDB-5.3
  • Percona-Server 5.5.20 – the current Percona flagship
  • MySQL-5.5.20 – the current Oracle flagship

The result:

Indeed MariaDB-5.3.4 scales significantly worse than MySQL-5.5.20. However Percona-Server 5.1.61 behaves nearly identical and Percona-Server 5.5.20 is only slightly better. If we run MariaDB-5.3.4 with the InnoDB plugin instead of the default XtraDB, then MariaDB outperforms Percona-Server 5.5.20.

As a consequence, we will deliver the InnoDB plugin with future binary releases of MariaDB. MySQL-5.1.61 with the InnoDB plugin (not shown in the graph) is between MariaDB-5.3.4(InnoDB-plugin) and MySQL-5.5.20. We are currently investigating why MySQL-5.1 is faster than MariaDB-5.3, even with the same storage engine.

Please keep in mind that Sysbench is a highly synthetical benchmark. Real world workload will probably yield different results. Also this benchmark used solid state disks. A storage system with spinning disks might saturate much earlier and would then flatten out the peaks.

If I should draw a conclusion, then I would say that for highest concurrency levels you should go for MySQL-5.5. Or wait for MariaDB-5.5 :)

If your workload includes complex (sub)queries, then you will probably benefit more from MariaDBs new optimizer features. And if you are running replication, then MariaDB 5.3 with group-commit will greatly outperform any other MySQL version.

Benchmark details: sysbench OLTP multi-table. 16 tables with 250K rows each. 20G InnoDB buffer pool. Data and logs on SSD. Dual socket hexacore Xeon with HT enabled (24 logical cores). You can download the tarball with benchmark scripts, config and raw results.

Instead of the usual text-heavy blog posts that appear here, I thought it would be fun to mix things up and do a screencast showing exactly how easy it is to upgrade MySQL to MariaDB:

MariaDB Screencast: Installing MariaDB
Watch this video on YouTube.

Some notes:

  • The laptop I’m using had MySQL 5.1.55 installed with one database (apart from the system database). Installing MariaDB does not impact existing data in any way and once the install completed I had instant access to my data.
  • As part of the install you are given the option to set a new password for the root user. I choose to do it in the video, but you don’t need to. If you leave the password field blank the root password will not be changed. Other database users are preserved, of course.
  • As with any database upgrade, before doing this to a production system you should have backups and test.

Links:

Links shown or mentioned in the video:

Comments?

What do you think? Should we make more screencasts? If so, what would you like to see demonstrated?

The MySQL community has something new on their radar. First up, it looks like MySQL is now part of Oracle Software Security Assurance, and this is something all MySQL users should be happy about. Next, it is worth noting that MySQL is now part of the Oracle Critical Patch Update (Oracle CPU), as the MySQL product line has made it into its first Oracle CPU advisory for January 2012.

As part of the MySQL community, CPU’s are new to us — they are released on the Tuesday closest to the 17th day of January, April, July and October. This kind of reminds us of Patch Tuesday, but let’s not digress.

This is the first time MySQL is part of the Critical Patch Update, and the advisory suggests that there are 27 new security fixes for Oracle MySQL, with one of the vulnerabilities having the possibility of remote exploitation without authentication. As developers of a MySQL branch we are naturally concerned towards the nature of these CPU’s.

For starters, it’s good to note that MariaDB is always based from a branch of MySQL (MySQL 5.1 for MariaDB 5.1, 5.2 & 5.3, and MySQL 5.5 for MariaDB 5.5). So whenever there are security fixes which Oracle makes into MySQL 5.1 or MySQL 5.5, we inherit them. This is one of the benefits of being a branch as opposed to being a fork.

“Oracle advisories include all issues that appeared since the last advisory. But this is the first advisory for MySQL. So either Oracle found 27 new problems since October 2011 or this includes everything that’s been outstanding,” said Sergei Golubchik, VP of Architecture for MariaDB and former MySQL security contact when I asked him about the 27 security fixes.

Upon looking up all the CVE numbers, the reports were vague, like “Unspecified vulnerability in the MySQL Server component in Oracle MySQL 5.1.x and 5.5.x allows remote attackers to affect availability via unknown vectors.” Additionally, the reports do not reference bug numbers, so from a bit of guesswork, we might assume that this commit is possibly the fix for the most serious vulnerability — the one that can be remotely exploited without authentication. That bug, incidentally, was fixed in May 2011, and has long been present in both MySQL and MariaDB (though our implementation varies from upstream).

We notice most CVEs being reported in January 2012, but have no idea when they were reported to the Oracle bug database (or to bugs.mysql.com), or when they were fixed. We believe that this is perhaps Oracle including MySQL into their Software Security Assurance program, which is what triggered all security bugs to be reported on cve.mitre.org, all on the same day.

Whether these 27 fixes are new or existing ones now being bundled up and reported in a Critical Patch Update remains open until more accurate information on what bugs they address is provided. We’re actively working on finding out the answer.