I’m getting more and more concerned about the current Oracle approach to MySQL security. And the fact that I was solely responsible for the security@mysql.com for about ten years, doesn’t make it easier, on the contrary, it only emphasizes changes in the attitude.

Starting from the obvious — somewhat slower response to critical bug fixes, which can be expected, Oracle is a big company, right? Very little information about security vulnerabilities is disclosed, CPUs are carefully stripped from anything that might help to understand the problem, it takes hours to map them to code changes. Heck, even test cases are kept private now. This seriously smells Security through Obscurity to me.

But all that isn’t news. Here I want to talk about the recent wave of security vulnerabilities. If you search for, say, “mysql security full-disclosure”, you will find the original postings to the full-disclosure@lists.grok.org.uk mailing list, as well as further discussion and my replies. Not Oracle replies, though. In short, the vulnerabilities  announced in the early December were:

  1. CVE-2012-5611 MySQL (Linux) Stack based buffer overrun PoC Zeroday
  2. CVE-2012-5612 MySQL (Linux) Heap Based Overrun PoC Zeroday
  3. CVE-2012-5613 MySQL (Linux) Database Privilege Elevation Zeroday
  4. CVE-2012-5614 MySQL Denial of Service Zeroday PoC
  5. CVE-2012-5615 MySQL Remote Preauth User Enumeration Zeroday
  6. CVE-2012-5627 MySQL Local/Remote FAST Account Password Cracking

All posted to the full disclosure mailing list, accompanied by exploits in Perl. Out of these issues, the 3rd was a configuration issue (FILE privilege granted to untrusted user, no --secure-file-priv used), 4th was already fixed in the latest MariaDB and MySQL releases (the reporter used an outdated MySQL version), 1st — the most dangerous one — was already fixed in the latest MariaDB release (we were lucky to discover this issue independently few weeks ealier and immediately released a fix). Everything else was valid, exploitable, and very much public, after the original postings were quoted in various blogs and news articles.

In a due time MySQL 5.5.29 is released. The 1st issue — CVE-2012-5611 — looks fixed. The second one — CVE-2012-5612 — is fixed too. Hurra! But 5th and 6th vulnerabilities — CVE-2012-5615 and CVE-2012-5627  — are not fixed. Which, apparently, leaves MySQL installations at the mercy of any script-kiddie, who can use google. And the 1st one (the worst of all) is fixed incompletely — the fact, that is simply impossible to miss, given the amount of testing that MySQL continuous integration framework (Pushbuild2) is performing.

Really, I wouldn’t expect a database vendor to blatantly ignore publicly announced vulnerabilities, that have CVE identifiers, known and easily available exploits, and were posted and reposted all over. So, Oracle, what was that, eh?

2013/Feb/05 UPDATE: MySQL 5.5.30 doesn’t seem to fix these issues either. Two months after the vulnerabilities went public. After MariaDB 5.5.29 release (with the fixes) and this blog post! I don’t think the Hanlon’s razor can explain that.

About a week ago I was looking at MySQL 5.5.27, and noticed a curious thing. Despite the fact that the new MySQL release contained its usual share of bug fixes, not a single one of them was accompanied with a test case.

Now, let me tell you something about tests. For many years MySQL was using its own testing framework, called mysql-test. The first version was written as early as 1999. Over the years it has accumulated a lot of tests. Tests for new features and regression tests — those that guarantee that a bug, once fixed, will never ever show up again. We had pretty strict policies about it in MySQL AB (and, later, Sun Microsystems) — every new bug fix always had to come with a test case for the bug. And because these tests were always run on many platforms for every push (by the continuous integration tool called Pushbuild — developed in-house by Kristian Nielsen) we were reasonably sure that any bug, once fixed, will stay fixed forever. I’m not with MySQL anymore, but I still cannot imagine that Oracle would weaken that rule. So, it must be something else then.

One of the changes that 5.5.27 brings in an extension to the mysql-test-run script — the driver script of the mysql-test suite — which makes it look for test cases in a new directory. In addition to the usual location (that is, the mysql-test/ directory in the source tree), it will now look for test cases in the internal/mysql-test/ directory. Does this mean that test cases are no longer open source? Oracle did not reply to my question. But indeed, there is evidence that this guess is true. For example, this commit mail shows that new test cases, indeed, go in this “internal” directory, which is not included in the MySQL source distribution.

MySQL test cases were always an important part of the MySQL source tree. They were particularly useful for storage engine developers and for other people extending MySQL, for example, at Facebook, Twitter, and Taobao.  But also for Linux distributions which add their patches to the base MySQL, and even to users, who don’t modify the sources — they still want to confirm that a particular bug was fixed or that their custom-built binary has no obvious flaws.

In May, at the Ubuntu Developer Summit in Oakland, Oracle had 7 representatives there, and they promised that Oracle will be more contributor- and distribution-friendly. It is sad to see that instead of that the MySQL source tree is being closed down.

MySQL AB was never very good at building a development community around the product. There weren’t many MySQL developers or contributors to the project outside of MySQL AB, and the company didn’t do much to increase their number. But now Oracle has noticed them — and it intentionally kills whatever is left of the MySQL development community. Without test cases MySQL becomes as opaque to external developers as any piece of closed source software, and only those most experienced and familiar with the MySQL code base will be able to continue working with it.

UPDATE: It’s difficult to find anything more valuable to external developers than test cases. But arguably the revision history is. It groups changes to these millions of lines of source code into change sets, one change set per a distinct feature or a particular bug fix. It allows to see who changed a specific line of code, when, and why. And it seems that Oracle is going to keep this information to itself too. Public MySQL trees on launchpad with the revision history are not being updated.

Now, is there a single successful Open Source project with no developer community at all ?

UPDATE 2: We’re not the only ones this issue affects. See also:

 

In this primer I will show how to improve the security of your MariaDB installation by using two-step verification and how to use it from your Windows GUI client.

Let’s suppose you have your data in MariaDB, installed, say, on Ubuntu. And your users connect to it to run ad hoc queries, using some sort of a Windows GUI client. You don’t want them to write the access password on post-it notes or have it auto-entered by the client. And you don’t want anyone see the password when one of the salespersons connects to the mother ship from his laptop in the Internet café. So you decide to use the two-step verification, just like Google does, to secure the access to the data.

Continue reading