Today we upgraded the PCRE library bundled with MariaDB-10.0 to PCRE-8.34. This PCRE release includes some improvements, fixes for better stability and performance, and gives more compatibility with the Perl regular expressions.

I’d like to give details on the PCRE changes that especially affected MariaDB.

PCRE now includes support for [[:<:]] and [[:>:]]  as used in the BSD POSIX library (written by Henry Spencer) to mean “start of word” and “end of word“, respectively. This is a good news for those project (like MariaDB) migrating from the Henry Spencer’s library to PCRE, as this non-standard syntax seemed to be used quite widely. Many thanks to Philip Hazel and the PCRE team who kindly added this extension into PCRE and who gave us a patch before the final 8.34 release, so we were able to fix an incompatibility with MySQL RLIKE earlier (see “MDEV-5357 REGEXP word boundaries don’t work“, fixed in Maria-10.0.7).

PCRE-8.33 has also fixed a crash caused by stack overrun in pcre_compile() in cases when the pattern contains a very deep level of nested parenthesis. PCRE now has a compile-time limit (250 by default) on the depth of nesting of parentheses. This works perfectly fine with programs using the OS default stack size settings, and instead of crashing, pcre_compile() now returns an error safely. However, unfortunately, this new limit did not help us, because MariaDB uses a smaller individual thread stack size, needed to handle dozen thousands concurrent connections and controlled by the @@thread_stack MariaDB system variable with the default value of 288Kb (versus the default Posix thread stack size of 8Mb). With the default @@thread_stack=288Kb, MariaDB still would crash with the verbatim copy of PCRE-8.34 on a query like this:

The exact number that would hit the crash might vary on different operating systems. It was about 210 during my tests on a Fedora box, which a little bit smaller than the new PCRE default limit

To prevent the crash we had to keep our patch that adds a callback function into PCRE. See pcre/mariadb-patches/pcre_stack_guard.diff for details. pcre_compile() calls this callback function every time when a parenthesis is met in the regular expression pattern, before going into the next recursion level. If the thread stack size gets dangerously small, mysqld indicates this by the callback function result, pcre_compile() returns with an error, and the entire SQL query display the error message:

If you give more stack size by starting mysqld with say --thread-stack=589824, threads don’t run out of stack too early, so the compiled PCRE limit is hit instead, which is indicated by a different error message:

We’ll be watching if the future versions of PCRE add some built-in means to control the used stack size. In the meanwhile, we’ll have to keep compiling the bundled patched version even though if PCRE is installed in the system.

Please see http://www.pcre.org/changelog.txt for the full list of changes in PCRE-8.34. We’d like to thank the PCRE team for a good release.

 

It is not a secret that we’ve been kicking the tires and playing with JIRA for project management. After using it since the beginning of the year most of us like the feel of it and we’ve decided that it makes sense to start using it more.

As you know, the MariaDB project has many fragmented resources. We report bugs in Launchpad. We store our plans in worklog. We’ve never used the Launchpad Blueprint feature for this very reason. We don’t use Launchpad Answers because we have the Knowledgebase.

With this move to hosted JIRA (yes, this is an important link: http://mariadb.org/jira) we can report bugs, have future plans, and also give users a roadmap which is pretty cool. One nifty feature is that in the past two releases, we had a roadmap and we didn’t slip in terms of a schedule. We had on time releases and that’s awesome!

So what does this mean for you? To report bugs, you will now do so on JIRA. To make feature requests and talk about our future plans, you will also now use JIRA.

We plan to deprecate Worklog and Launchpad bugs by 30 June 2012. Launchpad will however continue to host the source code for MariaDB.

What will happen to bugs already reported on Launchpad? We have migration scripts ready for this and when we press the button bug reports will nicely migrate over to JIRA. After that is done we’ll place a notification on the MariaDB bugs page in Launchpad about reporting new bugs in JIRA.

What will happen to feature requests and ideas already in worklog? Worklog will be put into a read-only mode and there will be notifications about the move to JIRA. Whenever needed we’ll copy & paste worklog items into JIRA.

What does it mean to the openness of the MariaDB project? It’s not affected at all. The MariaDB project will remain an open community friendly project and as a bonus it will be easier to follow what is going on in the project since you don’t have to jump between several tools to get the complete picture.

The consolidation to JIRA provides the means to report and track project status easier than before, which allows the MariaDB team, community members and other to better coordinate and prioritize work.

As a side note, JIRA (and other software by Atlassian) has sometimes been criticized in the open source world because of its commercial nature and many are unaware of the fact that Atlassian do offer a free Open Source Project License to open source projects, which is what is being used with MariaDB.

As another side note, I’m not going to dive into comparing features in e.g. Launchpad with features in JIRA. I do know it would be possible to use blueprints for feature specifications etc. in Launchpad. The most important aspect in my mind is that you pick a tool that you like the feel of, has the features you need and tightens collaboration between developers, project managers, community members and other persons involved in the project.

In short, this is all about three project tools becoming one.

As you may know, since version 5.2.0 (released in April 2010) we support Pluggable Authentication. Using this feature one can implement an arbitrary user authentication and account management policy, completely replacing built-in MariaDB authentication with its username/password combination and mysql.user table.

Also, as you might have heard, Oracle has recently released a PAM authentication plugin for MySQL. Alas, this plugin will not run on MariaDB — although the MySQL implementation of pluggable authentication is based on ours, the API is incompatible. And, being closed source, this plugin cannot be fixed to run in MariaDB. And — I’m not making it up — this plugin does not support communication between the client and the server, so even with this plugin and all the power of PAM the only possible authentication method remains a simple username/password combination.

But writing authentication plugins is easy, I said to myself. I will do my own authentication plugin! With blackjack and hookers.

Continue reading