Today we upgraded the PCRE library bundled with MariaDB-10.0 to PCRE-8.34. This PCRE release includes some improvements, fixes for better stability and performance, and gives more compatibility with the Perl regular expressions.

I’d like to give details on the PCRE changes that especially affected MariaDB.

PCRE now includes support for [[:<:]] and [[:>:]]  as used in the BSD POSIX library (written by Henry Spencer) to mean “start of word” and “end of word“, respectively. This is a good news for those project (like MariaDB) migrating from the Henry Spencer’s library to PCRE, as this non-standard syntax seemed to be used quite widely. Many thanks to Philip Hazel and the PCRE team who kindly added this extension into PCRE and who gave us a patch before the final 8.34 release, so we were able to fix an incompatibility with MySQL RLIKE earlier (see “MDEV-5357 REGEXP word boundaries don’t work“, fixed in Maria-10.0.7).

PCRE-8.33 has also fixed a crash caused by stack overrun in pcre_compile() in cases when the pattern contains a very deep level of nested parenthesis. PCRE now has a compile-time limit (250 by default) on the depth of nesting of parentheses. This works perfectly fine with programs using the OS default stack size settings, and instead of crashing, pcre_compile() now returns an error safely. However, unfortunately, this new limit did not help us, because MariaDB uses a smaller individual thread stack size, needed to handle dozen thousands concurrent connections and controlled by the @@thread_stack MariaDB system variable with the default value of 288Kb (versus the default Posix thread stack size of 8Mb). With the default @@thread_stack=288Kb, MariaDB still would crash with the verbatim copy of PCRE-8.34 on a query like this:

The exact number that would hit the crash might vary on different operating systems. It was about 210 during my tests on a Fedora box, which a little bit smaller than the new PCRE default limit

To prevent the crash we had to keep our patch that adds a callback function into PCRE. See pcre/mariadb-patches/pcre_stack_guard.diff for details. pcre_compile() calls this callback function every time when a parenthesis is met in the regular expression pattern, before going into the next recursion level. If the thread stack size gets dangerously small, mysqld indicates this by the callback function result, pcre_compile() returns with an error, and the entire SQL query display the error message:

If you give more stack size by starting mysqld with say --thread-stack=589824, threads don’t run out of stack too early, so the compiled PCRE limit is hit instead, which is indicated by a different error message:

We’ll be watching if the future versions of PCRE add some built-in means to control the used stack size. In the meanwhile, we’ll have to keep compiling the bundled patched version even though if PCRE is installed in the system.

Please see http://www.pcre.org/changelog.txt for the full list of changes in PCRE-8.34. We’d like to thank the PCRE team for a good release.

 

A very happy new year to you from the MariaDB Foundation. Since I last wrote in June the Foundation has been doing plenty of work on everyone’s behalf.

First there’s the extensive technical work that Monty wrote about, which has been carried out by the team the Foundation employs who all work for Monty. He would like to grow that team a little, so your donations – or even corporate sponsorship or membership — would be especially welcome at the moment. With the move of the Monty Program staff he used to employ over to SkySQL, there’s no longer any cross-subsidy between consulting work and the fix-sustain-build mission of the Foundation’s core team, so we depend completely on donations and corporate support to make that happen.

Second, we’ve been growing the Foundation membership. During 2013 we gained three Corporate Members — Booking.Com, Parallels and SkySQL — to whom we are very grateful for the support that has encouraged us to keep going and paid the bills. We’re starting 2014 with new Corporate Member enrollments. There are two others in progress that I hope to announce soon, but I’m delighted to say that Automattic, the company behind the WordPress this blog uses, has become a Corporate Member. Many thanks to them for their generous support.

During 2013 we also had a number of Corporate Sponsors — Percona, Jelastic/Planeetta , WebYog and Open Query — whose financial contributions have been most welcome in addition to their community involvement. We are starting 2014 with several new Sponsors who I’ll announce with the new Members soon. I hope we will be able to welcome more Members and Sponsors this season.

Thanks too for all the individual donations we have received. If you are able to send a small amount each time you download from the Foundation, your contributions will soon add up to a substantial sum, as we’ve seen a steady increase in downloads. We very much welcome these “subscription donations” as they help me plan our budget.

Third, we’ve been working on new, community-led governance. The bylaws are now drafted and ready for community comment. Once the trademark paperwork from SkySQL is confirmed, I’ll put them online for review and discussion. It’s taken longer than I had hoped to reach this point, partly because we needed to wait for the merger of the former Monty Program Ab into SkySQL to complete. This happened on September 30 so we’re now good to go. I hope we will get the whole process started this quarter.

One key question we need to address is use of the MariaDB trademark. SkySQL acquired it along with Monty Program Ab, but as you’ll note from their CEO’s comment, they are committed to good stewardship of that valuable asset for the community. We are working with SkySQL to devise an unambiguous and community-focussed trademark arrangement that will allow the Foundation to act in the interests of the whole community. We’re completing a first stage now to secure the Foundation’s name and I will let you know when the collaboration bears fruit.

I hope to meet many MariaDB Community members this weekend at FOSDEM in Brussels, but if you want to discuss anything please get in touch. The coming year is going to be interesting!