Oct 29 2018
Oct 29

Drupal 7.61 - the next planned minor release of Drupal 7 - is scheduled for Wednesday, November 7th, 2018. Minor releases include new features, usability improvements, and backwards-compatible API improvements.

The release will feature full compatibility for PHP 7.2 and so far includes the following changes:

  • File upload validation functions and hook_file_validate() implementations are
    now always passed the correct file URI.
  • The default form cache expiration of 6 hours is now configurable (API
    addition: https://www.drupal.org/node/2857751).
  • Allowed callers of drupal_http_request() to optionally specify an explicit
    Host header.
  • Allowed the + character to appear in usernames.
  • PHP 7.2: Fixed Archive_Tar incompatibility.
  • PHP 7.2: Removed deprecated function each().
  • PHP 7.2: Avoid count() calls on uncountable variables.
  • PHP 7.2: Removed deprecated create_function() call.
  • PHP 7.2: Make sure variables are arrays in theme_links().
  • Fixed theme-settings.php not being loaded on cached forms

At core committer discretion important bug fixes might be added prior to release and this post will be updated to inform of further changes.

Thanks for your patience,

The Drupal 7 core committer team

Dec 01 2015
Dec 01

It's the season again. The season for cookies, candles, christmas trees and ... Drupal 8.

Recently released Drupal 8 features a lot of performance improvements, but as Drupal 8 needs to scale from the smallest blog on shared hosting to the biggest websites in the world (with lots of webservers), it is sometimes difficult to find one-size-fits-all solutions. This little series will feature one little detail about Drupal 8 performance every day, but entries will be deliberately short. Let us start!

Day 1 - Use APCu exclusively for caches

Drupal 8 has a so-called fast-chained backend as the default cache backend, which allows to store data directly on the web server while ensuring it is correctly synchronized across multiple servers. APCu is the user cache portion of APC (Advanced PHP Cache), which has served us well till PHP 5.5 got its own zend opcache. You can think of it as a key-value store that is stored in memory and the basic operations are apc_store($key, $data), apc_fetch($keys) and apc_delete($keys). For windows the equivalent on IIS would be WinCache (http://drupal.org/project/wincache).

However when you are having just one server with Apache, MySQL, PHP you don't need the overhead of the fast_chained backend (which uses the database as consistent backend) and can store everything in APCu directly (provided you have enough memory).

To achieve that do the following steps:

Step 1 - Customize the default backend in settings.php

For this to work you need to add 4 lines to your settings.php:

 $settings['cache']['default'] = 'cache.backend.apcu';
 $settings['cache']['bins']['bootstrap'] = 'cache.backend.apcu';
 $settings['cache']['bins']['config'] = 'cache.backend.apcu';
 $settings['cache']['bins']['discovery'] = 'cache.backend.apcu';

While we can override the default backend easily globally, this won't work for bootstrap, config and discovery, which all use the cache.backend.chainedfast, so we need to override those bins explicitly again in settings.php.

Step 2 - Benchmark

Benchmarks are always tricky as it very much depends on the used scenario. But here is one for page_cache and dynamic_page_cache disabled:

As we can see we save a good 5% and 4% also in function calls. As xhprof is always a little tricky, here is one without:

Before: 0.030 s

After: 0.028 s

which is again around 6%, so the change is worth it!

And that is it for today!

Nov 17 2015
Nov 17

Drupal 8 the newest version of the Drupal content management system is around the corner and will be released Nov 19. PHP7 - the newest generation of the PHP language powering Drupal - will be released later in 2015.

PHP7 features lots of new things, but the most significant part is a 20-50% speed increase, which comes from the completely rebuilt Zend engine VM that powers PHP under-the-hood.

The new code is not only easier to understand, but also cleaner, faster and doing much less memory copying.

However as with all bleeding-edge software there are bugs and sometimes those bugs can be so tricky that during normal development they do not occur. In fact the bugs Drupal 8 found pushed the release date of PHP 7 back - http://news.php.net/php.internals/89102 - but more to that later in this post. So when should you switch?

Drupal 8 already early on had a plan to support PHP7 out of the box with Drupal 8. (https://www.drupal.org/node/2454439).

During that effort by several Drupal 8 contributors, bugs and segfaults have been reported to the PHP team (bugs.php.net) and also incompatibilities have been resolved in the Drupal 8 and Symfony 2 code bases.

For example Null, False, True, which had been classes in Symfony 2, are now reserved keywords. Or Drupal 8 used the String namespace, which also now is reserved. Also Drupal 8 had been relying on uasort behavior being deterministic (which it is not) - though some cases still remain: https://www.drupal.org/node/2466097.

In the end Drupal 8's test suite did almost pass on PHP7, but there were some remaining test failures. Drupal’s tests run on DrupalCI, which is hosted and fully integrated into Drupal.org itself, supporting PHP 5.3-7, MySQL, sqlite and PostgreSQL for both Drupal 7 and Drupal 8 core and contrib modules. As the core maintainers wanted Drupal 8 to ship with full PHP7 support and no known bugs and, because PHP7 stable release was one week before Drupal 8's release now was the last chance to fix that. So the issue became critical again.

With the help of a 10 hour Drupal 8 Accelerate grant (thank you very much Drupal Association), neclimdul, me (Fabianx) and alexpott embarked  on a journey to track down and fix the remaining really really tricky test failures. (https://www.drupal.org/node/2603152, https://bugs.php.net/bug.php?id=70805 and https://bugs.php.net/bug.php?id=70808)

They succeed after close collaboration with the PHP internals team with me (Fabianx) providing the core developers with an EC2 instance, where the bug was easily reproducible and such the PHP internals team finally tracked down the bug in the garbage collector. I reported another bug with a script to reproduce ("array_merge_recursive corrupts memory of unset items") and it was fixed within 3-5 hours after posting the bug report(!).

And such PHP7 became green on Drupal CI on Oct 30, 2015 (https://www.drupal.org/pift-ci-job/73342).

Now with it being green an incentive was started to make PHP7 the default test environment (https://www.drupal.org/node/2607222).

This incentive lead to also testing Postgres and SQLite (and those environments are now available for testing) and while SQLlite passed (yeah), postgres failed with several strange bugs.

I followed up again with the PHP team, created two bug reports (https://bugs.php.net/bug.php?id=70861, https://bugs.php.net/bug.php?id=70862) and a pull request (https://github.com/php/php-src/pull/1619), which solved many of the issues and led to overall more stable PHP7 code base as it was a weird edge case (again!).

One issue is as of this time still remaining with postgres (https://bugs.php.net/bug.php?id=70866), but I am pretty sure that the awesome PHP team will track this one down, too. (Even though Drupal CI passes now, it still fails on my test machine.)

However as we still branch test Postgres with PHP 5.5 daily and that is stable, this will only affect a minority of the users and does not affect our ability to switch to PHP7 for general patch testing (with MySQL 5.5). As of now Drupal 8 has 100% test passes with PHP 7 and MySQL, PostgreSQL and SQlite (https://www.drupal.org/node/3060/qa).

Switching to PHP7 for patch testing will allow to reduce patch test time by around 30% (from 30 min to 21 min) and it will allow us to find potential regressions early. This should reduce the costs of the testing infrastructure to the Drupal Association significantly.

Lessons learned

  • Drupal 8 and PHP7 is an open-source community collaboration success story with close collaboration of PHP with Drupal 8 and Drupal 8 with Symfony.

  • Drupal 8 was able to find many many strange bugs with its very extensive test suite and being a complex application itself. That made PHP7 more stable for everyone, as well as finding Symfony/PHP7 incompatibilities earlier than they might otherwise have been discovered

  • Being able to run Drupal 8 on PHP7 directly and know that it will work is a huge benefit to Drupal 8 itself. In fact I also switched over my own Drupal 8 development environment now to PHP7, which means I can test my code faster.

  • Running and switching your test suite to PHP7 not only will give you confidence that your application is well supported on the newest PHP version, but it will also give you beneficial speed improvements.

  • All of this means that Drupal 8 and PHP 7 will be viable for production more or less as soon as they reach stable releases, so that real sites can take advantage of the ~30% performance improvement relative to PHP 5.6.

Oct 07 2015
Oct 07

TL;DR the command you want is:

    git rebase --onto [the new HEAD base] [the old head base - check git log] [the-branch-to-rebase-from-one-base-to-another]

And my main motivation to putting it here is to easily find it again in the future as I always forget the syntax.

Mental model

To make all of this simpler think of:

You have:

  • Two red dishes on top of two blue dishes
  • One yellow dish

You want:

  • Those two red dishes on top of the one yellow dish

You do:

  • Carefully go with the finger down to the bottom of the two red dishes, which is the first blue dish
  • Take the two red dishes
  • Transfer them over to the one yellow dish

That is what rebase --onto does:

          git rebase --onto [yellow dish] [from: first blue dish] [the two red dishes]

Note: The following is meant for an intermediate audience that is familiar with general rebasing in GIT

Longer explanation

It happened! A branch you had been based your work on has diverged upstream, but you still have work in progress you want to preserve.

So it looks like this for you:

   "Your Commit A" - "Your Commit B" - "Upstream Commit: This is the last upstream commit" - "Upstream Commit: This is another upstream commit"

But now the branch you based your work on squashed their work together or removed a file due to privacy concerns and now upstream has:

  "Upstream Commit: This is the last upstream commit and this is another upstream commit together in one."

So all that you want is to move your two commits onto the new base.

The cherry-pick way

Lets assume your branch is called 'work' and you based your work upon 'upstream/develop'.

What you could do is to checkout the develop branch, reset it to the new version and cherry-pick your two commits:

# Reset the develop branch

git checkout develop

git reset --hard upstream/develop

 

# Checkout a new branch

git checkout -b work2

 

# Now find the two commit IDs

git log work

 

# Cherry-pick those

 

git cherry-pick 28d3fc5

git cherry-pick a67b2c

 

# And reset your work branch to the new work

git checkout work

git reset --hard work2

git branch -d work2

( And yes I am aware that cherry-pick supports commit ranges now, too. )

The rebase way

Usually git knows when you call rebase how to rebase as it has a common history. It gets only problematic if the history diverges as happened above.

In that case you will need to tell git to rebase which on what.

So when you normally use:

git fetch upstream
git rebase upstream/develop

You would now use with the new base:

git fetch upstream
git rebase --onto upstream/develop [old base] work

and thats it.

To get the old base just use the commit ID of the first commit that does not contain changes you made.

A real drupal.org contribution example

This is super helpful when having to juggle several branches for e.g. the Drupal 8 core queue that depend on each other.

Lets say you worked on issue-X and issue-Y and issue-Y is build on top of issue-Y.

Now you work mostly on issue-Y, but now you work more on issue-X and for your personal merit decide to make the history a little nicer, so now issue-X has diverted.

But a simple:

git checkout issue-Y
git rebase --onto issue-X [old base of X in issue-Y] issue-Y

can again save the day. The same is sometimes needed when the branches are based of off different core versions.

May 30 2013
May 30

TL;DR: After an extensive 5-day sprint during DrupalCon, Twig finally has landed in Drupal 8 Core and http://drupal.org/node/1987510 has been committed by the one and only alexpott in the single issues and then pushed as a whole. We still want to get lots done as part of https://drupal.org/node/1757550.

After Dries' post about "Reducing risk in the Drupal 8 release shedule." there was a concern in the Twig community how we can effectlively get the Twig initiative done. Single incremental patches were not possible anymore. So should we revert to the "Blocks" initiative approach and create one really really big mega patch that is hard to review and difficult to patch? Or working again in a sandbox, which in the past had given us too less core coverage and such led to inefficient review.

Focusing

The first thing we did was to focus again on what we need to achieve to be in core and no longer pose a release risk. While our goal is to convert every single theme_ function to a template - or if that is not possible, make it at least possible to implement a template for them (like field.html.twig or views-view-field.html.twig are working already now), it was very inefficient for getting Twig done. This would have meant getting 97 issues RTBC, verified and merged into one big patch, which was a major undertaking, we have been working on for months.

However to pass the "risk-reduction" we only would need to make sure that "every commit puts us closer to release". So we changed strategy and focused only on the tpl.php conversions as Drupal 8 cannot ship with two theme engines. After a 5-day long sprint and lots of profiling (with XHProf-Kit) we finally made it in and are secured in core.

Thank you to all the contributors that helped make this possible!

Next Steps?

We (Jenlampton, John Albin and me) also had an enthusiastic session about Twig, where we outlined some of the next steps. In the twig initiative we kinda all have our personal wishlists and we also have our big meta issue with lots of things to help with.

There is still time to code freeze date, so lets sing the twig song: "One-Seventy-Five-Seventy-Five-Five-Zero - Come review a patch, be a Drupal 8 Hero!". In this case writing patches is also appreciated.

Our wish- and worklists:

If you want to have a great theme system, join us now in our efforts to make the theme system of Drupal 8 consistent, easy to use and flexible!

You can join us in IRC on FreeNode in #drupal-twig channel or jump right into the issue queue!

Thanks a lot!

May 30 2013
May 30

TL;DR: Update php-memcache extension to at least 3.0.8 or downgrade to 2.2.6. This can haunt Drupal sites using features especially and is quite hard to track down and leads to very instable system behavior.

Several Drupal sites of a friend of mine have been plagued by a vague bug, which led to very "interesting" site behavior.

The least concern was that exported indexes suddenly disappeared, but on another site the whole exported rules did suddenly not work anymore and such the whole business logic of the site failed, which was obviously a major problem for my friend. Problematic was also that the problem disappeared then after an apache restart, but happened again months later.

Just clearing the cache solved the problem in both cases, so cache corruption was the most likely.

This bug is described in the PHP bug tracker in the following bug reports:

The former has a good description of the bug report, but the second one uses valgrind to "see" the bug.

Both instances ran php-memcache 3.0.6, but this bug is just fixed in 3.0.8.

So on ubuntu I added the corruption.php to a file:

<?php
$obj = new StdClass();
$obj->obj = $obj;
$memcache = new Memcache;
$memcache->connect('127.0.0.1', 11211);
$memcache->set('x', $obj, false, 300);
$x = $memcache->get('x');
$x = $memcache->get('x');

installed and ran valgrind

apt-get install valgrind
$ valgrind --tool=memcheck php corruption.php

and got several errors. I updated to 3.0.8 manually and got no more errors.

Drupal is affected by this bug in several ways and I found corruption in search API search indexes and rules at least. There might be more.

Unfortunately for Ubuntu no new packages for 3.0.8 do exist, so I upgraded manually by compiling the source to 3.0.8. Hopefully new packages come out soon.

It is fixed for RHEL and CentOS though.

I am blogging about this, because cache corruption can have very problematic side effects for Drupal sites. From failing features (suddenly a feature is overridden), missing functionality (certain rules no longer triggered) or whole pages disappearing, different things can happen. And this bug can be very hard to track down.

So upgrade or downgrade your packages and enjoy stable production systems.

Thanks,

Fabian

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web