tvn
Dec 22 2014
Dec 22

It’s been awhile since our last week notes post, but before a lot of you will go off to celebrate holidays, we wanted to tell you about some of the recent exciting changes on Drupal.org

Responsive Drupal.org

As a New Year’s present to all Drupal.org visitors, today we deployed the initial responsive version of Bluecheese theme. This means that Drupal.org will now look much better on small and large screens. Drupal.org subsites on Drupal 7, such as Api.Drupal.org and Assoc.Drupal.org, will follow as they are tested.

The work on responsifying the theme started at Drupal Dev Days in Szeged earlier this year, with the final testing happening during BadCamp in November.

We’d like to thank our wonderful volunteers for making this happen:
LewisNyman, dasjo, emma.maria, mallezie, Manuel Garcia, sqndr, thamas.

This is the initial deployment, though more page-by-page improvements will be coming. Drupal.org is a big site, with lots of pages that will need specific mobile-first design changes.

Improved user profiles

During the last few months we’ve been slowly migrating user profile fields to Field API fields, which gives us more flexibility to improve profile pages. While migrating the fields, we started to rearrange layout of user profiles. A lot more work will happen on layouts now that field migration is complete, but even now you can see that profiles look much better.

One particular change which happened during fields migration is the ‘My mentors’ field layout. Previous it was just a list of blue links, but now we actually show user pictures of mentors.

New database servers

Drupal.org’s database servers were due for a hardware refresh. New hardware was purchased, provisioned and deployed for Drupal.org that has improved page response times across the site. The new hardware brings solid state drives, and quadruples the amount of processors and memory.

---
As always, we’d like to say thanks to all volunteers who are working with us and to the Drupal Association Supporters, who made it possible for us to work on these projects.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

and

Dec 22 2014
Dec 22

OS X 10.10 Yosemite Local Development Environment: Apache, PHP, and MySQL with Homebrew

OS X 10.10 Yosemite comes with Apache and PHP pre-installed, but it's not in a great configuration, requires root to make lots of changes, and can introduce issues with file ownership and permissions. We prefer to use Homebrew to avoid these problems and because it's easier to keep up to date with newer versions of each component and extend customization. We can also set things up to be fully automatic so you can create new websites locally without needing to edit any configuration files.

With the arrival of Yosemite, some of the changes previously used in 10.9 for setting up Apache, PHP, and MySQL with Homebrew don't work quite the same. This guide will walk you through using Homebrew to install Apache, PHP, and MySQL for a "MAMP" development environment. We'll also use DNSMasq and Apache's VirtualDocumentRoot to set up "auto-VirtualHosts" so you don't need to edit configuration files when starting new projects. Finally, we'll add a firewall rule to allow the default http port 80 to be used without running Apache as root.

The following steps are intended for use on a Yosemite system without any previous attempts to use Homebrew for Apache, PHP, or MySQL. If you have attempted to install a similar stack and run into conflicts, or you've upgraded your operating system from 10.9 and things broke, the final section has some troubleshooting pointers. If that fails, leave a comment and I'll try my best to help you out.

At the conclusion of this guide, you'll be able to create a directory like ~/Sites/project and access it immediately at http://project.dev without editing your /etc/hosts file or editing any Apache configuration. We'll configure PHP and MySQL to allow for enough flexibility for development.

Because some of the commands span several lines, each command will be in a separate code block. This means you should copy and paste each code block in its entirety as a single command.

Before diving in, yes, this is a lot of steps. You can do it faster and pay money for something like MAMP Pro, but this is more fun, and you may learn something along the way! And, while you can simplify things with Vagrant or other virtual machine format, some people prefer to run things on "bare metal" and not have the overhead of a virtual machine.

If you've not already installed Homebrew, you can follow the instructions at the bottom of http://brew.sh. I used to include the command in previous walkthroughs, but it could change after posting, so definitely always check their website to install it properly.

If you do not have git available on your system, either from Homebrew, Xcode, or another source, you can install it with Homebrew now (if you already have it installed, feel free to skip this step to keep the version of git you already have):

brew install -v git

In previous guides on 10.9 and earlier, I added a change to $PATH in ~/.bash_profile to ensure that Homebrew-installed applications would run by default over similar ones that were already installed on OS X. Thankfully, Yosemite's $PATH order is different than earlier OS versions and now includes the default Homebrew location of /usr/local/bin in front. If you installed Homebrew to a custom location, or are not seeing /usr/local/bin at the beginning of your shell's $PATH, check out the file /etc/paths or the directory /etc/paths.d/.

Install MySQL with Homebrew:

brew install -v mysql

Copy the default my-default.cnf file to the MySQL Homebrew Cellar directory where it will be loaded on application start:

cp -v $(brew --prefix mysql)/support-files/my-default.cnf $(brew --prefix)/etc/my.cnf

This will configure MySQL to allow for the maximum packet size, only appropriate for a local or development server. Also, we'll keep each InnoDB table in separate files to keep ibdataN-type file sizes low and make file-based backups, like Time Machine, easier to manage multiple small files instead of a few large InnoDB data files. This is the first of many multi-line single commands. The following is a single, multi-line command; copy and paste the entire block at once:

cat >> $(brew --prefix)/etc/my.cnf 

Uncomment the sample option for innodb_buffer_pool_size to improve performance:

sed -i '' 's/^#[[:space:]]*\(innodb_buffer_pool_size\)/\1/' $(brew --prefix)/etc/my.cnf

Now we need to start MySQL using OS X's launchd, and we'll set it to start when you login. First, create the ~/Library/LaunchAgents folder if it doesn't already exist, copy the startup plist into it, and then start MySQL:

[[ ! -d ~/Library/LaunchAgents ]] && mkdir -v ~/Library/LaunchAgents
ln -sfv $(brew --prefix mysql)/homebrew.mxcl.mysql.plist ~/Library/LaunchAgents/
launchctl load -Fw ~/Library/LaunchAgents/homebrew.mxcl.mysql.plist

By default, MySQL's root user has an empty password from any connection. You are advised to run mysql_secure_installation and at least set a password for the root user:

$(brew --prefix mysql)/bin/mysql_secure_installation

Start by stopping the built-in Apache, if it's running, and prevent it from starting on boot. This is one of very few times you'll need to use sudo:

sudo launchctl unload /System/Library/LaunchDaemons/org.apache.httpd.plist 2>/dev/null

The formula for building Apache is not in the default Homebrew repository that you get by installing Homebrew. While we can use the format of brew install external-repo/formula, if an external formula relies on another external formula, you have to use the brew tap command first. I know, it's weird. So, we need to tap homebrew-dupes because "homebrew-apache/httpd22" relies on "homebrew-dupes/zlib". Whew:

brew tap homebrew/dupes

A slight deviation from my prior walkthroughs: we'll install Apache 2.2 with the event MPM and set up PHP-FPM instead of mod_php. If those terms mean anything to you and you're curious as to why I decided to go this route; it's because: 1) switching PHP versions is far easier with PHP-FPM and the default 9000 port instead of also editing the Apache configuration to switch the mod_php module location, and 2) if we're therefore not using mod_php, we don't have to use the prefork MPM and can get better performance with event or worker. As to why I'm using 2.2 instead of 2.4, popular FOSS projects like Drupal and WordPress still ship with 2.2-style .htaccess files. Using 2.4 sometimes means you have to set up "compat" modules, and that's above the requirement for a local environment, in my opinion.

Onward! Let's install Apache 2.2 with the event MPM, and we'll use Homebrew's OpenSSL library since it's more up-to-date than OS X's:

brew install -v homebrew/apache/httpd22 --with-brewed-openssl --with-mpm-event

In order to get Apache and PHP to communicate via PHP-FPM, we'll install the mod_fastcgi module:

brew install -v homebrew/apache/mod_fastcgi --with-brewed-httpd22

To prevent any potential problems with previous mod_fastcgi setups, let's remove all references to the mod_fastcgi module (we'll re-add the new version later):

sed -i '' '/fastcgi_module/d' $(brew --prefix)/etc/apache2/2.2/httpd.conf

Add the logic for Apache to send PHP to PHP-FPM with mod_fastcgi, and reference that we'll want to use the file ~/Sites/httpd-vhosts.conf to configure our VirtualHosts. The parenthesis are used to run the command in a subprocess, so that the exported variables don't persist in your terminal session afterwards. Also, you'll see export USERHOME a few times in this guide; I look up the full path for your user home directory from the operating system wherever a full path is needed in a configuration file and "~" or a literal "$HOME" would not work. This is all one command, so copy and paste the entire code block at once:

(export USERHOME=$(dscl . -read /Users/`whoami` NFSHomeDirectory | awk -F"\: " '{print $2}') ; export MODFASTCGIPREFIX=$(brew --prefix mod_fastcgi) ; cat >> $(brew --prefix)/etc/apache2/2.2/httpd.conf 
  FastCgiConfig -maxClassProcesses 1 -idle-timeout 1500
 
  # Prevent accessing FastCGI alias paths directly
  
    Order Deny,Allow
    Deny from All
    Allow from env=REDIRECT_STATUS
  
 
  FastCgiExternalServer /php-fpm -host 127.0.0.1:9000 -pass-header Authorization -idle-timeout 1500
  ScriptAlias /fastcgiphp /php-fpm
  Action php-fastcgi /fastcgiphp
 
  # Send PHP extensions to PHP-FPM
  AddHandler php-fastcgi .php
 
  # PHP options
  AddType text/html .php
  DirectoryIndex index.php index.html

 
# Include our VirtualHosts
Include ${USERHOME}/Sites/httpd-vhosts.conf
EOF
)

We'll be using the file ~/Sites/httpd-vhosts.conf to configure our VirtualHosts, but the ~/Sites folder doesn't exist by default in newer versions of OS X. We'll also create folders for logs and SSL files:

mkdir -pv ~/Sites/{logs,ssl}

Let's populate the ~/Sites/httpd-vhosts.conf file. The biggest difference from my previous guides are that you'll see the port numbers are 8080/8443 instead of 80/443. OS X 10.9 and earlier had the ipfw firewall which allowed for port redirecting, so we would send port 80 traffic "directly" to our Apache. But ipfw is now removed and replaced by pf which "forwards" traffic to another port. We'll get to that later, but know that "8080" and "8443" are not typos but are acceptable because of later port forwarding. Also, I've now added a basic SSL configuration (though you'll need to acknowledge warnings in your browser about self-signed certificates):

touch ~/Sites/httpd-vhosts.conf
(export USERHOME=$(dscl . -read /Users/`whoami` NFSHomeDirectory | awk -F"\: " '{print $2}') ; cat > ~/Sites/httpd-vhosts.conf 
    Options Indexes FollowSymLinks MultiViews
    AllowOverride All
    Order allow,deny
    Allow from all

 
# For http://localhost in the users' Sites folder

    ServerName localhost
    DocumentRoot "${USERHOME}/Sites"

    ServerName localhost
    Include "${USERHOME}/Sites/ssl/ssl-shared-cert.inc"
    DocumentRoot "${USERHOME}/Sites"

 
#
# VirtualHosts
#
 
## Manual VirtualHost template for HTTP and HTTPS
#
#  ServerName project.dev
#  CustomLog "${USERHOME}/Sites/logs/project.dev-access_log" combined
#  ErrorLog "${USERHOME}/Sites/logs/project.dev-error_log"
#  DocumentRoot "${USERHOME}/Sites/project.dev"
#
#
#  ServerName project.dev
#  Include "${USERHOME}/Sites/ssl/ssl-shared-cert.inc"
#  CustomLog "${USERHOME}/Sites/logs/project.dev-access_log" combined
#  ErrorLog "${USERHOME}/Sites/logs/project.dev-error_log"
#  DocumentRoot "${USERHOME}/Sites/project.dev"
#
 
#
# Automatic VirtualHosts
#
# A directory at ${USERHOME}/Sites/webroot can be accessed at http://webroot.dev
# In Drupal, uncomment the line with: RewriteBase /
#
 
# This log format will display the per-virtual-host as the first field followed by a typical log line
LogFormat "%V %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combinedmassvhost
 
# Auto-VirtualHosts with .dev

  ServerName dev
  ServerAlias *.dev
 
  CustomLog "${USERHOME}/Sites/logs/dev-access_log" combinedmassvhost
  ErrorLog "${USERHOME}/Sites/logs/dev-error_log"
 
  VirtualDocumentRoot ${USERHOME}/Sites/%-2+

  ServerName dev
  ServerAlias *.dev
  Include "${USERHOME}/Sites/ssl/ssl-shared-cert.inc"
 
  CustomLog "${USERHOME}/Sites/logs/dev-access_log" combinedmassvhost
  ErrorLog "${USERHOME}/Sites/logs/dev-error_log"
 
  VirtualDocumentRoot ${USERHOME}/Sites/%-2+

EOF
)

You may have noticed that ~/Sites/ssl/ssl-shared-cert.inc is included multiple times; create that file and the SSL files it needs:

(export USERHOME=$(dscl . -read /Users/`whoami` NFSHomeDirectory | awk -F"\: " '{print $2}') ; cat > ~/Sites/ssl/ssl-shared-cert.inc 
)
openssl req \
  -new \
  -newkey rsa:2048 \
  -days 3650 \
  -nodes \
  -x509 \
  -subj "/C=US/ST=State/L=City/O=Organization/OU=$(whoami)/CN=*.dev" \
  -keyout ~/Sites/ssl/private.key \
  -out ~/Sites/ssl/selfsigned.crt

Start Apache

Start Homebrew's Apache and set to start on login:

ln -sfv $(brew --prefix httpd22)/homebrew.mxcl.httpd22.plist ~/Library/LaunchAgents
launchctl load -Fw ~/Library/LaunchAgents/homebrew.mxcl.httpd22.plist

Run with port 80

You may notice that httpd.conf is running Apache on ports 8080 and 8443. Manually adding ":8080" each time you're referencing your dev sites is no fun, but running Apache on port 80 requires root. The next two commands will create and load a firewall rule to forward port 80 requests to 8080, and port 443 requests to 8443. The end result is that we don't need to add the port number when visiting a project dev site, like "http://projectname.dev/" instead of "http://projectname.dev:8080/".

The following command will create the file /Library/LaunchDaemons/co.echo.httpdfwd.plist as root, and owned by root, since it needs elevated privileges:

sudo bash -c 'export TAB=$'"'"'\t'"'"'
cat > /Library/LaunchDaemons/co.echo.httpdfwd.plist 


${TAB}Label
${TAB}co.echo.httpdfwd
${TAB}ProgramArguments
${TAB}
${TAB}${TAB}sh
${TAB}${TAB}-c
${TAB}${TAB}echo "rdr pass proto tcp from any to any port {80,8080} -> 127.0.0.1 port 8080" | pfctl -a "com.apple/260.HttpFwdFirewall" -Ef - && echo "rdr pass proto tcp from any to any port {443,8443} -> 127.0.0.1 port 8443" | pfctl -a "com.apple/261.HttpFwdFirewall" -Ef - && sysctl -w net.inet.ip.forwarding=1
${TAB}
${TAB}RunAtLoad
${TAB}
${TAB}UserName
${TAB}root
EOF'

This file will be loaded on login and set up the 80->8080 and 443->8443 port forwards, but we can load it manually now so we don't need to log out and back in:

sudo launchctl load -Fw /Library/LaunchDaemons/co.echo.httpdfwd.plist

The following is for the latest release of PHP, version 5.6. If you'd like to use 5.3, 5.4 or 5.5, simply change the "5.6" and "php56" values below appropriately.

Install PHP and PHP-FPM. Excluding snmp is a workaround for a problem specific to Yosemite:

brew install -v homebrew/php/php56 --with-fpm --without-snmp

Set timezone and change other PHP settings (sudo is needed here to get the current timezone on OS X) to be more developer-friendly, and add a PHP error log (without this, you may get Internal Server Errors if PHP has errors to write and no logs to write to):

(export USERHOME=$(dscl . -read /Users/`whoami` NFSHomeDirectory | awk -F"\: " '{print $2}') ; sed -i '-default' -e 's|^;\(date\.timezone[[:space:]]*=\).*|\1 \"'$(sudo systemsetup -gettimezone|awk -F"\: " '{print $2}')'\"|; s|^\(memory_limit[[:space:]]*=\).*|\1 512M|; s|^\(post_max_size[[:space:]]*=\).*|\1 200M|; s|^\(upload_max_filesize[[:space:]]*=\).*|\1 100M|; s|^\(default_socket_timeout[[:space:]]*=\).*|\1 600|; s|^\(max_execution_time[[:space:]]*=\).*|\1 300|; s|^\(max_input_time[[:space:]]*=\).*|\1 600|; $a\'$'\n''\'$'\n''; PHP Error log\'$'\n''error_log = '$USERHOME'/Sites/logs/php-error_log'$'\n' $(brew --prefix)/etc/php/5.6/php.ini)

Fix a pear and pecl permissions problem:

chmod -R ug+w $(brew --prefix php56)/lib/php

Until this pull request is merged, set the location of php_ini in pear to $(brew --prefix)/etc/php/5.6/pear.conf:

pear config-set php_ini $(brew --prefix)/etc/php/5.6/php.ini system

The included Opcache extension will speed up your PHP environment dramatically, and it's already installed, but needs to be enabled. We'll also bump up the opcache memory limit:

sed -i '' "s|^\(\[opcache\]\)$|\1"\\$'\n'"; Load the opcache extension"\\$'\n'"zend_extension=opcache.so"\\$'\n'"|; s|^;\(opcache\.enable[[:space:]]*=[[:space:]]*\)0|\11|; s|^;\(opcache\.memory_consumption[[:space:]]*=[[:space:]]*\)[0-9]*|\1256|;" $(brew --prefix)/etc/php/5.6/php.ini

If you're not using PHP 5.5 or 5.6 and wish to add the opcache extension: See https://gist.github.com/alanthing/7acb05bf3e3dfc972b86.

Finally, let's put the PHP-FPM launchd plist file in place and start PHP-FPM now:

ln -sfv $(brew --prefix php56)/*.plist ~/Library/LaunchAgents
launchctl load -Fw ~/Library/LaunchAgents/homebrew.mxcl.php56.plist

Optional: At this point, if you want to switch between PHP versions, you'd want to stop PHP-FPM by unloading the plist above, run brew unlink php56, repeat all of the above PHP steps for php55/php54/php53, and loading the new plist. No need to touch the Apache configuration at all!

A difference now between what I've shown before, is that we don't have to run on port 53 or run dnsmasq as root. The end result here is that any DNS request ending in .dev reply with the IP address 127.0.0.1:

brew install -v dnsmasq
echo 'address=/.dev/127.0.0.1' > $(brew --prefix)/etc/dnsmasq.conf
echo 'listen-address=127.0.0.1' >> $(brew --prefix)/etc/dnsmasq.conf
echo 'port=35353' >> $(brew --prefix)/etc/dnsmasq.conf

Similar to how we run Apache and PHP-FPM, we'll symlink a launchd plist to ~/Library/LaunchAgents and start:

ln -sfv $(brew --prefix dnsmasq)/homebrew.mxcl.dnsmasq.plist ~/Library/LaunchAgents
launchctl load -Fw ~/Library/LaunchAgents/homebrew.mxcl.dnsmasq.plist

With DNSMasq running, configure OS X to use your local host for DNS queries ending in .dev:

sudo mkdir -v /etc/resolver 
sudo bash -c 'echo "nameserver 127.0.0.1" > /etc/resolver/dev'
sudo bash -c 'echo "port 35353" >> /etc/resolver/dev'

To test, the command ping -c 3 fakedomainthatisntreal.dev should return results from 127.0.0.1. If it doesn't work right away, try turning WiFi off and on (or unplug/plug your ethernet cable), or reboot your system.

We set up Apache to run on boot on ports 8080 and 8443 with auto-VirtualHosts for directories in the ~/Sites folder and PHP-FPM via mod_fastcgi. The OS X firewall will forward all port 80 traffic to port 8080 and port 443 to port 8443, so we don't have specify the port number when visiting web pages in local web browsers or run Apache as root. MySQL is installed and set to run on boot as well. DNSMasq and some OS X configuration is used to direct any hostname ending in .dev to the local system to work in conjunction with Apache's auto-VirtualHosts.

What do I do now?

You shouldn't need to edit the Apache configuration or edit /etc/hosts for new local development sites. Simply create a directory in ~/Sites and then reference "http://" + that foldername + ".dev/" in your browser to access it.

For example, download Drupal 7 to the directory ~/Sites/firstproject, and it can then be accessed at http://firstproject.dev/ without any additional configuration. A caveat - you will need to uncomment the line in Drupal's .htaccess containing "RewriteBase /" to work with the auto-VirtualHosts configuration.

What if this "auto-VirtualHost" doesn't work for [other project]?

If you need to create a manual VirtualHost in Apache because the auto-VirtualHost does not work for your configuration, you will need to declare it before the auto-VirtualHosts if you're also going to use .dev as the TLD. Otherwise the auto-VirtualHost block will be accessed first. I have a commented-out sample for a manual VirtualHost entry in "~/Sites/httpd-vhosts.conf" you may use.

The commands above can be run on a fresh Yosemite system without issue as I've tested with fresh installs in VMware. Nearly every problem I've heard about has been related to upgrading from 10.9 or earlier, or switching to this style of setup from another or similar setup. The easiest thing to do would be to brew uninstall each component referenced above, delete all related config files in $(brew --prefix)/etc and ~/Library/LaunchAgents, and start over. If that's a bit heavy-handed, check each of the configuration files edited in this guide and look for duplicate entries or typos.

You can also check log files for error output:

  • Apache: $(brew --prefix)/var/log/apache2/error_log, or run httpd -DFOREGROUND and look for output
  • PHP-FPM: $(brew --prefix)/var/log/php-fpm.log
  • MySQL: $(brew --prefix)/var/mysql/$(hostname).err
  • DNSMasq: no log file, run dnsmasq --keep-in-foreground and look for output

Leave a comment if you have any questions or problems!

Dec 22 2014
Dec 22

Reroute Email module uses hook_mail_alter() to prevent emails from being sent to users from non-production sites. It allows you to enter one or more email addresses that will receive the emails instead of delivering them to the original user.

This is useful in case where you do not want email sent from a Drupal site to reach the users. For example, if you copy a live site to a test site for the purpose of development, and you do not want any email sent to real users of the original site. Or you want to check the emails sent for uniform formatting, footers, ...etc.

As we don't need the module configured on production (we don't need to reroute any emails there), it's best to do this in code using settings.local.php (if you have one) or the standard settings.php file.

The first thing that we need to do is to enable rerouting. Without doing this, nothing will happen.

$conf['reroute_email_enable'] = TRUE;

The next option is to whether to show rerouting description in mail body. I usually have this enabled. Set this to TRUE or FALSE depending on your preference.

$conf['reroute_email_enable_message'] = TRUE;

The last setting is the email address to use. If you're entering a single address, you can add it as a simple string.

$conf['reroute_email_address'] = 'person1@example.com';

In this example, all emails from the site will be rerouted to person1@example.com.

If you want to add multiple addresses, these should be added in a semicolon-delimited list. Whilst you could add these also as a string, I prefer to use an array of addresses and the implode() function.

$conf['reroute_email_address'] = implode(';', array(
  'person1@example.com',
  'person2@example.com',
  'person3@example.com',
));

In this example, person2@example.com and person3@example.com would receive their emails from the site as normal. Any emails to addresses not in the array would continue to be redirected to person1@example.com.

Dec 22 2014
Dec 22

Whew! 2014 is almost in the books. We wanted to take a moment to say thank you to everyone who attended live or watched a recording of one of our webinars, and provide a recap of all we learned throughout the year. We held 12 Webinars last year on a wide range of topics. We introduced you to new technology partners and showcased beautiful and innovative Drupal Commerce sites. We discussed the upcoming transition to PCI version 3.0 and talked about a new breed of eCommerce sites, focused on the merging of Content & Commerce.

We also kicked off a new type of webinar series, focused on the unique technical aspects of Drupal Commerce. We held 6 Drupal Commerce Tech Talks and covered features and functionality to enhance and redefine how eCommerce is done with Drupal.

And we launched a new cloud development and hosting solution, Platform.sh, and managed to do several live demos (including ones in German and French!) All in all, it was a hugely successful year for our webinar series...and we’re looking forward to continuing that trend in 2015.

We invite you to browse our entire archive, recapped here for you in the form of little gifts that you can give yourself at your own pace. We hope you will find it to be a valuable resource to you in your continued work with Drupal Commerce.

Happy Unwrapping

Dec 22 2014
Dec 22

What a year.

I suppose there is a temptation to say that toward the end of every year but really, WHAT A YEAR FOR DRUPAL.

Here are six things we learned in 2014:

1. There is no challenge too large or complex for Drupal

If there were any remaining questions about Drupal’s ability to scale, they should be more than answered with Weather.com’s migration to Drupal. The site is one of the largest in the world, serves more than 100 million unique visitors per month, and contains more than 20 million pages of content. There were many other big wins for Drupal in 2014, including becoming the CMS of choice for the Australian federal government and NASA.

2. Drupal is still very popular among beginners and hobbyists

Drupal may be a top solution for large, complex sites, but it is also still popular among beginners and hobbyists. In 2014, the Drupal Association-backed Global Training Days program saw significant growth in participation around the world. Thirty-five countries hosted 170 free or low-cost Drupal training courses designed as an introduction for beginners and hobbyists. Additionally, about a quarter of DrupalCon attendees in 2014 identified themselves as “beginners.” Why is this important? See the next item.

3. Learning Drupal is a smart career move

It’s no secret that Drupal skills are in high demand and have been for some time. In fact, 82% of employers we surveyed during the summer indicated they plan to hire Drupal talent within the next 6 months. Forty percent of hiring managers say they are in “constant” hiring mode for Drupal talent. Few career paths experience that kind of demand.

4. Drupal is free, but it’s also very valuable

If you want to get a feel for the value of technology markets or ecosystems, pay attention to where investors are placing bets. In 2014, big bets were made in the Drupal ecosystem. Within the space of a month, Acquia raised $50 million and Pantheon raised more than $21 million. That kind of confidence by investors requires strong businesses built on mature technologies like Drupal.

5. Signs point to broad adoption of Drupal 8

In our recent Drupal community survey we asked respondents whether they had plans to adopt Drupal 8. More than 80% of Drupal service providers and organizations using Drupal said Drupal 8 is firmly planted on their roadmap. Another 10% said they had plans to evaluate it (look for all results from the survey in the next few weeks). Drupal 8 sessions at 2014 DrupalCon events were among the most widely attended talks. Plus, the momentum toward a final release is building with Drupal 8 now in beta and efforts like the Drupal Association’s “Drupal 8 Accelerate” funding program. Drupal 7 catapulted Drupal into the mainstream. Where will Drupal 8 take the project?

6. The project and the community are strong

The year saw the two highest-attended DrupalCon events ever in Austin and Amsterdam. The number of contributors has topped 2,400 and it's still growing. And here is an interesting number compiled by the Drupal Association engineering team: more than 18 thousand unique people had 10 or more contributions on Drupal.org and its subsites in 2014. “Contributions” in this case could include anything from commenting on an issue to providing a translation, to writing a module. The project also weathered a severe security vulnerability under the leadership of the Drupal Security Team. The year showed just how durable and stable the project and community are.

There were many other triumphs during the year -- too many to list. But all taken together, they point to an exciting 2015 and beyond. I don’t know about you but I’m very excited for the future.

Dec 22 2014
Dec 22

The older I get, the quicker the years seem to fly by. As I begin to reflect on a great 2014, one thing is crystal clear again. People are the most important thing to any organization. Having a great team is more important than having a great idea. A good team will figure out how to make something great happen; they'll pivot, evolve and claw their way to success. I see it every day at Acquia, the Drupal Association or the Drupal community. I'm fortunate to be surrounded by so many great people.

By extension, recruiting is serious business. How do you figure out if someone is a great fit for your organization? Books have been written about finding and attracting the right people, but for me the following quote from Dee Hock, the founder of Visa, sums it up perfectly.

"Hire and promote first on the basis of integrity; second, motivation; third, capacity; fourth, understanding; fifth, knowledge; and last and least, experience. Without integrity, motivation is dangerous; without motivation, capacity is impotent; without capacity, understanding is limited; without understanding, knowledge is meaningless; without knowledge, experience is blind." — Dee Hock, founder of Visa.

Most hiring managers get it wrong and focus primarily on experience. While experience can be important, attitude is much more important. Attitude, not experience, is what creates a strong positive culture and what turns users and customers into raving fans.

jam
Dec 22 2014
Dec 22

Part 2 – Larry Garfield and I had a long chat in front of my camera at DrupalCon Amsterdam to warm him up for writing "Building Bridges: Linking Islands" in the Future of PHP guest blog series on Acquia.com. In this second part of our conversation, we touch on Drupal's specialist value-adds over and above straight PHP, what defines community, sustainable contribution and services v products businesses, rebuilding Drupal's foundations to make a better project for everyone, the php[world] conference and Drupal 8 itself as manifestations of all the good changes coming with PHP interoperability, how communities are building bridges between their islands and sharing innovation, and how to do the Drupal Hug™.

In part one, we covered Larry's start in Drupal, some project history, what Drupal can teach (and needs to learn) about contribution and community ("celebrating the newbie" and the power of "thank you"), The New PHP, fuzzy project boundaries and inter-project collaboration.

Community beyond the code

In the past, I think a lot of us in the community thought "Drupal is this code and we happen to be working on together so I'll hang out with these people." When the codebase (What is Drupal? What is not Drupal?) becomes more amorphous, how do we maintain a positive community identity?

Larry explains, "At some point our mission statement shifted from 'build a product' to 'be a community'. That means in some ways, we've lost focus on the user; lost focus on what we are trying to accomplish. I would argue growing the community for the sake of growing the community is the wrong approach. If our goal is to build a world-class content management system (or platform), then there are some places where community is not necessarily the answer. It's a part of it. We need to have some hard conversations around [whether there are] parts of building Drupal, the platform, that are better handled in a non-grassroots-community fashion. Dries talked about some of this in is [Amsterdam] keynote; we've got 2000+ contributors to Drupal 8, but still 80-90% of the work is done by 200 people. Dries talked about how we need steady resourcing and how we can incentivize companies to pay people to work on core in large chunks. That does change the dynamic. It might be, if we make that change, Drupal 9 – years down the line – has fewer contributors than Drupal 8. We should not automatically think that that's a negative. It's not a bad thing if there's still a healthy community and a successful platform as a result.

Building the better Drupal

He continues, mentioning something I say a lot (and I wonder if it was me who said this to Larry in Denver :-) ... "One of the things that makes Drupal so powerful and so special is that we can disagree, we can argue, we can fight, we can have different views, but at the end of the day, you know the people you're dealing with when working on Drupal, their goal is to make life easier for somebody else. They may disagree on how to do that, on what that means, but at the end of the day, our goal as the Drupal community is to make life easier for someone else ... using Drupal. That's something we should make sure we never lose."

But to get to the best experience for everyone from developers to end users, Larry points out the need for strong fundamentals: "If you try to build on something that is complex, you inherit all of that complexity. If you have a weak foundation and you build on top of it, there's a limit to what you can do. If you fix that foundation, if you make that lower level simpler and less coupled and higher quality, that ripples out to everything you build on top of it."

"The analogy that I used for Drupal 7 to Drupal 8 back when we first started was that our house is getting old. We've got some nice furniture, but the foundation is cracking, so let's pick a foundation off the shelf (Symfony components), build a new house on top of that. That house can be much taller, have much more modern electrical wiring ... and then move our furniture over. To do that, you need to have a better foundation and that ripples out through the whole system."

"Making sure that core is solid. Making sure that core is architecturally clean and well-maintained has a ripple effect. If core gets crappy, then modules, no matter how good they are, are hamstrung. And that means site builders are hamstrung. And that means end users are hamstrung. If core is rock solid and unit testable and clean and fast and powerful, that means modules could be solid and clean and powerful (or crappy) and then you can let evolution take its course there. Then that means site builders can build something that's clean, on clean modules, on clean core and that's how you get the really powerful end user experience. You need to have that whole pipeline. You need to be thinking in terms of how we enable the next layer up, rather than focusing on the thing at the end."

Links/References

Interview video

[embedded content]

Image credit

linking_islands_1.jpg - Image by Jay-P - https://www.flickr.com/photos/esqenzo/248879957/ License https://creativecommons.org/licenses/by-nd/2.0/

Dec 22 2014
Dec 22

"I think our site’s been hacked."

These six scary words strike fear in the hearts of most developers. Even Drupal devs hear it now and then, though the security measures built into the codebase make it extremely robust. For attacks that target Drupal, exploiting known security holes are the easiest to implement on a large scale. This is why it’s important to keep your code up-to-date, and to host your Drupal sites with a company that specializes in Drupal, as well as to have a process in place so you can quickly update your sites should an exploit be discovered.

However, even the most diligent Drupal developers could be forgiven for allowing their site to fall prey to the recent (mid-October 2014) database exploit ‘SA-CORE-2014-005 - Drupal core - SQL injection’, better known as Drupageddon. Automated scripts were widely hacking sites a mere seven hours after the exploit was announced. There were also disturbing details after the announcement - some people were logging into their sites to find them already upgraded. It turns out that some of the hackers were upgrading the sites to keep additional hackers out!

The good news for ThinkShout was the fact that we host most of our sites on Pantheon, a company that hosts ‘10s of 1000s of Drupal sites’, and thus had advance notice of the security announcement. Since security is an integral part of our Support & Success practices, we were able to patch all of our client sites while Pantheon was monitoring and blocking the ongoing attacks.

About a month later though, we were contacted by a former client whom we had not heard from in a few years. They were concerned about the vulnerability of their Drupal installation, which had not been upgraded or patched since we installed it in late 2012. It was not on Pantheon, but rather a generic shared hosting site, and they were very budget-constrained.

We were given access to a cPanel login and asked to patch, review and (if needed), repair the site, or provide a detailed repair estimate if there was going to be substantial cost. At this point, it’s worth reviewing the official Drupal checklist for repairing a hacked site (paraphrased for brevity):

  1. Take the website offline

  2. Notify the server’s administrator (so they can look for backdoors)

  3. Remove all the website’s files and database from the server

  4. Restore all files and database from backups from before 15 October 2014

  5. Update or patch the restored Drupal core code

  6. Put the restored and patched/updated website back online

  7. Manually redo all site changes since the date of the backup

  8. Audit anything merged from the compromised website, such as custom code, configuration, files or other artifacts, to confirm they are correct and have not been tampered with.

Logging into cPanel for the affected client site, we ran into problem #1: no backups! Even if there had been a backup system in place, it would have been close, since most hosts do not keep backups older than 30 days.

What about version control? If I could get access to a Git repo, we should be able to recreate the file system locally from a master branch, and use that to compare against the live site. After setting up SSH keys on the server and importing to a local machine, we were able to log into the shared hosting server. Alas, no version control. Worse, the file permissions in the web root meant we couldn’t install Git.

At this point, it felt like we were running out of options, so we took a step back and reviewed our goals for the site:

  1. Upgrade Drupal core.

  2. Check if the site had, in fact, been hacked.

  3. If hacked, determine the scope and remediation steps.

Working from here, we went back into cPanel and created a backup of the site and its db. We then downloaded them and installed them on a local development machine. Drupal core had not been updated, so it had not been subject to one of the more sophisticated ‘upgrade’ hacks. We updated core, verified that the site still performed as expected, then did the same for the live site.

We then ran the Drush drupalgeddon-test on our local machine to determine if any obvious hacks had taken place. Success! It reported that it had been hacked, due to the presence of two suspicious users. Logging into the site showed a user with no email, and a test@test.com account that had last logged in two weeks ago. We were lucky here - the site only had 10 registered users, so it was easy to spot outliers.

At this point, we began a timestamp-based file-by-file check to ensure that no scripts had been inserted. This showed that only files in /sites/default/files had been altered. A manual review of these files showed only the usual stuff: images, videos, and PDFs. We then reviewed the site db, in particular, the menu_router table, looking for attempts to insert code or write to the server - these were also negative. Based on this, and the hosting site’s good practices regarding file permissions (the thing that kept us from installing Git), we felt confident that there was no malicious scripting inserted in the site’s codebase.

At this point, the scope became fairly clear - the site had been ‘bot hacked’ once or twice, based on there being two suspicious users. At some point after the hack, someone, perhaps another bot, logged in using one of the fake accounts and had a look around the site, and either found nothing of value, or no easy way of compromising the site further (or some mix of both). Again, the site was lucky. With no e-commerce, minimal users, and a well-secured file system, it was a very low-value site for a hacker.

Remediation was fairy simple - update core, remove the bad users, and advise the client to set up automated backups on their hosting account. We were able to do so in the eight hours allotted, despite the lack of backups of version control, and the client suffered no data loss or downtime. It was a very positive outcome, but the client was very lucky. If they had been a higher-value site with stored sensitive data, they would have been in a world of hurt.

The lesson from all of this is to think proactively - don’t just take the best approach in preventing a hack (updated code, secure hosting), but also have systems and procedures in place (automatic backups, source control) in the event a hack does occur. By doing so, you’ll not only make a hack less likely, you’ll feel slightly less dread if someone utters those six scary words.

Dec 22 2014
Dec 22

As if the holidays are not busy enough, for many of you it’s also crunch time in planning your marketing and digital efforts for 2015.

To ease some of the pressure, we can help get your Drupal website in tip-top shape with a detailed kick-starter package that takes the guesswork out of forming a revenue generating digital strategy. 

Highlights include: 

  • Attracting the Right Prospects to your Drupal Website
  • Improving the ROI of your Drupal Website
  • 4 SEO Tips Your Can Implement Today
  • and more!

Happy Holidays & digital planning!

Drupal Strategy in 2015

Dec 22 2014
Dec 22

Recently, I started building new projects on Drupal 8. Everyday, I spent some time on checking the progress of a few D8 issues that I needed and the overall D8 progress. I thought it might be fun to have some progress numbers to show up on my Moto 360 Watch, and it's almost Christmas, so I decided to build my own Watch Face for Android Wear to monitor Drupal 8 Release.

The version 1.0 release of this watch face looks like the picture below

Development Story

Thanks to Drupal 8 Release Project, the app can easily get a few things down from the Internet:

  • The current number of critical issues
  • The current number of major issues
  • yesterday number of critical issues
  • yesterday number of major issues
  • Overall estimated Drupal 8 release date

The watch face logic and component is straightforward:

  • It has a simple (analog) watch.
  • Every 6 hours (this interval is set by Drupal Release Date Project to crawl Drupal.org), the watch sends the signal to the companion handheld device to download the new data.
  • The handheld device uses Retrofit library (built by Square) to efficiently access Drupal Release Date Project API and parse the JSON data.
  • The handheld device calls the Data Layer API (the actual communication is over Bluetooth), and then to sync the data to Android Watch.
  • The Android Wear app draws data on the watch face.

Try the App
The source code of this project is at ReleaseWatch
The Watch Face App is available on Google Play.

Android app on Google Play

Merry Christmas! and if you have ways to improve or extend the project, please use the comments below to enlighten me!

Dec 22 2014
Dec 22

In our previous blogpost we presented how to install and set up Apache Solr. But what do you do when you need to search in files as well?

For a recent project I had to enable users to search the content of attached files mainly in .pdf format. The Apache Solr with Tika seemed to be a good solution.

This guide is based on Drupal 7 with Search API 7.x-1.1.3, Solr search 7.x-1.6, Search views 7.x-1.13, Search API attachments 7.x-1.4 and Views 7.x-3.8

  1. Install Solr
    If you haven't installed Solr yet, check our blog post how to set up easily a basic Solr service on your *nix system or read the official instructions how you can do it.
  2. Install Drupal modules
    Install and enable the following modules:
  3. Download tika
    Download the Tika app .jar file (tika-app-1.6.jar as per the time of this post), and copy it to sites/all/libraries/tika, or in case you build your site as an install profile, copy it to profiles/{my_profile}/libraries/tika.
    Be sure that you have the java JDK installed. If you use Ubuntu like I do, you can read here the "Installing default JRE/JDK" section for further info.
    Important: Once you have downloaded the .jar file, you may need to adjust its permissions.
  4. Set up Search API
    Once you are done go to admin/config/search/search_api.
    • Add a new server.
    • Add an index to the newly created server.
      • On Filters tab enable File attachments:
    • On Fields tab select the desired fields to be indexed:

    • Open the Search API Attachments tab, select Tika Extraction method and fill in the Tika Extraction Settings section. Save configuration when you are done.

  5. Create a view
    Go to admin/structure/views and add a new view.
    • From Show list select your index created earlier.
    • For display format you might select rendered entity.

    • Click Continue & edit.
    • At Filter Criteria section set up your filter.

    • Check "Expose this filter to visitors" and select the Searched fields from the list.

And basically that's it. Congratulations, you have just set up a full text search for attached files.

Dec 22 2014
Dec 22

Earlier today, Propeople (together with our parent company, Intellecta) made two big announcements that I am really excited about. The first is that Blink Reaction is joining our larger Intellecta family, and will be merging with Propeople - creating the largest professional Drupal services company in the world! The second piece of news is that Propeople is also merging with Bysted, one of Denmark’s foremost creative agencies. Together, these two deals are strategic in starting an exciting new chapter for Propeople.

Intellecta’s purchase of an 80 percent stake in Blink Reaction also means that Blink Reaction will be operating under my direct management. Our companies will initially operate as independent units but will join together as a new company under a unified brand by the deal’s completion on March 31st, 2015. This new agency will have a truly unique international reach - with 350+ employees worldwide, working across 9 countries. The new agency’s global footprint and working capacity will be unmatched in the Drupal space.

It is an honor to lead the start of a whole new chapter for Propeople and Blink Reaction. I hold the company that Nancy Stango (Founder and CEO of Blink Reaction) and her team have built in very high esteem, and can’t wait to see what we’ll be able to achieve together.

Welcoming Blink Reaction into the Intellecta family will greatly expand our technical capacity, specially when it comes to the development of digital solutions built on Drupal. At the same time, bringing Bysted into Propeople will bring our creative, design, and strategy offerings to new levels. Both of these developments will prove to be strategic in charting the future direction in which Propeople is heading.

This future direction is driven by the vision of being a full-service agency for the digital age. Propeople is an agency that has had a truly technical upbringing; this is at the core of our identity, manifests itself through our entire organization’s culture and touches everything that we do.

The majority of the prominent agencies in the larger digital space tend to come from strong creative or communication backgrounds. The fact that Propeople comes from a technology background is a significant distinction in a world where technical matters have become increasingly more important for organizations as they develop their brand’s digital presence. And I’m not just talking about the importance of coding - but the larger way that analytics, data, and integrations with a variety of systems seem to be the common threads running through everything that a company does.

Today marks an important milestone for me, the Propeople team around the world and our customers. I, for one, can’t wait to get started!

Dec 22 2014
Dec 22

Panels standard renderer has very flexible undocumented feature of controlling the sequence of panels being rendered.

By default you have possibility to use 'render first' and 'render last' in your content type definition so you already can control what pane should be rendered for example last. Undocumented part is more interesting that is hook_panels_panes_prepared_alter() that runs after all panes were set up. So you can alter the array of panes in which way you can control sequence of panes being rendered. This feature is super handy when you have dependent panes.

Example can be if you have several panes with list of news articles. Lets say one block displays 3 items and another block displays other 5. But the problem is that editors can place blocks independently and you do not know whether you have two blocks on the page or only one. But you know that block that have 3 items should be rendered first and then block of 5 so first block has more recent news articles. Using properties 'render first' and 'render last' you can do the trick. When you render 3 items block you can save some static variable so when you render 5 items block you can check that variable and if previous block set it to TRUE you need to shift your list to 3 items so you won't duplicate news articles in both lists.

Meanwhile if you have multiple combinations of similar blocks you can use hook_panels_panes_prepared_alter() and control which block renders first and then you will still have very nice lists of articles.

I would like to thank Andrii Tiupa for pointing me to this brilliant feature.

Dec 22 2014
Dec 22

[embedded content]

Previously, I demonstrated how to Valuate Content Using Google Analytics and Drupal. The video today establishes how the Intelligence dashboard can serve as a visual executive summary of your content marketing. Learn how your efforts are valued and how the results are used to increase qualified leads.

Measurements available on the Intelligence dashboard:

Traffic

The traffic measurement is represented on the first of two line charts, in the color blue, versus the objectives in gray, which is what our marketing and sales department sets at the beginning of every month. In the video, you will notice that our Actual Traffic versus our Objective, is slightly higher. Simple reverse math will help you determine the number of conversions you need (for the line chart in the middle) and the amount of traffic you need to set your objectives. Assuming you know the historical percentage traffic volume you need; you will know the number of conversions you want.

The first set of bars below the first line chart, represent top sources of traffic - in this case its direct, organic, social, and referral.

Conversion

The conversion measurement is represented in the middle line chart of the Intelligence dashboard. Note: to know the percentage of conversions you need, will result to the qualified leads you want. 

These bars below the second line chart show the top conversion types - Top of the Funnel (ToFu), Middle of the Funnel (MoFu), Contact Form, and Demo request for everyday of the month. Note: you can label these conversion types however you deem appropriate for your business. 

Content & Engagement

The last chart, represented as a bubble chart, is a snippet of all the efforts towards the content creation and the drive of engagement. Each circle represents the content posted each day. The size of of each circle is determined by the value each piece of content is producing, i.e.: views and reach. 

These bars beneath the circle, represent the the number of social shares and comments per post, per day. 

Keep a lookout for our next video in our series of Open Enterprise Intelligence. 

Previous videos: 

Dec 21 2014
Dec 21

How to use Drupal hooks

What is a Drupal Hook?

In short, a Drupal Hook is a method that allows you to interact with Drupal data (by hooking it). Drupal Hooks are extremely powerful and are one of the main attractions for many people to Drupal. These hooks allow you to view, change and work with various data at specific points of time in Drupal’s processes. For example, you can change data of a form structure before it’s displayed to the user, or you could modify a Node’s data values before it’s saved to the database.

How to create a Drupal Hook

A Drupal hook name is displayed as a function name that starts with the word “hook”. For example, “hook_form_alter” is a very popular hook that is called before a Drupal form is rendered for output.

The word “hook” will be replaced with either the name of your custom module or the name of your theme where your hook will be located.

NOTE: Hooks placed in themes will only function for processes that occur for that theme. For this reason it is recommended to place your hooks into a custom module. This will allow your hook to be called no matter which theme is currently being used.

MORE INFO: For a brief introduction to creating your own custom module see “Creating a Basic Drupal 7 Module”

If we look at the documentation on Drupal.org for hook_form_alter we’ll see it has three arguments:

hook_form_alter(%$form, $%form_state, $form_id)

If you have a custom module called "custom_hooks" then your hook would look like this:

function custom_hooks_form_alter(%$form, $%form_state, $form_id){
  
// logic goes here
}
?>

Notice that all we did was replace the word “hook” with “custom_hooks” (the name of your module). Now, once your module is enabled, every time a form is displayed on your website you will be able to inspect and alter the form’s data before it is output.

NOTE: If your hook does not appear to be functioning try clearing the site’s caches.

A basic and contrived example of this hook’s usage could like this:

function custom_hooks_form_alter(&$form, &$form_state, $form_id) {
 
// Check the Form's ID, if it's a Page Node creation form then let's modify it
 
if ($form_id == 'page_node_form'){
   
// Change the Form's title
   
$form['title']['#title'] = 'Enter Page Title for this page';
   
// Give the Page a default value
   
$form['title']['#default_value'] = 'Drupal hooks are awesome.';
  }
}
?>

This little bit of code in your module would modify all Page Node creation forms to give them a new title and default value for the title field. Of course, this could be achieved through the page’s content type configuration but this should give you an idea of how to harness the power of the form_alter hook. This can be used to alter titles, descriptions, default values and even add new fields to any form that is rendered using Drupal’s form API system.

Drupal 7 Custom Hook example hook_form_alter

How to find Drupal Hooks

Now that you we understand what a Drupal Hook is and how to implement it, we just need to know what hooks are available. A good starting place is the listing of Drupal's core hooks which can be found on Drupal.org. To help find a the hooks available to a specific module you can try looking in that module's directory. Most, but now all, modules contain a file called MODULE_NAME.api.php. This file has no functionality and just contains a list of hooks for that module, along with examples of each hooks' usage. All core and most contributed modules should contain this file.

The screenshot below shows the blocks.api.php file contained in the core Blocks Module directory. The hooks shown in this file will allow you to create your own Drupal Blocks or modify existing ones. Like most module.api.php files, this one has well documented examples of the hook's usage.

Drupal 7 blocks api hooks example

Using a module's module.api.php file should list all hooks available to that module and should be a great place to get started if you need to act upon, modify or interact with a module's data. Hooks are extremely powerful, lending to Drupal's flexibility and are well worth understanding their usage.

Example Hook Usages

Dec 20 2014
Dec 20

By

'Twas the night before Christmas, here at the North Pole,
and I feared our big deadline was a hopeless goal.

Our long lists of boys and girls, and addresses too,
were now vexing to manage — a redo overdue!

The data were buried in a mess of spreadsheets,
and the elves were crying "Help!" in emails and tweets.

They cursed and they swore when Excel crashed once more.
(So a mansion in Redmond I vowed to ignore.)

When outside the workshop there arose an odd noise,
I waddled from my old desk with more hope than poise.

Out in the snow was parked a snazzy flying car
with shiny blue paint that reflected the North Star.

The driver emerged and my dark worries did cease
because I knew right away it must be "Saint Dries".

I welcomed them all — a team led by the tall gent
with spiky hair-icicles and Belgian accent.

I told them our sad tale of a data nightmare,
of flat files, scattered papers, and flaky software.

The content mismanagement was too much to bear,
but they assured my poor staff we need not despair.

"Replace that tangle of one-off utilities
with a website using Drupal's abilities."

They spoke no more praise but began working like mad,
and soon produced wireframes that looked totally rad.

They chose modules with care, these downloads they came.
The team typed happily as they called them by name:
"Now Token and then Rules!
Next Views and CTools!
Then JQuery Update!
Plus Backup and Migrate!
From Drupal.org,
download and install!
Add them to the website!
Enable them all!"

We gave them plenty of our taxonomy tags,
such as "toys" and "dolls" and "coal" and "bags".

The cool website they built would surely save the day,
and cookies and milk is all they asked for in pay.

They left in their car, like a streaking water drop,
when its flux capacitor gave a thunderous pop.

And I heard them exclaim as they faded from sight,
"Merry Christmas to all, and to all a Drupal night!"

Dec 20 2014
Dec 20

I was not posting to the blog for a long time, and finally it’s time to share my experience with new project. This post will also cover some badly structured thoughts about PHP frameworks :)

As part of Qwintry team, we’ve built a great b2b product in logistics sphere - Qwintry Logistics - delivery from US to Russia - and it was built on a new PHP framework, so it was a big deal for us - lots of new experience, and it’s great that it was not just experience but a business success as well :)

Qwintry Logistics (I will use QWL further in text to avoid typing these words again) is an IT system which provides our b2b customers (US stores and freight forwarders) a new way for a high-quality and affordable delivery to Russia, with simple API for integration. We are not directly competing with monsters like USPS or UPS for cross-border delivery, but we are doing very similar thing here - delivery from stores/warehouses to customer door (or pickup point - which in many ways can be more convenient for the end customer than courier delivery to door).

Under the hood the system connects Qwintry warehouses (pickup hubs, if we use right terminology) in Oregon and Delaware, multiple US companies for trucking freights to airports, IATA agents and companies that book air freight for us, customs brokers (to do customs clearance in Russia), and multiple delivery companies in Russia/Belarus/Kazakhstan. All these multiple partners are connected into single workflow, and we get beautiful tracking for each individual package as a result, without investing billions of dollars to build our own infrastructure (that’s where we differ from UPS and USPS):

At the same time, the IT system and some know-hows and experience in automation of fulfillment and logistics, that we got while working on Drupal-powered Qwintry b2c website (freight-forwarding business with 100k+ registered customers) allow us to get the amazing quality and speed while keeping costs low and pricing matrix attractive.

Framework selection: Drupal8, Symfony2, Yii2

When we decided to start building the QWL website, our team had plenty of experience in Drupal 5/6/7 and some dirty hands in Drupal8, and decent experience in Symfony2 (several big projects on air), with my mind still being restless due to the fact that none of these frameworks completely met following requirements for the future logistics system of QWL:

  1. being blazing fast for authorized users and API calls
  2. configuration in code, no configuration in db
  3. ability to write less code, with the code being beautiful, clean, and fun to write
  4. being simple PHP platform to quickly hire developers without skyrocketing budgets

Why not Symfony2?

While a lot of members of our team was (and still are) in love with sf2 - I was not, since writing sf2 code was never a fun for me - never even remotely looking like fun. Yes, I’m lazy, and I’m probably not this perfect developer which sf2 is built for :)
I realized that the level of sf2 complexity and over-engineering is too big for me - very clearly - when I noticed that I kept forgetting what was the business (real) task I was trying to solve - just after five minutes of staring into a bunch of routing or configuration files in our not-too-complex project, or into Factory to generate some basic form. I just dived into the code and tried to solve issues that sf2 was throwing in my face most of time. Not business issues, but issues related to level of perfect abtraction Symfony2 provides me. I think that’s a fundamental difference between me and sf2 - is that I want to solve real business issues quickly without diving too much into this game of absolute extend-ability and clean dependency-injectionability this great and 100% testable framework provides :)
My opinion can change some day, but right now I believe that most projects around me and processes around me are not that complex, but they still have tight deadlines and the quicker I can provide real value to the business with my code - the better.

Why not Drupal?

At the same time all the problems of Drupal -

  • big db structure, where writing some raw db queries is a pain - you do ten joins to get basic data from 10 fields - (don’t even try to explain that to financial analytics who are great in what they do, but have very basic SQL knowledge and read-only access to db),
  • bad performance for authenticated users and very bad performance in node_load/node_save,
  • ugly code you need to write (well, comparing to Wordpress it’s a great code, but lets face it - it is not good comparing to modern frameworks with good ORM/AR),
  • configuration in db

- were still there, and we were very well aware about all these issues since Qwintry.com is a big project with 30+ custom modules created by our team - starting from Endicia/USPS integration and ending with our own coupons engine, full-fledged order picking system UI, and highly customized referral program - it’s all there, and we’ve been maintaining it for several years with impressive number of incoming packages and orders, so every line of code that could fail - failed at least several times :)

In big projects during relatively long maintenance periods you quickly realize that this Drupal code has issues:

$order->field_paid[LANGUAGE_NONE][0]['value'] = 1;
node_save($order);
qwintry_log($order->nid, 'Order was paid');
?>

Because sometimes node_save with single changed field still fail due to innodb locks or whatever, and you at least need to wrap it into try { } catch blocks like this:

$order->field_paid[LANGUAGE_NONE][0]['value'] = 1;
try {
 
node_save($order);
 
qwintry_log($order->nid, 'Order was paid');
} catch (
Exception $e) {
 
drupal_set_message('Critical warning for operator');
 
qwintry_log($order->nid, 'Exception thrown during invoicing');
}
?>

Why am I posting this example? Because when your projects grow that big so you have everyday problems like that (and you write a lot of custom code for new features, so your custom code base size is comparable to framework size) - it may be a good sign that you probably need some lower level framework (lower than Drupal), where transactions are there for all objects, working mostly automatically, and where it is a lot cheaper in terms of cpu and speed to save objects to db in ‘traditional’ way of framework - so you don’t do ugly db_insert or db_query(“INSERT INTO {table}”) instead of node_save when you need to process just 30-40 objects at a time.

I was even considering plain PHP, but it would be stupid in 2014 to invent own wheel :)
Another idea was to use Laravel, and I’m pretty sure it would be a viable option - never heard bad things about it - not too complex, but still elegant and fast.
But, instead of that, we decided to try Yii2 which was in alpha quality by the time when we started - and it turned out to be a great framework.

Why Yii2?

Historically, Yii is the most popular PHP framework in Russia, and probably in China (no links to chinese IT websites, sorry) - since the author of the framework, Qiang Xue, is Chinese (lives in US). It is not hugely popular in US, as far as I know.

habrahabr.ru vote for PHP framework in 2013:

Big popularity is not a sign that the framework is good (or bad) - but it usually means you can find developers easier and faster. Finding good Drupal developers is a big issue. Finding sf2 developers is easier, but still can be a problem. Finding Yii developers is easier than finding sf2 developers :) You have to be careful while testing them since the learning curve is not that big in Yii - so there can be quite a number of bad programmers among these guys - those PHP guys that Ruby and Python devs make jokes of :)

I have a lot of friends in Russia who have been using Yii1 for years, but during these years I’ve been successfully doing Drupal projects and we honestly never had a lot of projects where custom code base can be bigger than framework+contributed modules code base, and good speed under highload was not a big factor, so for that type of projects it would be stupid (and too expensive) to use Yii.
I still think that for those kind of projects (where you don’t write big - and 2-3 small custom modules+customized theme is not big - amounts of custom code) Drupal is a perfect engine. You basically develop most of stuff at the speed of prototyping, you have access to huge amount of great contributed modules and to drush - it’s just amazing how quick the complex web development can be nowadays.
But now, when I look back and think of projects that we’ve implemented in Drupal and which had huge amount of development hours invested in it (thousands of hours on code writing) - I think that Drupal was not the perfect solution for these - it gives more issues than value.

So, I used Yii1 in my sandboxes for some time. Than, when we finally needed some PHP framework for serious code writing - we started to use Symfony2 (read my blog post about this selection here) - the Yii1 already felt outdated, while Yii2 was not even remotely ready for usage. And the Symfony2 was exciting to try, giving us the new modern tools like Composer and Twig.

The Yii core devs decided to rewrite second version of their framework almost from scratch, just like Fabien did with Symfony2. It was a long-lasting project, and there was a risk to loose all the community that will switch to fresh sf2, Laravel and Phalcon and never come back. But now I think it was worth it (and I don’t think Fabien regret his sf2 rewrite, as well).

So, it was a period of ~2 years when our team was building Drupal7 projects and Symfony2 projects, after a period of ~7 years of mostly Drupal (since 2005).

So, in QWL, we needed great performance, and the code was mostly custom (a bunch of external APIs to integrate; a huge bunch of docs to generate from db objects - mostly for customs brokers; and a need for good own API) - so it was obvious that it’s not where Drupal shines.
And it was a great moment to try Yii2 since it was close to production quality already.
New kid on the block, with all the goodness of modern PHP development: composer, namespaces, autoloading, great ActiveRecord, prepared for REST APIs building. All that was implemented by slim and very simple framework core, and it works with a great real speed.

What is a real speed? I use that term to distinguish between speed of cached page performance and speed of page performance without cache.
In Symfony2 (which is not fast at all, comparing to Yii) you can’t even disable all the levels of caching, so I had a number of issues with that - when your routing annotations are cached, when your Yaml configuration is cached, when your Doctrine objects and schemas are cached, and when your Twig templates are cached, and when your DI LazyServiceMapPass is probably cached - and you can’t disable all these cache layers completely in development environment due to the complexity of framework features - it’s easy to get lost when debugging something, and if you delete all the cache - the framework will be surprisingly slow while warming up all that.

In Yii2 you can disable all caching, and the dev environment with all the debug panels will still be fast! You can (and you should) cache everything in production to be super-fast, but it’s really amazing how fast it is in development mode. It was one of the reasons to choose this framework.
Another reasons:

1) Configuration of components is dead simple, and is written in PHP:

if you see something like

'sms' => [
    
'class' => 'app\components\sms\SMSC',
    
'user' => 'username',
    
'key' => 'xxx'
],
?>

in website configuration file - you know that you can access this object through Yii::$app->sms and it’s implementation is a class like this:

namespace \app\components\sms;

class SMSC extends Component {
  public $user;
  public $key;
  /* some methods here */
}

so configuration params are transformed into class properties by default, and you are not required to write code for additional processing to transform configuration directives to something the class can use. Yes, it can be restrictive, and Symfony2 developers can give a bunch of examples where such configuration approach is lacking flexibility and may be it is not exactly self-documenting, but in most cases that’s what you need - in most cases you don’t want to know a heck about DI service compiler passes, and you don’t want to create additional XML with service definition (which will be cached, remember? good luck with typos in these xmls) - you just want to create this component and start using it right away.

2) Most of decisions are made by core developers

The Yii2 core feels monolithic compared to sf2 (so core developers made own Logger, own ActiveRecord, and great RBAC permission system is also in core) - but it felt good when I was doing real work, comparing to alternatives with bigger amount of features (using Doctrine+ACL).
I like the fact that Bootstrap3 is there in core of Yii2 and there are widgets to do less typing (e.g. \yii\bootstrap\Modal for modals). I have had experience injecting Bootstrap3 into Drupal7 and I can’t say it’s a big pleasure to do.

At the same time, the culture of contribution in Yii is weak comparing to Drupal world.
You don’t use a lot of contributed modules when you build Yii project - you mostly rely on the core and your own code. Of course, there are a lot of contributed modules but their number is ridiculously low comparing to number of Drupal user-contributed modules - at least, that was my impression. The level of extendability is significantly lower in Yii2 comparing to Drupal, but it didn’t bug me a lot since I know how complex and slow infinite extendability can get. Yii2 is still in many ways better extendable than Drupal - for example, it’s very easy to use your own user and session processing, and customize the registration forms - and this flexibility is more important in big projects than a way to inject something in theme hooks or change menu routing details via hook_menu_alter (Drupal shines here).

Now, in Yii2, most of contributed modules is hosted on GitHub (and Packagist - for composer).

3) ActiveRecord is a lot less typing than ORM like Doctrine.

Active Record is a concept that somehow mixes the object representing each table row and a super-object that can be used to retrieve specified table rows - in single class. Sorry for this terrible explanation, but here is a simple example:

Drupal 7

$shipment = node_load(db_query("SELECT entity_id FROM {field_data_field_tracking} WHERE field_tracking_value=:tracking", [':tracking' => $tracking])->fetchField()); // yes, I know about db_select, but using it in a bit more complex cases looks even uglier than that
$shipment->field_is_sent[LANGUAGE_NONE][0]['value'] = 1;
node_save($shipment);

Yii2

$shipment = \app\models\Shipment::findOne(['tracking' => $tracking]);
$shipment->is_sent = 1;
$shipment->save();

Symfony2

$em = $this->getDoctrine()->getManager(); // if you're in controller
$shipment = $em->getRepository('PixeljetsShipmentBundle:Shipment')->findOneBy(['tracking' => $tracking]);
$shipment->setIsSent(1);
$em->flush();

In ORM (Doctrine) the super-object whose job is to fetch rows from db is EntityRepository, while each db row is represented by Entity object, it means you need separate class for Repository and separate class for Entity, so in the code above the hierarchy of ORM is:
ShipmentRepository -> Shipment

In yii2 ActiveRecord this super-object concept is mixed into ActiveRecord class which also holds event handlers related to each-db-row objects and stuff like that.
Obviously, Active Record is created by developers who are lazy and hate typing and creating a bunch of classes, while ORM is more academic, loosely coupled, supporting SOLID, and beautiful concept, which works better for enterprise development.
But the resulting code of retrieving objects (and this is the code that you write ten times per day) of ActiveRecord is more elegant and short.

The underlying code of ActiveRecord is probably less than 10% of Doctrine code size and you can understand how it works in one day. Then try to explore the Doctrine code some day (I only did a quick overview, and yes, no surprise it’s big and complex)

Some beautiful ideas of Yii2 AR I think were borrowed from Eloquent (Laravel AR), for example the relations:

class Shipment extends ActiveRecord {
  function
getAuthor() {
    return
$this->hasOne(User::className(), ['id' => 'user_id']);
  }
}
?>

now you can get owner of shipment:

$shipment = Shipment::findOne($id);
echo
$shipment->author->username; // separate lazy sql query against users table executed here
?>

or, you can do it via join:

$shipment = Shipment::find()->where(['id' => $id])->joinWith('author')->one(); // select * from shipment left join user ...
echo $shipment->author->username; // no sql query here!
?>

Isn’t it beautiful and expressive?
While I always had problems remembering the arguments and names of Drupal zoo of db functions (in D7 it got even worse with EntityFieldQuery and db_select - I don’t really want to repeat myself so read more about this in my previous post) - it really lacked simplicity, straight-forwardness and “it works for everything in the system” impression, - and while I didn’t like Doctrine DQL and had bad experience extending this DQL syntax to add some condition to my search query - Yii2 simplicity was like a miracle for me.
You can fallback to raw sql in conditions when you need, still getting AR objects as a result:

// get 20 shipments which are older than 1 month or which author email is test@test.com
$q = Shipment::find()->joinWith('author')->where(['author.email' => 'test@test.com']);
$q->orWhere('shipment.create_time , [':time' => time() - 60*60*24*30]);
$shipments = $q->limit(20)->all();

foreach (

$shipments as $shipment) {
  echo
$shipment->tracking;
}
?>

By the way, Yii2 supports transactions out of the box, and you can save objects reliably and be sure that you don’t leave your db in a bad state.

Other things to note in Yii2:

Bower

- jQuery is in core but it is not glued into the code - it is downloaded as a Bower package, and the idea of Bower manager support via Composer looks very nice (minor issue: it seems to slow down the process of composer update process).
It means that Yii2 modules can specify dependencies to bower packages. (Bower is a package manager for frontend libraries, mostly for javascript libraries - built by Twitter and seems to gain some momentum now)

DB is primary, no schema generation from code

- DB is primary, while the code is secondary. Very important difference comparing to Symfony2 ORM and Drupal schema: if you remove your database completely, in Symfony2 you can get full schema of db from your ORM models, in Drupal you can do the similar thing, in Yii you can’t do that easily - the AR code ‘learns’ about your db structure on the fly. You need some good db admin software to do db design - but I think that SQLyog interface is superior to writing Drupal schemas by hand or using Symfony2 console utility. The Doctrine2 ability to generate ALTER queries from changed models still looks very cool :)
In Yii2, to do migrations, you go to db admin, alter the table using its ui, and copy&paste the resulting ALTER to migration file, that is later pushed to git. In db there is a separate table to remember migration states. This approach is pretty simple and gets things done.

Template layer

- the core template engine is plain PHP - the only thing I don’t like here is that it’s easy to forget to escape some user data, and it’s too much typing to escape everything:

= \yii\helpers\Html::encode($var) ?>

I don’t like this, so I made a shortcut function:

function e($var) {
  return \yii\helpers\Html::encode($var);
}

and now in templates I use

= e($var) ?>

- still not as beautiful as {{var}} in Twig, and not OOP, but I can live with it (for now).

BTW, there is a Twig integration for Yii2 - maintained by Yii core devs. Not tried it yet.

The big difference between Drupal templating and Yii2 templating is that Yii2 templates mean more typing and copy&pasting when you start - since you need to add real template (view) for each form, and for each page, and for each model CRUD, while in Drupal when you create new node type or simple form you don’t need to take care of creating any template, you need to think of it only if your need some fancy html structure.
At the same time, I found it easier to manage html structure of complex form in Yii2 than in Drupal7 (which has a great theme layer - but you can write a book on how you do specific things here - and that is a killer for newbies).


               
        = $form->field($model, 'broker_status')->dropDownList(Shipment::getAvailableBrokerStatuses(), ['prompt' => '']) ?>
       
               
        = $form->field($model, 'broker_message')->textarea() ?>
       

to do this basic styling in Drupal you have to fight with form_alter (add #field_prefix to mimic the required structure) or override template completely which means you need to add separate theme callback - and while it is not a rocket science - lazy developers may feel a bit upset when designer says that he needs this element floating there and he can’t do it just via CSS, using default Drupal form html structure.
In Yii2 you go this extra-mile at the start, using Gii which generates CRUD for models from db table, and then you have precise control over the html, it is easy to modify and support - and all the the templates in the system work the same way - and you know exactly in which file some html structure of registration form located. In Drupal, during maintenance period of some complex project, you always need some time to figure out how this specific div was generated - was it some form_alter from contributed module adding it, or was it your div from your custom theme?

In Symfony2, the Twig engine is great, and I can’t say anything bad about it - it is similar to Yii2 approach. I’m afraid the Twig won’t save Drupal8 theme layer from complexity unless devs are ready for big functionality degradation (because Drupal flexibility is complex, and if you convert all these theme_ callbacks to .twig files - you will also get performance degradation that will slow down Drupal even further - and cache cleanup will be a bigger pain than it is now.. but we’ll see the result when Drupal8 is released)

Forms

In Yii2 forms are usually created on the base of some model.
That is a nice concept - ActiveRecord and form models have validation rules described in same syntax - with rules() and scenarios() methods inside model class. Note that the amount of typing is minimal, and you can pass anonymous function for custom validation rule - and it’s again no redundant typing, and quick result.

Client side validation is automatically generated from validation rules described in model class.

The nice ‘side effect’ of same validation rules used not just for forms, but for base Model class is that you can launch your validation for some ActiveRecord object simply by calling

if (!$shipment->validate()) {
$errors = $shipment->getErrors();
}
?>

You also get CLI interface (which is easy to extend by putting controller in special folder), access control, various logger types (logging to files, db, syslog, email with granular setup is available), basic web ui for rapid html and model generation (called gii), mongodb support, memcache and redis support for caching (file caching is default), sphinx support.

Implementation of specific features:

QWL project required simple API, and due to tight deadlines we decided to use non-REST approach for first version of api.
You can check how the api looks like in our docs.
This is where validate() method of ActiveRecord worked great and it allowed us to create less duplicating code for API functionality - basically the API methods use almost exactly same POST data and apply the same validation rules that are used in control panel when customer uses web browser to create the package - the errors are just rendered in different way and returned in JSON.

Some notes regarding API authentication: in Yii2 this part is pretty well thought through and there is a separate handbook page for this. In QWL, when customer uses the web panel - he is logged in through sessions and cookies, and during the api call the current user object is created on different authenticator mechanism - header tokens in our case - and this didn’t require a lot of developer time to implement.

Another interesting task was creating a dashboard where operator can take a quick look at the packages that require special attention (if they are sitting for too long in pickup point, or they have active issue flag, or some status in tracking is incorrect and unexpected, etc). That is a very important feature that allows us to monitor the health status for each part of logistics chain.

We have pretty big and advanced dashboard for operators on Qwintry b2c website (Drupal powered), and it is implemented using Panels and Views. It has 15+ panes with misc stats and to make it open in adequate time that human can wait (2 seconds) we had to move most of panes to ajax loader (that was a custom hack for Panels) - otherwise the loading times could be 10+ seconds.

In QWL, for now, the dashboard is simpler than that - but it’s still amazing that 4 panes with 80 packages (which are converted to real ActiveRecord objects, while in Drupal Views rows are not even converted to nodes - it’s just arrays of data - for better performance) are opened in less than 200ms, that means we don’t need to think of Redis caching or other performance tricks like ajax loading now - we spend this time on development of new features instead.

We also implemented flexible file storage for documents - when package is created - all the files are put to local filesystem for faster access and preview, and when the shipment is marked as “delivered” - all the files are moved to S3 for long term storage, to save local file space.
We’ve created File AR model with ‘storage’ field and when the ‘storage’ field is changed to s3 - files are copied to s3 - during afterSave hook of the model. We’ve used AWS sdk for php for this, and very thin Yii2 component so s3 is available as a service.

We’ve also integrated littlesms.ru api for SMS notifications - and it was also available as a service throughout the system so sms are sent like this:

Yii::$app->sms->sendMessage($phone, $message)
?>


and in project config it is:

'sms' => [
           
'class' => 'app\components\sms\LittleSMS',
           
'user' => 'user111',
           
'key' => 'pw'
       
],
?>

when littlesms.ru was discontinued we switched to SMSC api, but we didn’t change any code using the sms service, we just created new (very simple) component on the base of SMSC code and changed the system config:

'sms' => [
           
'class' => 'app\components\sms\SMSC,
            '
user' => 'userSMSX',
            '
key' => 'pw2
       
],
?>

both components had same method sendMessage (it could be the same Interface with sendMessage method or even an AbstractClass that both classes inherit) so the change was seamless for the remaining code.

Another cool feature of QWL is implementation of AMQP protocol for queues of tasks (unfortunately, still nothing like that is implemented in core).
For example, when we need to generate documents for broker, it can be an archive with 200+ pdf documents.
We use mpdf for pdf generation, and it is not quick at all, unfortunately. So it’s obvious that this is a long running task that needs to be executed asynchronously.

In Drupal, we would use Batch API, but Batch API has some flaws - if you occasionally close or refresh the page - the process won’t be finished. You also can’t process Drupal batch from cron (you need to use Drupal queues in that case - but they are not good for web browser usage). So, we decided to create scalable solution that will work both from web browser and cron execution. My teammates with bigger Yii experience used some code from older Yii1 projects here, with adaptation to Yii2.
We use RabbitMQ and ActiveRecord model which keeps the progress information.
We also have PHP daemon (which is based on Yii CLI controller) for workers - and monit utility for daemon monitoring. We also use custom Yii2 widget which shows bootstrap3 progressbar which is updated by javascript that uses ajax to check the task progress status.
The result looks like this:


We use this task queue when sending sms and email messages, generating pdf documents, generating document archives, and in other places.

The QWL also has a registry of courier companies and a set of methods so final delivery company is selected on the base of multiple parameters (pickup point selected by customer, address city, box size, box weight, etc) - but that’s a topic for separate post. Pickup points map contains merged locations from multiple courier companies.

Conclusion on framework selection

Yii2 is simple, and if you have good PHP background - it will be easy to learn.
When I was learning it, and looking into how things are done here, I got this great feeling that authors of Yii2 did a lot of stuff exactly the same way I would do it in a perfect framework that I would create.
Many times my thought was “These guys are lazy just like me” :) this great balance of simplicity and flexibility, the balance between having fun writing code and still good code readability in long term maintenance is hard to achieve.
I’m pretty sure some people will say that Yii2 is a bit quick&dirty, and not perfectly testable, and services and configuration management are too simple here, and static calls to get services are ugly - most of it is true, especially comparing to sf2 - and I’m ok with that opinion - it’s just the matter of personal preference, experience, targets, number of developers in the project, and project type. I try to think about business targets most of time when we write our code. I saw lots of great talented developers that dream about 100% test coverage and spend hours discussing which PHP DI component is better - and then they fail meeting the deadlines or just distort the real task so it does not comply to customer needs, but fits into perfect architecture of system instead.
At the same time, nobody wants to get ugly unmaintainable code in repo, me neither.

It’s all about balance :) and for me, Yii2 seems to have good balance.
QWL project is in production, it’s used by customers, and it’s a pleasure to maintain and extend it. We’ll see how it goes.

20 December, 2014

Dec 20 2014
Dec 20

At the bottom of settings.php, add the following code:

$local_settings = __DIR__ . '/settings.local.php';
if (file_exists($local_settings)) {
  include $local_settings;
}

This allows for you to create a new file called settings.local.php within a sites/* directory (the same place as settings.php), and this will be included as an extension of settings.php. You can see the same technique being used within Drupal 8's default.settings.php file.

Environment specific settings like $databases and $base_url can be placed within the local settings file. Other settings like $conf['locale_custom_strings_en'] (string overrides) and $conf['allow_authorize_operations'] that would apply to all environments can still be placed in settings.php.

settings.php though is ignored by default by Git by a .gitignore file, so it won't show up as a file available to be committed. There are two ways to fix this. The first is to use the --force option when adding the file which overrides the ignore file:

git add --force sites/default/settings.php

The other option is to update the .gitignore file itself so that settings.php is no longer ignored. An updated .gitignore file could look like:

# Ignore configuration files that may contain sensitive information.
sites/*/settings.local*.php

# Ignore paths that contain user-generated content.
sites/*/files
sites/*/private

This will allow for settings.php to be added to Git and committed, but not settings.local.php.

Dec 20 2014
Dec 20
markering_403.png

Saturday, December 20, 2014 - 10:19

The editor used to edit posts at medium.com is a real slick, and I find it interesting and intuitive. Davi Ferreira have made an open source clone of it, so it could easily be used in other places.

@cweagans have done great work to get the Medium editor in it's own module, but I would rather myself have it inside the WYSIWYG API. so I took some parts of his work and did a patch, so if somebody else finds it interesting to get this editor to work with WYSIWYG API, please try it out, test, review, throw stuff at it...

As a first step I just added the text editing part, with further plans on try it to get it to work with Asset for images, videos etc.

Blog post author

mikke_mini.jpg

Solution facilitator

Stockholm

Dec 19 2014
Dec 19

This week we held our last board meeting of the year, and we covered a lot a of ground. Unlike November, where we had a relatively short public meeting, this session took the full two hours to get through. We covered a lot of topics, from DrupalCon Amsterdam to updates from the Working Groups. As always, you can review the minutes, the materials, or the meeting recording to catch up on all the details. Here's a summary for you as well.

Operational Update

The month of November was short given the US holiday (Thanksgiving), but we still have a number of initiatives that we managed to push significantly forward. Among them:

  • Licensing Working Group: We recently put out a call for volunteers for the Licensing Working Group, whose charter was approved at the November board meeting. If you are interested in licensing issues, we hope that you will consider applying. The Licensing Working Group will play a pivotal role in helping contributors navigate what is and isn't allowed quickly and in keeping our code GPL compliant and safe.
  • Social capital and the Driesnote: In Amsterdam, Dries laid out a vision for the future of contribution in our community. We also began sharing a plan for Drupal.org in 2015 at DrupalCon Amsterdam that aligns with that vision. We have been laying the groundwork over the last few months, working on commit messages and profile improvements that will make it possible to illustrate not just code contribution, but the many kinds of contribution that individuals (and soon, organizations!) make in the Drupal community.
  • 2015 workplans: Association staff have been very busy preparing workplans for 2015 as well. The DrupalCon team has been rethinking food and fun at DrupalCons based on recent survey feedback. The Drupal.org team has been working on the roadmap. Our revenue team has been planning for solidifying the new revenue programs we launched this year (like Drupal Jobs) and planning for new opportunities as well.
  • DrupalCon Latin America: We are all very excited to get to Bogota for DrupalCon Latin America next February. Everything is on track for this event from a logistics standpoint. We have speakers and space and now all we need are more people. We are planning for 400 people to be there and have about 90 registered so far. Normally, we would have a much higher percentage of tickets sold at this point, but with a very minimal price increase between rates, and with the holidays, we suspect we will see more registrations closer to the date of the Con.

Marketing & Branding Committee Update

We're coming up to a pretty pivotal time for Drupal marketing. As we near a Drupal 8 release, the Marketing and Branding Committee can help lead the community in making this the biggest Drupal release ever. In the meeting, the Board voted to approve the appointment of Gina Montoya of Blink Reaction as the new Chair of that committee. Congratualtions and thank you Gina!

DrupalCon Amsterdam Wrap

Over the last few Cons, we have worked hard to collect more data about our attendees and their experience and to analyze that data to understand what's working and what's not. We looked at a LOT of data for DrupalCon Amsterdam, and shared what we learned and what we will be applying to future Cons. In short - the Con was very successful financially, but we continue to struggle to collect session evaluations and, frankly, the food was terrible. We are very sorry about that. Basically, until the last two weeks before the Con, ticket sales looked slow, so we modified the catering order to mitigate the budget loss we were facing. When the upsurge in ticket sales began, it was too late to change our box-lunch order. We will definitely be rethinking food overall. It's one of the single biggest expenses at DrupalCons, and we know it's one of the best ways to keep attendees happy. Check out the complete overview.

2015 Budget and Leadership Plan Highlights

The board approved the 2015 Budget and Leadership Plan in executive session at the previous board meeting. We reviewed the highlights this month in the public board meeting. If you're interested in even more details, you can watch the recording of the webcast that we presented on Thursday, 18 December.

Governance Updates

Board Term Limits

The Board of Directors operate under a set of rules that govern issues like how the board is structured, the length of terms, etc. This set of rules is codified into the organization's Bylaws. Like any good governance document, and like any good governance group, it makes sense to review how the group operates and what rules might need to be changed in order to provide a better framework for governance. The Governance Committee of the board is charged with ensuring that the board is operating at its best, and making recommendations when things could work better.

In the original bylaws of the organization, terms for Class Directors (nominated and approved by the board, not community-elected seats), are set at 3 years, with a limit of 3 terms. That means that any Class Director could serve a total of 9 years on the board. This is not absolutely a problem, but we do know that board operate best when members are energetic and fully committed, and when new ideas and perspectives can be added to the mix. Nine-year terms work against both of those concepts. To solve for this, the board voted to change the bylaws and limit service to two 3-year terms, or 6 years total. A board member does have the option of taking a year off at that point and could be re-appointed after a year of downtime. We are currently updating the bylaws document to reflect this vote and will update the Association site when this work is complete. 

Community Elected Candidates

One other issue that has been raised by the board is preparing community-elected board members for their service on the board. This class of directors exists to provide a balance of perspective on the board, and everyone understands that many community-elected board members will likely have little board experience prior to their service. The board wants to ensure, however, that these members can jump into their term easily and figure out how to advocate for their agenda quickly. To that end, the boad agrees that it makes sense for candidates to at least have some experience with the mechanics of the Association Board. The Governence Committee recommended that a requirement of board meeting attendance would be a low-threshold to meet, and would expose candidates to how the board operates. The proposal was that, starting in the 2016 elections, candidates will need to attend a minimum of 3 board meetings, which can be tracked by Association staff.

This proposal was voted on and adopted by the board. However, I do want to note that it was not a unanimous vote; we had 2 nay votes. The point was made that currently, all board meetings are held at noon pacific on the third Wednesday of the month. That time slot is during waking hours for the US and Europe. It's early in Australia, but doable. However, anyone in Asia, in particular, can't participate in those awkward hours. The suggestion was made that we shift some of our meeting times to accomodate these other time zones if we are going to make attendance a requirement for running. There was general agreement with this sentiment, but no clear conclusion about how to actually make that happen. The board decided to call the proposal to vote now and work out the logistics of shifting board meeting schedules at a later date.

Working Group Updates

Lastly, we got updates from all of the Drupal.org Working Groups: Software, Content, and Infrastructure. In addition to the work they are pursuing related to the Drupal.org roadmap, Working Groups are also reviewing their charters. With more than a year of operations under their belts, and with a full tech team on staff at the Association, it's important to take a look at how things have changed and ensure that charters are still in alignment.

Goodbye 2014!

It has been a big year for the Association and the Drupal community. I want to take this opportunity to thank the Drupal community for all your support for the Association. It's a joy to come to this job every day and work together to take on the challenges and opportunities we face. Your generosity, smarts, and sense of humor makes it all that much more rewarding. I can't wait to see what we tackle together in 2015!

Flickr photo: Matt Westgate

Dec 19 2014
Dec 19
Drupal 7 provides a file field that allows for uploading files and images to your Nodes but it is limited in functionality. The core file field only allows for uploading one file at a time and does not permit drag and drop functionality. However, with a few modules and a little bit of configuration we can easily provide this functionality to your site.
Dec 19 2014
Dec 19

Chris Luckhardt, Drupal MasterThis post is part of an ongoing series detailing the new personas that have been drawn up as part of our Drupal.org user research.

Chris Luckhardt is a familiar face in the North American Drupal scene. An active member of the Toronto Drupal community and a frequent attender of camps, meet-ups, conventions, and more, Chris is a Drupal master in many ways, and an expert in others.

“I’m a Drupal specialist and I do photography on the side,” Chris says of himself. “I tend to do a lot of everything, which is why I call myself a specialist — because I specialize in different elements of Drupal. My favorite areas of Drupal are site building, dev-ops, and front-end development, and I do a lot of agile and scrum project management."

An Industry Veteran

Chris began his Drupal journey with Drupal 6, back in 2008. He’d already worked in web technology for a while, like with Microsoft's proprietary ASP and C#, dabbled in open source products like PHP, and worked with Linux, Apache, and mySQL.

“I knew that open source values aligned very well with my personal philosophies,” Chris said. “I’d worked on some proprietary software, and by the time I’d finished one particularly bad DotNetNuke project I decided I was going to move on in my career and go totally open source. Around that time, Drupal 6 came out and it coincided with a DrupalCamp Toronto event that was happening. It must have been Toronto's third or fourth DrupalCamp. James Walker, who had a hand in forming the Toronto Drupal group, was there, and I talked to him about what I was looking for in an open source solution to work with. He said, 'yeah this is the right option based on everything you’ve told me,’ and that was how I got started. I’ve considered him a mentor for years.

“I took the time to learn Drupal properly,” Chris continued. “I went to a few Lullabot workshops back in the day, took on some projects, and the rest is history. We all started at one point,” he said of his fellow Drupal users.

“For me, learning things the Drupal way was the biggest challenge, as opposed to coming in and doing some PHP coding. What helped me learn — and what helps me to this day —  is the user group meetings. I think by far being involved in the community is the most important thing. It's the gateway — asking questions and seeing presentations is really valuable. Of course, the issue queue is the best way to self-learn, but in my opinion the best learning happens from talking to people, because someone has stumbled across your problem before."

Drupal: Powered by People

Chris has been active in Drupal for years, both professionally and in the larger Drupal community. “Come for the code, stay for the community is the number one reason why I use Drupal,” Chris said. “There are so many other amazing developer communities out there, like PHP, HTML5, Javascript, Angular... I’ve dabbled in all of those and they’re all fantastic, but there’s just something about our community that is very representative of open source technology and community building."

When it comes to that community, Chris is concerned about how to grow it both locally and globally. “We have a very specific problem here in Toronto, but I think everyone has dealt with it too. We have a batch of old-school Toronto Drupal user group members who date back to 05-08, and we have an influx of new people. This means we have a set of introductory and beginner users — you know, people who come in like, “what’s a Drupal?” — and then we have the advanced users branching into all sorts of wild territory with Drupal.

"It’s hard to cater to both groups in one meet-up and even at our DrupalCamps that we plan every year. We recognize, if we try to cater to the introductory users we’ll turn away the advanced users, and they won’t be interested in coming out, but if we do really advanced sessions and training at our meet-ups, the new people show up and they won’t have any idea what’s going on.

“Between James and myself, we decided to address the problem by doing an introductory presentation and then a more advanced presentation during our meetups. For bigger events, it’s a little different. I created the schedule at the last DrupalCamp, and I engineered it so that there would be enough difference between overlapping session timeframes that it would work to the benefit of both the introductory and the advanced attendees…though unfortunately there's not much middle ground."

Linguistic Barriers to Entry

Chris’ other observation about problems with growing Drupal is the language barrier. “I was presenting at a DrupalCamp in Kyoto, and someone raised his hand and said, 'I want to learn Drupal, but I don’t understand Views. How do I learn it?’ So I told him that there are tons of tutorials on YouTube, and he responded, 'But...they are all English.' It occurred to me that those videos show you what to do, but if you don’t understand the spoken information -- why would I click this button, why would I do that -- the vocalization aspect is incredibly important. So there’s a real serious lack of Japanese documentation for people to learn Drupal— and not just Japanese, other languages, too. There’s some work being done by the Japanese community organizers around translating some of the books, like Emma Jane and Angie’s books, so it’s a start.

“So, the biggest challenge I see with Drupal and Drupal.org is how to manage the education… And, actually, sometimes I feel bad about calling myself a Drupal master because the learning curve never stops. It only becomes less dramatic with years of experience."

To see how we plan to address some of the challenges Chris has identified, keep an eye out for conclusion to our Personas series, or look at the results of the user research we’ve performed on Drupal.org.

Dec 19 2014
Dec 19

[embedded content]

Creating and promoting content is crucial for your business, and it can be very frustrating at times.

  • Wrestling with code
  • Formatting responsive content
  • Correctly aligning text and images

It’s second nature for programmers, but for marketers, creating complex page layouts often results in relying on a developer – which increases time and costs money. The good news is that it doesn’t have to be that way.

We've modified CK Editor to make it much more user friendly for those of you working with content on a daily or weekly basis:

It includes several pre-configured, Bootstrap features that can easily be mixed and matched to create an infinite number of page layouts.

  • Pre-configured widgets allow you to easily add Bootstrap's banner sliders, jumbotrons, trifolds, featurettes, multi-column layouts and blocks on any page of your website.
  • Items can be nested and maintain Bootstrap's coding standards so that you don't have to remember classes or touch the code.
  • And if you decide your team needs less flexibility, content editing options can be limited or expanded based on your preferences.

Let’s create a quick page layout to show you how a few of these features work.

Jumbotrons

Using a few of its built-in templates, I’ll start by adding a Jumbotron at the top of the page. Jumbotrons are often used on marketing pages to quickly grab attention and promote a big idea or concept.

Trifolds

Next, I’m going to add a trifold layout to the page. Trifolds generally display on homepages to promote your most valuable products or services. If you are familiar with modern web design, you’ve probably noticed these trifolds on several websites.

Featurettes

Now, I will add a couple featurettes to the page. Since most website visitors scan pages for information, featurettes offer a great way of promoting the benefits of your companies products and services very quickly.

Let’s go ahead and click save so that you can see your layout. As you can see, we very quickly built what would be a pretty complex, responsive page layout in a matter of minutes without writing a line of code.

Open Enterprise empowers you to focus less on design and development and more on creating, publishing and promoting valuable content. The result is a more efficient and cost effective way of reaching your audience. 

Have questions or comments? Let us know what you think.

Dec 19 2014
Dec 19

Code that performs well should be an assumption, not a requirement. I've never had a client ask me, “will your code make my site run slower?" Clients just assume that I'm going to deliver a codebase that does not hold things up.

However, over the years, I've discovered that code can actually have a major impact not just on how well the website runs, but also on how  much work a coder can get done. And here is the big surprise: a lot of the code I review for others, or sometimes code that I wrote years ago myself, doesn't allow me to work as quickly as I (or the client) would like. It actually sometimes slows me down so much that, by the end of the day, it would have been quicker for me to just rewrite it from scratch, using efficient code and accepted coding standards.

The background to this post

I was recently in contact with a client that had a website which was performing poorly - a pretty standard Drupal 7 website (which somehow took over a year to develop). When the site was eventually delivered, it was so full of bugs that the original development company couldn't (or wouldn't) fix it. The task was clear: make the site run faster (a normal content page would easily take over 10 seconds to load) and fix as many bugs as you can within three days.

Before meeting on site with the client, I got a copy of the codebase and began to make myself familiar with it. Oh man. This already looked monumental. I got scared. Scared because I wasn't sure that I could touch anything on the site without making it fall apart, or making things even worse, or conflicting with another piece of code somewhere else. Or ...?

I'm going to give you a couple of examples of what the client and I decided to do, along with some advice I'd like to give to the original developers if I had them sitting next to me. There will be more in future parts of this series.

Files and folders

There was basically no file/folder structure to speak of. Almost every module was placed in a flat hierarchy, except for some custom modules and the odd (unused) feature. So there was approximately 150 folders in sites/all/modules. But I had no way of knowing if they were patched contrib modules, a custom module, or a sandbox module. They could have been anything. Updating modules was a matter of convincing ourselves that the site would probably break somewhere, and we could take it from there.

I suggested that we left the structure as it was for now, but that they should consider creating a structure that looked similar to this (basic Drupal good practice):

drupal-root-folder
- sites
- - all
- - - libraries
- - - modules
- - - - contrib
- - - - contrib-dev
- - - - contrib-patched
- - - - custom
- - - - custom/features
- - - themes

If a module is patched, put the patch in the module folder, so it can be re-applied if needed when next updating that module. This will also make any new developers that come to the project aware that they should pay extra attention to this module. It's also good practice to contribute the patch back to the community if you think other projects might benefit from it.

Warnings and notices

I hate warning and notices. On this website, they were everywhere. Hundreds of warnings and notices on every page. The former programmer/site builder's solution to this issue appears to have been: turn off message logging. I know that makes sense in a production environment, so you don't expose a weakness to a potential hacker of the site, but it does not solve the underlying problem. To me, it just looked like a developer trying to fool a client, and ensure that, though the site had major issues, the client wouldn't be aware of them.

At one point it was so bad that the customer had to turn off the logging system to help the database that was being overloaded with error logs. However, this meant that if there was a problem - such as the site getting hacked - they couldn't do any meaningful debugging because there was no error log to check.

The solution was pretty simple: start from the homepage and fix all the code that was throwing warnings and errors. Next move on to a sub page, and so on. I spent long, long hours on this, but the work paid off in the end: we got the amount of warnings down to a level where it was possible to have the logging system running again so we could get some real debugging done.

But I still couldn't touch the logic in the code, because I still didn't have a clue what most of it was doing.

Indentations and variables

It is obvious to many programmers that coding standards matter. In this case the developers didn't care about correct indentations, or any other coding standards for that matter. This is an example of what a typical piece of code on that site looked like:

        case 'tasks_node_form':
          $key_value = array_keys($form['field_task_listing_ref']['und']['#options']);
          $ff = array_shift($key_value);
          $new_arr['_none'] = '- None -';
          foreach($key_value as $val)
          {

              $inode = node_load($val);

              //$new_arr[$val] = $inode->title.' -> '.second_mlid_title($val);
              // calling functtion to get complete listing (added by Bob the Builder)
      if(get_complete_listing($val) != ''){
              $new_arr[$val] = $inode->title.' -> '.get_complete_listing($val);
      }
          }
          $form['field_task_listing_ref']['und']['#options'] = $new_arr;
          $form['#after_build'][] = 'custom_after_build';
          $form['#validate'][1] = 'lets_compare_date';

      foreach ($form['field_task_elements']['und']['#options'] as &$option) {
            $option = decode_entities($option);
        }
      foreach ($form['field_section']['und']['#options'] as &$option) {
            $option = decode_entities($option);
        }


     break;

No, there's nothing wrong with your screen, this is how the code was presented when I saw it for the first time. It's almost impossible to read and understand when you have 2000 lines of code looking like that.

But, look carefully and you'll also see a XSS security issue.

You can read more about how you should indent your code (and other coding standards) here.

Comments

The code contained comments. Really, it did. But there was no point to them. They didn't help me to understand what was going on at all. Comments like, "Loop through array", "Call function that cleans up" and "Returning the array". And one that really scared me: "If the alias already exists, generate a new, hopefully unique, variant". We are hoping for a new variant? Why not just make sure it's unique?

The rest of the comments were primarily just bodies lying around (dead, unreadable or commented code). No tombstones (comments, telling why the code is commented), so I could't tell if the code had a reason to stay there or not.

Here are some general rules when commenting code:

  • Don't ever leave commented code in your code base, unless you write a comment above it, telling why this code should stay in there.

    • A "we might need this later" comment isn't good enough. That's what your SCM (Source Control Management, like git or SVN) is for. You can always go back in your scm's history, and resurrect your code from there.

  • Don't state the obvious.

    • I know a loop when I see it. I can read the return statement, there is no need to comment it. I want to know why you are doing something, not what is being done. Don't tell me that you called the clean-up function. Tell me why we need to clean something up.

  • In general, don't comment variables.

    • It's way better to give the variable a descriptive name. If you can't tell from the name of the variable what it contains, you should consider renaming it.

    • Of course, there can be situations where a variable can hold an unspecified data type or unspecified content, and here it is important to make sure that you write a comment saying that.

  • The most important comments in the code are the DocBlock (documentation block) comments.

    • These are the comments that are before EVERY function throughout your code. Don't EVER create a function, without creating a good DocBlock. They are not hard to write. Read more about them here.

    • A good DocBlock doesn't take long too write. If it does, you probably did something else wrong, like your function is doing too much. Try to split it into smaller more specific functions.

    • The first line of the DocBlock is an especially good indicator of whether your function is trying to do too much. The first line must contain a summary of what your function is doing. And it's a one-liner only (80 charecters max). If it takes more than that to write a good summary, consider splitting the function.

    • After the first line, you leave an empty line, and after that again, you can add descriptive, in-depth explanations.

    • Lastly remember to describe all the parameters and the return value as well, using @param and @return.

  • Keep in mind that the DocBlock design that I describe above isn't necessary when you are writing hook functions.

Drupal Coding Standards

Drupal's Coding Standards are there for a reason. They weren't created for fun, or to make text look pretty.

When you return to a broken or unfinished module, or when you read unfamiliar code, and the code is strictly written to comply to the standards, you know how to read it straight away. You know what to search for, because the variables are in the same format. You know what every single function is doing, because it's explained in the comment above the functions. You know what data type a function's parameters expects, and what every function returns.

That's what allows the developer to perform - to get work done quicker. In the long-term, it's a money saver.

Conclusion

The site was FUBAR BUNDY.

What I suggested to the client was something like "Lets get this patched up, so it can last at least a while longer, and then get something new built."

I suggested that they should build the new version themselves as they were now familiar with Drupal and how to code Drupal modules. They could then just get help from consultants like Annertech to help with any tricky aspects that might crop up.

I'd like some help from Annertech

Dec 18 2014
Dec 18

Lose weight. Eat better. Run a 5K. Travel more. These are resolutions we all make year after year. But this year, we challenged our team to think outside the box and inside the drop. Now that 2014 has come and gone, and we prepare to countdown to 2015, we asked our team what they are looking to accomplish in Drupal in the New Year.

“Get more of my modules out for D8.” - Andrew Riley

"Push the limits of what Mediacurrent can do with Drupal." - Brad MacDonald 

"My goals for 2015: Get up to speed on D8 (have lots training materials, yay continuing education!), port all of my modules and have a 1.0 release." - Damien McKenna 

"A real one for me would be to release more modules publicly. I guess to be more specifc, find more problems that need solved and solve them by releasing a contrib module to the community" - David Younker

"You know the usual... Eat better, workout more, lose 20 lines of code..." - Mark Casias

"For me, it's learn Drupal 8 and port Advanced Forum and Author Pane to it." - Michelle Cox 

"Resolution: Build something with headless Drupal and a JS MVC... And get more t-shirts." - Zack Hawkins

"D8." - Rob McBryde 

"One of the things I'd like to accomplish at DrupalCon 2015 is to speak on behalf of Mediacurrent.  I think this will be very challenging and intimidating and that's exactly the reasons why I want to do it. Also, I want to learn more about Drupal 8 and in particular Theming with Twig." - Mario Hernandez

Here at Mediacurrent, we're all about helping people reach their goals through our Drupal expertise and resources. So what is your Drupal New Year's Resolution and how can we help you make them a reality? Let us know in the comments what you'd like to see more of on our blog in 2015!

Additional Resources

Why You Should Speak at Tech Conferences | Mediacurrent Blog Post
10 Things I Wish I Knew About Drupal 2 Years Ago | Mediacurrent Blog Post
Resource Guide to Drupal 8 | Mediacurrent Blog Post

Dec 18 2014
Dec 18

PhotoIn the beginning there was the Common Gateway Interface, commonly known as CGI – a standard approach used to dynamically generate web pages. Originally devised in 1993 by the NCSA team and formally defined by RFC 3875 in 2004, CGI 1.1 took seven years to go from the original RFC to an endorsed standard.

In 1994, not long after the original CGI standard was documented by NCSA, Rasmus Lerdorf created Personal Home Page tools (PHP Tools), an implementation of the Common Gateway Interface written in C. After going through a number of iterations and name-changes this grew to be the PHP language we know and love.

One of PHP's strengths was the way in which it made many of the request and server specific variables, as defined by the CGI standard, easy to access – through the use of superglobals, namely $_POST, $_GET, and $_SERVER. Each of these is an associative array. In the case of $_POST, the request body is parsed for you and turned into an array of user-submitted values, keyed by field name, and conveniently supporting nested arrays. Similarly for $_GET, the query string is parsed by PHP and turned into a keyed array. In the case of $_SERVER, the gamut of server-specific variables are available for your script to interrogate.

Now, as Drupal developers, we rarely interact with $_POST and $_SERVER, and seldom interact with $_GET. Variables submitted through forms are abstracted behind Drupal's Form API; we ordinarily work with the $form_state variable instead of $_POST. In terms of the $_SERVER superglobal, Drupal provides handy wrappers for many of these values too, so instead we interact with those rather than with $_SERVER directly.

Please note that this article is accurate at the time of writing, however there is a proposal (see http://wdog.it/4/1/attributes and http://wdog.it/4/1/match) to improve the developer experience of working with important attributes stored on the Request object. The goal of these issues is to introduce methods on helper classes to make access to important elements of the request/response life-cycle more explicit instead of relying on using arbitrary string keys to fetch items from the Request object.

Under the proposal, items that relate to the routing system will be available from a RouteMatch class containing useful methods, for example RouteMatch::getRawArguments(). This would be used instead of $request->attributes->get('_raw_variables'). Under the proposal this will form a key developer facing API for working with upcast routing parameters so RouteMatch::getArgument('node') will replace $request->attributes->get('node'). This makes sense because the node object is not part of the request, it is a product of the routing of that request.

In addition the existing RequestHelper class will be expanded to add additional methods for interrogating concepts related to the Request but unrelated to routing. For example RequestHelper::isCleanUrl() can be used to determine if the incoming request utilizes clean urls, instead of $request->attributes->get('clean_urls').

Before using the example code in this article, be sure to first review the above issues.

If It Ain't Broke, Don't Fix It?

So what's wrong with using $_GET, $_POST, and $_SERVER in conjunction with some other Drupalisms™ like arg() and current_path()?
Well, does the code below look familiar?

attributes->get(RouteObjectInterface::ROUTE_NAME);
 
if ($route_name == 'mymodule.someroute') {
  // Do something.
}

Getting Request Headers

The Symfony Request object also includes a HeaderBag (Symfony\Component\HttpFoundation\HeaderBag), which gives easy access to any request headers you might need. The HeaderBag contains the same convenient methods as the ParameterBag

Deep Integration with the Routing System

Looking back at that same routing entry from forum.routing.yml, you'll note the 'defaults' section. Those values can also be found in the attributes parameter bag:

attributes->has('foo');
}

Now while that might seem simpler than having the Request object passed in as an argument, you should resist the temptation to use \Drupal::request() in object-oriented code. Calling out to the global \Drupal singleton in your object-oriented code makes unit-testing your code more difficult, as you need to setup a container, add the request to it, and then call the setContainer method on \Drupal before you can do so.

Injecting the Request Into a Service

If you are creating a service and you need access to the request, you can nominate it as a dependency in your services.yml file thusly:

services:
  mymodule.foo:
    class: Drupal\mymodule\MyModuleManager
    arguments: ['@database', '@request']

But this depends on the life-cycle of your service, as the request service isn't always available. For information on how to create a service that might need to be instantiated before Drupal enters the Request scope, see the Getting Advanced section later in this article.

Bring Your Own – Creating a Request

If you're writing code, such as a custom application or a shell script that lives outside the normal Drupal bootstrap, you might find that you need to instantiate a Request object. Or you might be working on some tests and wish to create a pseudo Request. In these cases, you need to build your own Request object. There are two approaches to doing so, depending on your use case.

Transforming Globals Into a Request Object

If you're working in a script or custom outside the standard Drupal bootstrap, you can easily transform the superglobals into a Request object using the static createFromGlobals method:

Making Your Own Request

If you need to create a request for simulation purposes while testing or some other purpose such as a sub-request, you can manually create one using the static create method. This takes arguments for the URL, method, parameters, etc.

  
Dec 18 2014
Dec 18

You may have heard and read a lot about Drupal 8 lately, without much support to go along with it. Well here at Blink Reaction, we are working on changing that and contributing as much help as we can to the community with the issues that we’ve come across so far in Drupal 8. In this post I will show you how you can try Drupal 8 by installing dependencies such as composer and drush so you can have a Drupal 8 site running on your local machine.

Disclaimer:

For this blog post I will be using Mac OS X Yosemite which is the OS I have on my development machine.

To follow this walkthrough you should have an *AMP stack installed and working on your machine. If you don’t already have an environment, here are some options: 

* Mac: Acquia Dev DesktopMAMP or php-osx
* Windows: XAMPPWAMP or Ampps
* Linux: Install with your package manager or Ampps
* Use a Virtual Machine

Whichever environment you choose, make sure you have the following:

* PHP 5.4.5 or higher (very important!)
* A MySQL server that is configured to work with PHP

Download and Install Composer

Composer is a dependency management library for PHP. Follow the instructions from the composer website at https://getcomposer.org/download. You can download it anywhere onto your local computer. 

Then, if you have curl installed:

$ curl -sS https://getcomposer.org/installer | php?

Or if you don't have curl installed:

$ php -r "readfile('https://getcomposer.org/installer');" | php

Accessing composer from anywhere on your system:

To access composer globally move the executable 'composer.phar' to a directory that is already in your PATH.

$ chmod +x composer.phar
$ mv composer.phar /usr/local/bin/composer

Download Drush

Clone the drush git repository in a directory that is outside of your web root.

$ cd /usr/share
$ sudo git clone --branch master https://github.com/drush-ops/drush.git

Install Drush

From the drush root directory, run composer to fetch the required dependencies.

$ cd /usr/share/drush
$ sudo composer install

Accessing Drush from anywhere on your system.

Create a symbolic link to the Drush executable in a directory that is already in your PATH.

$ ln -s /usr/share/drush/drush /usr/bin/drush

Download Drupal

At the time of writing this blog post the latest release of Drupal is 8.0.0-beta3. In order to download Drupal 8 we will use drush dl command an alias for drush pm-download.

$ drush dl drupal-8.0.0-beta3
$ mv drupal-8.0.0-beta3 drupal8.dev

Install Drupal

Change to the directory where Drupal was downloaded by drush and use drush si command as an alias for drush site-install.

$ cd path/to/drupal8.dev/
$ drush si standard --db-url=mysql://root:root@127.0.0.1/drupal --site-name=drupal8.dev --account-name=admin --account-pass=admin --account-mail=jesus.olivas@blinkreaction.com -y

Make sure you use your own user and database credentials when running drush si and never user root on production. In this example we are accepting any interaction - answering yes when passing -y argument.

As you can see, when using Drush the process is very similar to previous Drupal versions - drush dl & drush si.

Finally, if you want to access your Drupal 8 site using http://drupal8.dev you will have to:

Add a new host in your hosts file (‘/etc/hosts’ as I am using Mac OS X)

127.0.0.1 drupal8.dev

Add a new virtual host in our web server config. Apache, in my case.


   ServerAdmin jesus.olivas@blinkreaction.com
   DocumentRoot "/path/to/drupal8.dev"
   ServerName drupal8.dev
  
       Options FollowSymLinks
       AllowOverride All
       Require all granted
  

References:

* Composer: A dependency manager for PHP - https://getcomposer.org/
* Drush GitHub repository - https://github.com/drush-ops/drush
* Drupal project page - https://www.drupal.org/project/drupal
* Detailed information about drush commands -http://www.drushcommands.com

Note:

If you want to try Drupal 8 now. without worrying about configuring your local environment you may use the following online services:

https://insight.acquia.com/free?distro=drupal8
http://getpantheon.com/d8 
http://trydrupal8.com/ 
http://simplytest.me/

Have questions? You can catch me on Twitter @jmolivas. Stay tuned for more helpful posts on Drupal 8.

Blink Reaction sponsors the Console project for developing in Drupal 8.  

Visit http://www.drupal.org/project/console to try it. Please feel free to submit issues and requests!

Dec 18 2014
Dec 18

28 pages of unmarred perfection. This book is pure unadulterated genius

- Chris Arlidge

Never Be Shocked Again! - Budgeting your Web Project

Are you having trouble figuring out an appropriate budget for your next web project? Our whitepaper can help!

Download your FREE COPY, to learn the different types of website projects, understand the factors that play a role in the budgeting process, and determine where your web plans will fit when it comes to costs!

Don’t ever be shocked by web costs again! A clear guide to help you plan the budget for your next web project.

I have been privileged to be able to attend a number of conferences and events, such as DrupalCon Austin, Portland etc,  since we started Cheeky Monkey Media. In the past, we’ve talked about having your DrupalCon Survival kit prepared before you head out the door to help make the most out of any conference or large group you are attending.

BADCamp (Bay Area Drupal Camp) is no exception to this rule. For those of you who are not aware, BADCamp is a conference which focuses solely on the people behind the keyboards and is held annually in San Francisco. The organizers do a ton of work to make this event happen and it shows.

This was our first year going to BADCamp and what an experience. We had the opportunity to check out a great city such as San Francisco and to meet a lot of like minded Drupal developers. The speakers, the sessions, the mini-summits, all delivered on the value we had hoped they would.

Going to this event, we had to make sure we followed our own advice and got prepared.

Why are we going?

That was simple, to just have an opportunity to be there to learn and network with peers on a one on one basis. Typically when we hit events like DrupalCon, there are a lot of competing factors which can get in the way of having the time to take in sessions, such as manning the booth, meetings … the list can go on and on. This time though, we went simply to learn something new and bring it back to our team.

Where are we staying?

We found a great deal on Hotels.com for an older character hotel in the core. We got to experience a bit of glam and see some of the spots we were warned not to go at night time. From our location, it was a 5 to 10 minute taxi ride to most events, and if we were ever in a bind, we had a BART station right outside our hotel.

What events should we hit?

The simple answer, all of them. We had the privilege to be included in the Pantheon Partner dinner, which was amazing food and drinks, while connecting with friends. We knew we were going to hit the two main evening events listed on the site, which as always with a Drupal conference, was never disappointing. On the Friday night, we even found a few people who were wearing our t-shirts we gave away at DrupalCon in Austin, which really made our night.

Who do we want to meet?

This trip for me was not being Gene the owner, but about being Gene, one of the gang :) Before leaving the conference, I reached out to all the people I work with on a weekly basis, as a number of their companies sent people to BADCamp. I made sure to get cell numbers and arrange times to meet face to face. The highlight of my networking was finally getting to meet Drew Stephens after 4 years of talking on the phone, skype and a ton of emails.

What was most important?

For this trip what I found most important was to stay open to any and all opportunities to learn, which we most assuredly did. There wasn’t a single day where we didn’t walk away with more than one great gem to take back to our team, and for me, that is what made the cost and time for the conference worthwhile.

For my first BADCamp, it was an amazing experience, and I can’t wait to be back next year. It is my goal to work with my monkey troops, to come up with something awesome that we can share back with the Drupal community. If you haven’t attended a BADcamp yet, I highly recommend giving it a go, you won’t regret it. If you do decide to attend in 2015, look me up, as it would be great to meet more of our kind!

Dec 18 2014
Dec 18

[embedded content]

Traditional analytics primarily focus on traffic, page views, conversion and a few other high-level metrics. "These are all great indicators, until you try to determine what you should do to convert qualified leads." explains Kylon Gustin, Director of Marketing at LevelTen Interactive. That's where value-based content scoring provides deep insight.

Value-based content scoring reports not only show daily traffic to your website and additional page views, they also provides a more in-depth look into the actual value of your traffic, events and goals.

Open Enterprise IntelligenceTM, for example, focuses on value derived from a number of factors. How engaging is your content? Does your content produce valued results like leads, social shares and comments? Is your content attracting visitors, increasing brand awareness, conversions, and time on site?

When each of these is measured in individual silos, as with most analytics systems, you receive limited value.  However, seaming them together allows you to make educated decisions based on quantifiable data.

All web marketers agree that content is valuable, but not just each piece of content. Authors, type of content, subject matter and several other aspects play a huge role in how valuable a piece of content is to your overall marketing strategy.

For example, you can score traffic sources (organic search, direct, referral, social network) to determine which sources provide the most value to your organization. You can score website visitors to determine which visitors provide the most value through engagement, social shares and form submissions.

So how do you get your hands on this data? It requires a very deep integration between your CMS (we use Drupal), Google Analytics, Add This (social sharing tool), Twilio, MailChimp and other tools that provide crucial data to effectively measure value. Without combining all of your data together, you only get a small glimpse into your efforts. Combining ALL of your data is crucial.

If you are interested in gaining more value from your marketing efforts, we can help. Feel free to schedule a demo or ask questions within our comments below.

Dec 18 2014
Dec 18

We’ve all been there. The clock is ticking down to launch and the issues keep mounting.

Some issues are launch critical, but many of them could wait until after the site has been launched. Some are new features disguised as bugs. All of them, though, will delay the launch. And no matter where you sit on the project team, not launching on a deadline that you know you can meet can be extremely frustrating—especially when those delays can last months or even years. (Yes, I said years!)

Why are delayed launches bad?

Before we explore how to avoid unnecessary launch delays, let’s explore why delaying launches can be a problem. Maybe you need some convincing...

The biggest risk is that the developers and designers will forget what they’ve done and why they’ve done it. When you’re in the midst of building a complicated website, it’s challenging to remember why you made certain decisions along the way. But if you have to revisit something two or even three weeks later (not to mention months later), you might have forgotten that a changing a piece of code will actually affect three different components—not just the one you need to update. This leads to more bugs, more testing, and more delays.

Another critical factor in launching: morale. The project team has poured their time and energy into the project and delays deny them final gratification. Especially if they’ve done something cool on the project that they want to show friends, colleagues, or even prospective clients. But they can’t because the site hasn’t launched.

Last, the project will nibble away at the project team’s time and focus. It’s hard to get into the groove on a new project when you need to spend a few hours a week updating an old project that just won’t launch! It’s the tetris problem of project management and the more of these little pieces you have lying around, the harder it is to effectively allocate resources—and the harder it is for those resources (aka people) to actually finish a task.

To delay or not to delay?

Despite all these issues, sometimes a launch delay is necessary. Maybe a feature is more complicated than you expected and the timeline needs to be adjusted. Or maybe an entire set of requirements were missed somewhere along the line—and those requirements are absolutely, 100% critical for project success. Any of these should be reasonable and limited delays as the problems outlined above still stand. The longer the delay, the bigger these issues become.

A few common problems that delay launches unnecessarily

There are many, many little problems that unnecessarily delay your launch, but there are a few that have resurfaced a time or two for us.

If you are migrating data from one database to another, a big pothole on the road to launch will be inconsistencies in source data. Do not wait until the end of the project to test the data import—even if you’ve provided explicit instructions on how to format the data. There will always be problems with inconsistent data when doing an import. Expect it, embrace it, plan for it.

Probably the most annoying thing to delay a launch: infinite nitpicking. People reviewing the site want the logo a little to the left—no, move it back. Or they want to see the homepage with a different style component for featuring content on the homepage. These nitpicks do nothing to move the project closer to launch. What’s more, actually launching can often quiet the flow of the nitpicks.

But the biggest factor in delaying launches: scope creep. There are actually a few of ways that scope creep can sneak into a project and you need to keep a steadfast eye on them. More than anything, launching on time is about managing scope.

The fastest way for scope to creep: your client talking directly with your designer or developer and making a suggestion or two. The developer thinks these are fairly simple, reasonable requests, so implement them without consulting other people on the team. One or two of these won’t kill a timeline, but 5 or 6 will as each of them adds new testing time and even creates some confusion on the team. In the worst case scenario, what the developer thought would be easy ends up being more complicated—and once you’ve promised something, it’s difficult to back out.

Another way scope creep wiggles its way into projects: New features disguised as feedback on an existing ticket. This is not always an easy thing for developers to see as it can happen so gradually—an actual bug leads to another bug leads to some re-thinking on how the feature should actually work. Maybe the new direction is better than the old, but it’s still going to delay timeline and increase costs—both of which need to be accounted for.

Get that site launched!

So how do you do it? How do you maneuver the project out of launch limbo so that it can be released to the world? It ain’t easy. But there are a few practices we’ve developed over the years that can make a big difference.

The easiest way is to prepare for launch at the outset of the project. If you wait until time to launch to set clear expectations about your launch process, then you’re going to have a bad time.

First, you want to change their mindset about what a website actually is. Many people still view websites as stagnant, immovable forces that, once launched, will never, if rarely, get updated, which is why they want everything absolutely perfect prior to launch. This mindset needs to change. The web is, and has always been, an evolutionary medium. New learning can happen quickly and what you thought was perfect at launch, may prove to be fatally flawed. Better to launch something that is good enough that can evolve based on feedback.

Second, pursue a retainer as soon as possible. Not only will having a retainer allow you to reinforce the idea that the web is an evolutionary medium, but will also give you the runway to help them evolve their ideas without having to scrounge around at the end (which can often leave clients feeling like you’re doing a bait-and-switch on them).

Lastly, if you can, add some time at the end of the project—even if you don’t tell the client—or them to nitpick and tweak features. It’s going to happen no matter how great the client is or how much you prep them, so if you can make that part of the schedule, you can avoid unnecessary delays (and maybe even launch early if they have limited nitpicks).

All these suggestions are great, but are only effective if used at the outset of the project. What happens if you get to the end of the project and you’re running into these problems?

Well, you’re S-O-L.

Not really. (But it is harder to get the client focused if you haven’t been talking about it throughout the project.)

As the clock ticks down to launch, you need to change the rules for filing bugs a bit. Add a new facet to your bugs: launch-blocker. Only bugs flagged as a launch-blocker will get fixed prior to launch. Everything else will get prioritized and resolved post launch. This will require some finesse on your part as what the client thinks is a launch-blocker and what you think is a launch-blocker may be two different things. But you want to reserve the launch-blocker flag for things that are critically broken. (Hopefully you don’t have too many of those at this point!) Everything else can be fixed after launch.

In fact, you can even work this into your process and include that in your periodic process overview discussions.

Launch!

Launching a new or updated website can be challenging, but with enough preparation and planning, you can minimize the tendency that clients have for delaying launches. Help them understand why delayed launches are bad. Recruit them as partners to help you stay on time and only delay when absolutely necessary.

Dec 18 2014
Dec 18

As the Drupal market continues to rock and roll, more and more clients need "Drupal Developers". But what exactly is a Drupal Developer? A Drupal Developer is someone who knows Drupal right? Right?!

There always has been some confusion around job titles and skills in the Drupal world. This is especially true with some recruiters and even managers and clients that are hiring. In effect, there are three main areas of expertise in the Drupal world: site building, backend/module development and theming. The skills required for each are quite different.

Drupal Site Builder

A Drupal site builder is someone who builds Drupal sites with point and click in the admin UI without writing much or any custom code. I say much, because they might implement the odd hook here and there. But most of the functionality of the site/application comes from configuring Drupal core and contributed modules. Site Builders will have experience with a wide range of contributed modules and will understand how they work together to solve a particular problem. They will understand the limitations of modules and should be able to provide a concise argument of the pros and cons of various solutions. Site Builders will build the content types, taxonomy, navigation, image presets, rules etc. One of the magnificent things about Drupal is that it does not exclude non-developers. The Drupal community and the platform provides a very powerful tool set for people to build innovative and complex sites without the requirement to be a programmer. And mastering this is a very valuable skill in itself.

Drupal Themer / Frontend developer

A Drupal Themer is the specialist front end developer. They are experts in HTML, CSS and Javascript. They are also experts in the Drupal theme layer. They should be able to take a design and turn it into the working theme. Ideally they will be well versed in implementing responsive design.

Drupal Module Developer / Backend developer

A Drupal developer is someone who writes a lot of PHP and other server side languages. They write custom modules, automated tests, consume web services, automate deployment etc. They may also be know as “backend Drupal developers”. They may also get involved in some of the more advanced side of the theme layer as well. Often they will set up automated deployment.

A note on contributing and collaboration

Drupal is inherently a collaborative project. Site Builders, Module Developers and Themers will often contribute their work back to the community and collaborate with others. It is common for module developers to share and collaborate on contributed modules, themers on contributed themes and site builders on site building recipes and other forms of documentation.

The Three Disciplines

What is a drupal developer?

Drupal Generalist /Jack of All trades

It is very common to do all three. You may be more advanced in one area or another, but still act in a general capacity.

In most of the projects I have worked on, there has not been a dedicated site builder. Both backend developers and fronted developers will do elements of site building.

A Drupal developer who is a jack of all trades

How a business hire and use Drupal people varies. In one extreme, a business may hire specialists. To deliver a particular piece of functionality or feature, the work may have to go through each of the specialities before it is done. One of the other extremes, is a business may assume that a Drupal person should do all three specialities. In this case, it is common for the team to be a team of “Drupal Developers” and the role encompasses site building, backend development and theming. Both approaches have their pros and cons, but that is for another day!

Other roles

Just like any other web development setup, there is a range of other roles included in the process of building and support Drupal applications. This includes:

Sysadmin / Devops - run the live stack, will often deploy Drupal sites to the live environment, deal with performance issues, setup a CDN, Varnish, Memcache etc.

QA - test all changes to ensure quality and that they meet the requirements. Setup automated tests.

Project Manager / Scrum Master - run the scrum team, remove impediments to progress, ensure delivery of the project on time and budget.

Product owner - comes up with the requirements. Works closely with the project manager to prioritise the backlog. Normally has final sign off of all changes.

Design / UX - comes up with the design and user experience. They might build prototypes that can then be converted into a Drupal theme.

Who is not a Drupal developer

This advice should be given to any recruiter or hirer in the Drupal space. A PHP developer is not necessarily a Drupal developer. It does not matter how good the PHP developer is at PHP, unless they have decent Drupal experience and understand its API’s and, dare I say it, “the Drupal way”, they will make many mistakes. You will then need to hire an experienced Drupal developer to clean up the mess left behind. Of course, I am being a bit simplistic here. In current times where demand far out strips supply, it as not as simple as that. You might be forced to hire a PHP developer who doesn’t have much (or any) Drupal experience. That is fine, but the minimum requirement should be that the developer wants to learn the Drupal standards, understand the Drupal culture and try and do it the Drupal way. You don’t want someone who treats it as just another framework that can be bent and broken to the developer's way of thinking. That is a recipe for creating a maintenance nightmare that will cost the business in the long run.

Where do you go from here?

When I first launched the early edition of my book, Master Drupal Module Development, I asked people on my mailing list whether they were Site Builders, Backend Developers or Themers. The split was mostly between Site Builders and Backend Developers but most of the developers were people who developed in tech outside of Drupal and were looking to find the best way to learn Drupal development.

Developer from another technology

If you are a developer from another technology, there are a couple of routes available. One is to build a couple of sites using pure site building. And then, once you are happy with the basics of Drupal, you could dive into module development. Being an experienced developer will almost certainly give you a leg up, because you obviously already know how to do development. Now you need to learn the Drupal APIs and the Drupal way.

Site Builder

If you are a site builder, you might want to move into module/backend development. Because you already know your way around Drupal, you also have a leg up. However, you still need to learn the Drupal APIs and the Drupal way of developing. In addition, if you don't have any or much programming experience, you will need to learn some of that to. You don't need to be a programming genius to get started with Drupal development. You can get started with just the basics of PHP in addition to something like my book.

Themer

If you are a themer, you might not want to do anything else but get better at theming and frontend development. After all, there is a ton to master there, especially when you include CSS and Javascript. But you might want to learn some module development. It will certainly help you broaden your horizons and achieve more.

Module Developer

If you are a Module/Backend Developer, then where do you go? You can always learn more about theming. But might also consider learning more about being a solutions architect. If you are happy being a backend developer, then getting more experience in other PHP frameworks (such as Laravel or Symfony2) is very beneficial to your career prospects and capabilities as a developer.

Where do you fit in?

Do you consider yourself a Site Builder, Themer, Module Developer or something else? Are you looking to move into another area? If so, what are your main obstacles? I'd love to hear from you in the comments below.

Dec 18 2014
Dec 18

Got some Drupal 7 modules that use the Form API lying around? Want to learn how to port them to Drupal 8? The process could just be the crash course you've been looking for to learn Drupal 8 object-oriented module development and Drupal 8's Form API.

Back in the day, Lullabot trainers produced a module to demonstrate how to use Drupal's Form API. The module was called Form Fun and today, I'm going to tell you a little bit about the "fun" I had porting the Drupal 7 version of this module to Drupal 8.

In this blog post, you'll learn:

  • How to use Drupal Module Upgrader to upgrade a D7 module
  • How hook_menu was replaced with routes and controllers
  • How to find information about API changes in Drupal 8

To get a jumpstart on porting this Drupal 7 module to Drupal 8, I decided to give the Drupal Module Upgrader module a try. I saw a demonstration of it by Angie Byron at the Pacific Northwest Drupal Summit in October and I have been determined to try it out ever since.

Getting Ready to Port

To start, I created a Drupal 8 site on my local machine. At the time, this was Drupal 8, Beta 3. For information about how to install Drupal 8, read this Installation Guide.

I also have Drush 7 installed. You will need this, too, if you want to use the Drupal Module Upgrader. In Terminal, or your command-line interface of choice, type drush --version to see which version of Drush you have installed. If it returns 7.0-dev, you can move on. (Likely any 7.x version will work.) But if you find you need to upgrade Drush, check out my blog post on Upgrading Drush to Work with Drupal 8 for some tips.

Both Drush 7 and Drupal Module Upgrader require Composer. If you are brand new to Composer and would like to learn about what the heck it is, check out this free video on Drupalize.Me from our partners at KnpUniversity: The Wonderful World of Composer. If you don't remember if you've installed Composer already, if you're on a Mac, fire up Terminal and type which composer. If a path is returned, then you're all set. If you need to get Composer, see this doc page.

Drupal Module Upgrader

I recommend installing the latest dev version of Drupal Module Upgrader (DMU). I ran into some problems just installing the stable version which were fixed in the latest dev. So, save yourself some troubleshooting and go ahead and install the latest dev using drush by running this command in your command-line interface (CLI): drush dl drupalmoduleupgrader --select and select [1]. For this tutorial, I downloaded the 8.x-1.x-dev | 2014-Nov-27 | Dev version.)

Follow the instructions in Drupal Module Upgrader's README.txt file. If you run into errors with any of the dependencies that Composer is updating, like Pharborist, try running composer update in the /modules/drupalmoduleupgrader directory. For other troubleshooting help, search the Drupal Module Upgrader Issue Queue.

Now we're almost ready to port this thing, but first we need to copy the D7 module into the modules/ directory.

Back in your CLI, with the DMU module installed, we now have some new Drush commands. Run drush help --filter=drupalmoduleupgrader to see the shiny new commands provided by DMU module.

Analyzing the API Changes

Now run drush dmu-analyze. This command creates an HTML file in your module's directory with a breakdown of every API change that affects your module with links to the change records. (Note: I discovered that this page renders much better in Chrome than in Firefox, on a Mac anyway, so take that for what it's worth.) Right away, I can see that form_set_error() is now a method of FormStateInterface. but there are many more changes listed, with links to the change records containing more information about the API change.

form_set_error() is now a method of FormStateInterface.

Running the Upgrade

After perusing the changes that will need to be made, we're about ready to run the upgrade process. But first, make sure you've got a backup of your module before proceeding.

My module's machine name is form_fun. To have DMU attempt to upgrade it to D7, I run drush dmu-upgrade form_fun (replace "form_fun" with the machine name of your module, if you're playing along).

In the case of my module, it was a bit anti-climatic. But, navigating to modules/form_fun, I can see a bunch of new files! I'm excited! (Or maybe I should be afraid!?)

More Than Just Form API Changes

Even though my module was all about demonstrating Drupal 7's Form API, it used other Drupal 7 APIs and function calls to create menu links, URLs, and page content. This means that in this process of porting this D7 module to D8 using Drupal Module Upgrader, I'm also learning about how Drupal 8 replaced hook_menu and other functions. I'm also getting a crash course on object-oriented PHP and using controllers, classes, and methods instead of as a bunch of procedural functions inside .inc files.

Hook_Menu's New Split Personality

hook_menu has been removed from Drupal 8

hook_menu has been removed from Drupal and in its place, routes and controllers. A route defines what happens at a certain URL and a controller determines what content or functionality should be found at a certain URL.

So, for purposes of comparison, here's a snippet of the D7 version of Form Fun's hook_menu implementation that defines pages, their paths, access, files, functions or "page callbacks", related pages, and in what order those sub-pages should be listed in a menu:

D7 example usage of hook_menu

/**
 * A super-simple menu hook that tells Drupal about our Fun Module's page.
 */
function form_fun_menu() {

  $items['form_fun'] = array(
    'title' => 'Fun with FormAPI',
    'page callback' => 'form_fun_page',
    'access arguments' => array('access content'),
  );

  $items['form_fun/cake'] = array(
    'title' => 'Death, or cake?',
    'page callback' => 'form_fun_cake_page',
    'access arguments' => array('access content'),
    'file' => 'form_fun.cake.inc',
    'weight' => 1,
  );

  $items['form_fun/existential'] = array(
    'title' => 'Existential questions',
    'page callback' => 'drupal_get_form',
    'page arguments' => array('form_fun_existential'),
    'access arguments' => array('access content'),
    'file' => 'form_fun.existential.inc',
    'weight' => 2,
  );

...

  /**
   * These menu callbacks should be ignored! They're secret. Suuuuuuper secret.
   */

  $items['form_fun/death_image'] = array(
    'page callback' => 'form_fun_death_image',
    'access callback' => TRUE,
    'file' => 'form_fun.cake.inc',
    'type' => MENU_CALLBACK,
  );

...

  return $items;
}

In the Drupal 8 version of Form Fun, created by DMU, the functionality in the D7 Form Fun hook_menu has now been placed in the following three files:

  • modules/form_fun/form_fun.routing.yml
  • modules/form_fun/form_fun.links.menu.yml
  • modules/form_fun/src/Controller/DefaultController.php

So now if you wanted to tell Drupal to do something at a certain URL, like build a page, display a form, or create some menu links programmatically, you will need a YAML file (perhaps more than one) and potentially a Controller php file. In the case of form_fun, DMU created src/DefaultController.php to output content for one of the pages in the module, created form_fun.routing.yml to tell Drupal about the names of the "routes" declared in this module, and form_fun.links.menu.yml to tell Drupal about how a menu of links should be built for this module.

Defining Routes with YAML

The form_fun.routing.yml file is a YAML file that describes route names, like form_fun.page. It describes any necessary related information like the path, title, what type of page it is, and where to find the method that builds whatever it is. That could be a controller definining page content, or a class handing a form. Think of the structure of a YAML file as an array, but with colons instead of commas and meaningful indentations.

Here's a snippet of the form_fun.routing.yml file created by DMU in the upgrade process:

form_fun.routing.yml (snippet)

form_fun.page:
  path: /form_fun
  defaults:
    _title: 'Fun with FormAPI'
    _content: '\Drupal\form_fun\Controller\DefaultController::form_fun_page'
  requirements:
    _permission: 'access content'
form_fun.cake_page:
  path: /form_fun/cake
  defaults:
    _title: 'Death, or cake?'
    _content: '\Drupal\form_fun\Controller\DefaultController::form_fun_cake_page'
  requirements:
    _permission: 'access content'
form_fun.existential:
  path: /form_fun/existential
  defaults:
    _title: 'Existential questions'
    _form: \Drupal\form_fun\Form\FormFunExistential
  requirements:
    _permission: 'access content'
 

Note that in Beta 4 of Drupal, you won't have to specify that a route uses a _content controller. You will just need to specify _controller. (See this change record: Routes use _controller instead of _content.)

As you can see, if a route is a form, you specify a _form key with the value being the name of the class that defines your form and its workflow.

Defining Menu Links for the Module

There's now a separate place for defining menu links in your modue. In my Form Fun module, these are contained in the new form_fun.links.menu.yml file created by DMU. Notice the parent and weight keys that create a hierarchy and order menu links.

form_fun.links.menu.yml (snippet)

form_fun.page:
  route_name: form_fun.page
  title: 'Fun with FormAPI'
form_fun.cake_page:
  route_name: form_fun.cake_page
  title: 'Death, or cake?'
  weight: 1
  parent: form_fun.page
form_fun.existential:
  route_name: form_fun.existential
  title: 'Existential questions'
  weight: 2
  parent: form_fun.page

Controllers

Finally, the Controller. A Controller is the place to put methods that output some content to the page. It replaces the page callback procedural function. In the case of the Form Fun module, it's the page that defines a method that returns a list of links to the other forms. It's also the home of several methods that output some images to certain pages. These pages will serve as redirect locations in one of Form Fun's forms. We'll even output a form to a page in a Controller, replacing Drupal 7's drupal_get_form procedural function. I found this handbook page a helpful resource in deciphering the changes that DMU made in replacing hook_menu: An introductory example to Drupal 8 routes and controllers.

src/Controller/DefaultController.php (snippet)


/**
 * @file
 * Contains \Drupal\form_fun\Controller\DefaultController.
 */

namespace Drupal\form_fun\Controller;

use Drupal\Core\Controller\ControllerBase;
use Drupal\Core\Url;

/**
 * Default controller for the form_fun module.
 */
class DefaultController extends ControllerBase {


  public function form_fun_page() {
  /* Returns a render array that produces an HTML list of links. */

}

  public function form_fun_cake_page() {
    $form = \Drupal::formBuilder()->getForm('Drupal\form_fun\Form\FormFunCake');
    return $form;
  }
...

So, wow! The changes to hook_menu are really significant. I found it really helpful to learn about those changes in a super-practical way but going through this process of having Drupal Module Upgrader port my D7 module to D8.

@FIXME: Where DMU Only Gets You So Far

Now, DMU did a pretty good job of creating these files and refactoring functions into classes and such, but where it failed, it added comments with @FIXME. For example, I had several @FIXMEs related to building a $links array and displaying those links as a list using a theme function. Here's the comment left by DMU:


// @FIXME
// l() expects a Url object, created from a route name or external URI.
// $links[] = l(t('Death, or cake? (The basics)'), 'form_fun/cake');

Ok. So what to do? What I found the most helpful in this case was to head over to the change records on Drupal.org. There were some direct links to change records provided by the upgrade-info.html file generated by drush dmu-analyze. But where I didn't find a reference there, I simply searched for the function name in question.

So, for this @FIXME, on the change records page I filtered by keywords l() and found this change record: l() and url() are removed in favor of a routing based URL generation API. Usefully, on this change record, there was an example for how to create a Url object. So...

$links[] = l(t('Death, or cake? (The basics)'), 'form_fun/cake');

...is refactored to...


  // Death or Cake? (The basics)
  $url = Url::fromRoute('form_fun.cake_page');
  $text = 'Death, or cake? (The basics)';
  $links[] = \Drupal::l(t($text), $url);

The other thing that was refactored by DMU was the theme function that produced an HTML list of links contained in the $links array, but I found this caused many errors and obviously needed to be changed. I finally found an example of an item_list theme function and refactored DMU's output: return _theme('item_list', array('items' => $links)); to this:


  // Preparing a render array of a HTML item list of the $links array.
  // @FIXME
  // #title value outputs twice on page, as H1 and H3. Fix in twig file.

  $item_list = array(
    '#theme' => 'item_list',
    '#items' => $links,
    '#title' => t('Fun with FormAPI!'),
  );

  // Returning the render array that produces an HTML list of links.
  return $item_list;

You'll notice that I left a @FIXME for myself to fix the duplicated title output, but the takeaway is that Controllers expect render arrays, response objects, or html fragments—not strings. At least that's what the error messages in the "Recent log messages" insisted upon. Also, I discovered that _theme used to be theme() and shouldn't be called directly, so as to encourage the consistent building of renderable arrays. As this is a custom module, I thought I had better refactor. See theme() renamed to _theme() and should not be called directly.

Lessons Learned and What's Next

When I started this process of porting the D7 Form Fun module to D8, I naïvely thought that I would primarily be learning about changes to Drupal's Form API. But what I discovered was equally valuable: lessons on object-oriented PHP, how a major function like hook_menu was replaced with YAML files and Controller php files, new types of files and structure in modules, an invaluable resource in change records, and a great helper in Drupal Module Upgrader.

In my next post on upgrading the Form Fun module to Drupal 8, we'll explore interfaces, a bit more with a Controller, and we'll define a class that defines, builds, validates, and submits a form. Throughout this series on the Drupal 8 Form API, we'll dive into various types of form handling including multi-step and AJAX forms. Stay tuned!

Resources

Dec 18 2014
Dec 18

The Drupal Rules Filter Module is a simple module that makes it easy to sort through a long list of Drupal Rules. This is a module that is especially useful on those larger scale Drupal websites that rely heavily on the rules module and have many contributed Drupal modules installed.

It categorizes and adds a search box to the Rules UI page that makes instant search possible. It's very similar to the Module Filter module. The Module Filter module makes searching Drupal modules easy, the Rules Filter module makes searching Drupal rules easy.

There is not much configuration needed for the Drupal 7 Rules Filter module. Just make sure you download the List.js plugin to your sites/all/libraries folder (placing the list.min.js file in a listjs folder).

What do you think of the Drupal 7 Rules Filter module? Do you think it's a module you would use when building out a bigger Drupal 7 website?

Dec 18 2014
Dec 18

Kate Marshalkina, Drupal ExpertThis post is part of an ongoing series detailing the new personas that have been drawn up as part of our Drupal.org user research.

Kate Marshalkina has been using Drupal for three and a half years. A web developer by trade, Kate was approached by a friend who wanted her to do Drupal work with him. After doing some research on the system, Kate agreed.

“It’s quite difficult to learn Drupal without paid work because it requires a lot of time and experience to learn the Drupal way of doing things,” Kate said. “I had joined a security startup, and a security company obviously cares about security on the web. So we decided to use Drupal because it’s a safe, well known open source system. I learned a lot while I was working on my tasks, but I spent a lot of my free time to learn Drupal. Once I started learning, I couldn’t stop— I’d previously worked with other content management with less documentation and information and then I started learning Drupal and... because of the community, and all of the learning resources and videos that are available, I was hooked."

“After working with Drupal for three months, I started my blog and not long after that I presented a session at DrupalCamp Moscow. Now, I’m a Drupal lover after three and a half years working with the platform."

Drupal.org: A Valuable Resource

Every day, Kate checks in to Drupal.org: she says she visits the site to find new modules, check the issue queues, and check API documentation. “I’m very comfortable with Drupal.org, but it was hard getting used to it when I started. Initially, it was a question for me why I should even use Drupal.org, and I didn’t know what the benefits are.

"I really like my dashboard on Drupal.org,” said Kate. “It’s a great page where I can see daily updates on my issues — and of course I follow a lot. It’s nice that I can also easily view updates on issues in critical bugs in core and so on, see crucial updates, core releases, and of course I also follow the Drupal Planet RSS feed."

Drupal Planet is one of the most helpful tools for Kate when it comes to getting new Drupal knowledge, and she often encourages her colleagues to follow it. "I think Drupal Planet is an exciting part of Drupal.org. It’s a great resource for Drupal related articles for everyone; beginner to expert, frontend to backend to sysadmin, the information for all these people is usually very high quality on Drupal Planet. When I’m working with fellow developers who have questions, I always ask them to look on Drupal Planet because I know that the information there is of a high quality, and that anyone can find the knowledge they need in there."

It's About the People

Some of the recent changes made on Drupal.org, including the addition of user pictures to the issue queue, have made Kate’s Drupal experience vastly better.

“[The pictures] are great because it makes Drupal.org more personalized, and you can more easily remember the people you talked to because of their photos. And, it reminds people that Drupal isn’t just a CMS, it’s a community, and the people are important.”

“It’s a big question for me how to enroll younger developers,” said Kate. “Looking at the contribution opportunities, [new people] may feel like they can’t be a contributor. So sometimes, they may encounter a bug they don’t know how to fix and think, “oh no, a bug!” instead of recognizing it as an opportunity to learn and grow. If we can encourage more people to become contributors, they will benefit from it and Drupal will benefit from them."

Kate’s advice for new Drupalers is to “start right out and register on Drupal.org. Share modules, create patches, learn how to use git and so on… it’s not easy, but it’s worth it."

Growing With the Project

As for herself, Kate hopes to increase her skill level by contributing to Drupal 8 core.

"I participated in DrupalCon Amsterdam, and really liked what Dries said about getting more benefits to small companies who contribute so that it will be easier for employers to understand why they spent their time and pay for developers on core. I would be much more experienced if I could participate in Drupal core development."

"I also want to someday give a session at DrupalCon,” Kate added. "I give a lot of sessions in my local community, camps and so on. I’ll be speaking at Moscow Drupal Camp in November, but hope to speak at a DrupalCon soon."

We all wish you the best of luck, Kate, and hope to see you on a stage at DrupalCon soon!

Dec 18 2014
Dec 18

Intro

Solr is an open source search engine with many powerful features.

I encounter a lot of experienced developers that either don’t know how to use it or avoid it as it’s in the to hard/scary basket. In this introductory article we’ll talk briefly about how to install Solr in order to use it for development.

There are many ways to install and configure Solr but in this article I’ll show you how to set it up quickly so you can get started developing with it. While the installation and setup will be generic to any framework you want to develop with, I’ll also, show you a couple of extra steps if you’re using Drupal.

Requirements

The only real hard requirement/prerequisite for running Solr is Java. Version 1.6 is recommended for Solr version 4 and upward. Ubuntu and Mac should come with Java pre-installed.

I’m not a windows guy so sorry you guys are on your own There’s pleant of resources out there.

You can find out the version of Java you are running with the following command.

$ java -version

java version "1.7.0_72"
Java(TM) SE Runtime Environment (build 1.7.0_72-b14)
Java HotSpot(TM) 64-Bit Server VM (build 24.72-b04, mixed mode)

I am using Ubuntu 14.04 but the instructions in this article should work if you’re on a Mac or another variant of *nix

Download

Open a terminal and make a folder we can use for working in, and change to that directory:

$ mkdir solr
$ cd solr

Navigate to the Solr download page and find the closest mirror for you. At the time of this article the latest version of Solr is 4.10.2.

Copy the download link for either the solr-VERSION.tar.gz or the solr-VERSION.zip. You don’t want the solr-VERSION-src.tgz (this is the source code and will need to be compiled) and download it with wget.

$ wget -c http://mirrors.sonic.net/apache/lucene/solr/4.10.2/solr-4.10.2.tgz

Unpack

Once downloaded (it’ll be about 150M) we can un-compress it and change into the directory.

$ tar -zxvf solr-4.10.2.tgz
$ cd solr-4.10.2/

Make a working copy

In the current directory there is a folder called example we want to make a copy of this folder.

We could just use the example folder but it’s nice to leave that clean on case you want to use this copy of Solr for other sites as well. So we’ll make a copy and then change directory into the newly created copy.

$ cp -prv example my_solr
$ cd my_solr

Make it work

Now we’re ready to run it for the first time. To run Solr it’s really simple. Simply run:

$ java -jar start.jar

You should see a whole bunch of output (to stop solr press CTRL^C). After a few seconds if you open your browser and navigate to http://0.0.0.0:8983/solr/ you should see something similar to the following (the actual screen may differ depending on your version)

Solr dashboard

That’s it. Solr is now set up and ready to use. Depending on your client frame work you may need to makes some config changes to Solr itself. Consult the installation instructions of your chosen framework. If you’re using Drupal keep reading and I’ll show you the steps required to make Solr ready for Drupal integration. First lets stop SOlr from running by hitting CTRL^C in your terminal.

Modules

There are a couple of modules you can use for Drupal integration with Solr. I wont go into the Drupal configuration side of things (I’ll leave that for another day) but will talk about the steps required to get the Solr server we’ve set up ready for Drupal usage depending on the Solr module you’ve chosen.

Search API and ApacheSolr

If you’re using the search_api you will need to ensure you have the search_api_solr module installed. Otherwise the [apachesolr] module is the way to go.

In both the search_api_solr and [apachesolr] modules, you’ll find a folder called solr-conf in this folder there will be version folders 4.x, 3.x etc. Choose the version of Solr you downloaded. This folder contains all the config files you need to install in your Solr install. I could probably write a whole bunch of articles about the contents of these files but since this is a beginner tutorial we’ll just take the easiest route.

You want to copy the contents of the solr-conf/4.x/ folder into your solder core. We can do this with the following, go back to your terminal, and run (change the path to your Drupal module):

$ cp -v **/path/to/apachesolr/or/search_api_solr/**solr-conf/4.x/* solr/collection1/conf/

That will copy the config for your Drupal site into the my_solr/solr/collection1/conf/ directory.

Conclusion

Solr is now ready for use by your Drupal install. You can run it whenever you like by changing into the my_solr directory and starting it.

$ java -jar start.jar

I wouldn’t recommend using this setup in production. However, for developing on your local machine, it’s perfectly fine.

In the next article, I’ll talk about how to configure the search_api and search_api_solr to use Solr as a search engine for your Drupal site.

Please enable JavaScript to view the comments powered by Disqus.

Dec 17 2014
Dec 17

Nathaniel Catchpole , the Drupal 8 release maintainer and Tag1 Senior Performance Engineer, suggested that Drupal shops everywhere could support the release of Drupal 8 by upgrading their own sites and addressing the issues they find along the way. This series chronicles the journey from Drupal 6 to Drupal 8.

Part 1: Readiness Assessment

Before performing a major version upgrade, I usually go through a series of steps that help me determine:

  1. What compelling business needs or new functionality are prompting the move? In this case, the site builder view on the status of D8 will help us identify and hopefully address real barriers to adoption, as well as to prepare materials that help document the journey.
  2. What does the site do? If there are functional tests or test plans, I review those. Then, I manually review the site, paying attention to functionality for particular roles. I also check with at least one or two of the stakeholders with each role to learn what features they regularly use, what improvements would better support them, and what they’d really like to see. I don’t normally act on that wish list answer as part of the upgrade proper, but knowing what new functionality they desire can help make decisions about the new modules to use.
  3. Are there features that are no longer in use? I’m always surprised, though by now I really shouldn’t be, by the number of things that seemed like a good idea at the time to someone which were never used at all or which are no longer in use.
  4. Is now the right time? Are the contributed modules on which the site depends ready or is there a new way to achieve the required functionality.

1- Compelling Functionality

As noted above, this particular journey is compelled by the desire to identify and fix bugs during the Beta period of Drupal 8. That doesn’t mean we’re not excited by:

  • WYSIWYG in Core- I am ecstatic at the prospect of NOT having to choose and configure basic WYSIWYG functionality.
  • Views in Core - I pay a lot of attention to the experience of content contributors. They’re the everyday users that Dries talked about back at DrupalCon San Francisco, people who use the interface of Drupal as their day job. What a site builder does for them (or fails to do) can make or break the success of the site as well as cement the organizational satisfaction with Drupal itself.

I want Tag1’s busy staff to find it a pleasure to keep their blogs, community contributions, and case studies up-to-date.

2 - Current Functionality

Our speciality is performance and scalability for complex websites, which never necessitated building a complex external site for ourselves. We have internal sites that put Drupal 6 though its paces, but the main Tag1 site is a combination of blogs and brochure:

  • Review what the current site does
    I start any readiness evaluation looking at the site from the user perspective, clicking around, then reviewing any documentation. The site revealed no surprises. I jotted notes about those user-facing features, mapped to Drupal functionality, and stored them as a sheet in the Readiness spreadsheet. For a basic site, the notes aren’t too impressive, but the process was a valuable refresher on what the site does.
  • Inventory the enabled modules
    After looking at the site from the user interface, I make an inventory of enabled modules. The following Drush commands make nice spreadsheets:
    drush pm-list --no-core
    drush pm-list --core
    

    You can take a look on Sheets 2 and 3 of the Readiness spreadsheet to see the entirety of what was and and wasn’t available.

    In our case there are just two areas of concern:

    Blogs have been removed from core, but with views moved in, this won’t be daunting to rebuild. They may work a little differently, but odds are that will be for the better!

    The only missing module that had no alternative and looked like it might hurt was Pathauto. It’s nice to keep URLs tidy and hard to remember to set them, but wouldn’t be strictly necessary for our purposes and there were were no other glaring show stoppers.

    Note: It looks like upgrade work is happening on Github so I updated the spreadsheet to make note and will evaluate the status as part of the process.

  • Look for customizations
    In addition to custom and contributed modules, it’s good to check for inappropriate customizations done directly to core or contrib. Whatever reason they had for being done back then may need to be accounted for, hopefully in a best-practices way, on the new site.

    Although it seemed unlikely that any relevant changes had been done, for good measure, I always run Hacked. It’ll let you know about files that have been changed from the source.

    There were some theme customizations and a .gitignore file was missing, so as suspected, nothing to worry about.

3 - Features no longer in use

I’ve learned over the years that not every module that is enabled is in use! I try to find a link that illustrates somewhere on the site that the module is visibly in play. Even when there is a place in production where the module is used, it’s entirely possible that the production feature itself doesn’t make sense to upgrade. A serious investigation of analytics and logs can be insightful. Data can guide conversations with product owners about what’s still valuable and support decision-making.

On our straightforward site, there’s not much to not use. As I learned running the first migration, though, even features that were briefly explored but never used can come into play. More on this later.

4 - The right time

Once you’ve completed your archaeology, you’ll need to find the right time for your organization. Talk with stakeholders about the change and be on the lookout for in-process or upcoming feature requests that you might not be prepared to accommodate.

In our case, because the motivation for this upgrade is process, not product, we’ll proceed and go as far as we reasonably can with the upgrade. At the time of this writing, I’ve done a D8 install and begun running a migration. (That’s how I know to be on the lookout for certain disabled modules!) I’m feeling confident that we can get a long way.

Unless the migration process is flawless and the Drupal 8 to Drupal 8 update path is in place I'd be unlikely to recommend deployment of the work, especially with an impending re-design. While re-entering some data manually has always been a part of previous major version upgrades, I’d be loathe to commit to beta adoption if I thought we’d find ourselves stranded in the latest version. Fortunately, all the preparatory work can be done without the upgrade path in place!

Coming next in this blog series, Part 2: Preparing to Migrate.

What modules and processes do you use to prepare for a major version upgrade?

jam
Dec 17 2014
Dec 17

Part 1 – Larry Garfield and I had a long chat in front of my camera at DrupalCon Amsterdam. Part of the idea was to help Larry prepare his thoughts for writing the blog post that has turned into "Building Bridges: Linking Islands" in the Future of PHP guest blog series on Acquia.com. In this part we cover Larry's start in Drupal, some project history, what Drupal can teach (and needs to learn) about contribution and community ("celebrating the newbie" and the power of "thank you"), The New PHP, fuzzy project boundaries and inter-project collaboration.

Building Bridges

Larry explains, "The PHP ecosystem is coming together. The web ecosystem is coming together. It's a lot more integrated. It's a lot more collaborative. So projects that are collaborating need to ask themselves 'What is our value-add?'. Drupal's value-add over the PHP baseline is not that it does dependency-injection. It's not that it can serve HTML. It's not that it has forms. It's value-add is entities, views, and the community. We are a content management system and a very good one. By that I mean a structured system to manage content, not a tool for building pages. And we have a community behind it that has your back, that you can rely on, that is not going away any time soon. That's Drupal's value-add. So put our emphasis and effort there; on building a really solid, flexible CMS platform with a community that can support it."

"Those other things that get us there? That's not our value-add. People don't use Drupal for hooks. They use Drupal for nodes. They use Drupal for Views. The more we can outsource our irrelevancies and focus on core competencies, the more we can say, 'The important things that make Drupal worth using, let's focus on those.' And the things that are not the reason people come to Drupal, we can save time by outsourcing that. It may not be perfect ... Could we write something better [than the Symfony routing component] ourselves? Probably. Would it be done right now? Not even close. But we brought in all this code and got 3/4 of the way we wanted to be by adding one, single library."

I proposed one more aspect to Drupal's value-add, "Drupal is architected in a way that is extensible and flexible. You get some more baggage, but its advantage over a specialized system is when your next requirement comes in, we're ready to incorporate it or communicate with it." Larry adds, "That goes to the whole framework versus application debate we've been having since there's been a Drupal. In Drupal 8, in some ways, we became more of a framework, in other ways, more of an application." Pointing to himself, "As one of the framework people for a long time, I think the [framework side] actually lost that battle and we lost it to modern PHP."

"Meanwhile, the application has evolved to being a platform and we should focus on thinking of Drupal as a PHP-powered content management platform. And think of it not in terms of, 'What is the canned user experience we can offer?' But [rather] how can we make a toolchain that let's you build a great user experience. How can we build a toolchain that let's you do all the content modeling shenanigans for whatever site or task you're doing and have reasonable defaults. Look at it as not an application, but as the core of a platform ecosystem. That means designing things and planning things in different ways than you would a pure application. You need to think about extensibility in a a different way between a framework and an application and a platform. I think at this point platform is the right way to think at the high level ... with some framework stuff alongside it."

Links/References

Interview video

[embedded content]

Image credit

linking_islands_1.jpg - Image by Jay-P - https://www.flickr.com/photos/esqenzo/248879957/ License https://creativecommons.org/licenses/by-nd/2.0/

Dec 17 2014
Dec 17

How do you insert a text field into a rendered entity, all within a field collection? This question came to mind on a recent Drupal 7 build. What at first sounded complicated turned into a concise and elegant solution.

The basis of the requirement broke down into the need for events to list those participating in the event. The build already had existing Event and Profile content types, so the new functionality would only require implementing the list of Profiles participating in an event. Each participant would include their role at the event, along with links to contact them.

blog

To implement this, an Event Participant field collection was added as a field to the Event content type. Each instance of the field collection included a single Profile content type and a single text field for their role at the Event. The Profile would then be displayed as a rendered entity within the field collection so that the Profile’s image and contact links could be displayed. The point at issue then arose. How would the Participant’s role at the event be placed within the rendered Profile teaser, along with the other Profile fields? To the user it needed to look like a single unified entity, with the Profile’s image, role at the Event, Profile name, and contact links all included.

Screen Shot 2014-12-11 at 4.27.43 PM.png

After some initial brainstorming a handful of different approaches came to mind. The initial impression was to use template_preprocess_field() since it allows the alteration of values in the theme layer. The participant role field would be removed from it’s initial location and inserted into the participant profile object. This did not work though. Despite the repositioning of elements within the render array, the changes occur too late in the rendering execution. The render array had already been rendered and thus the display remained the same.

After quickly realizing a theme function wasn’t the answer, hook_node_view() came to mind. The hook allows for the alteration of a node before it is rendered and so all that would be needed would be to shift the location of the Participant Role field within the $node array. The issue with this implementation and the reason it wasn’t chosen, is that it would require reaching out to the global state to determine what changes are needed. In other words hook_node_view() doesn’t realize what context it is called under and would thus require a hack of sorts to determine when it was being called under an Event node. This could be worked around by checking the URL path, but this is not best practice as it is brittle and prone to breaking with future site updates.

As they say the third time’s the charm. The final solution was to use hook_field_collection_item_view(). This hook allows for the alteration of a field collection item before rendering. Unlike many of the other options, this hook is called for each field collection item, allowing one to both check the field collection name and adjust it’s internal field ordering, before any rendering occurs. Using this hook, each Event Participant field collection item is altered, removing the Role field and inserting it into the Profile rendered entity. Simple as that. Now each Event Participant is correctly displayed as a single unified entity, with an image, name, and a set of contact links.

function hook_event_field_collection_item_view($field_collection_item, $view_mode, $langcode) { if ($field_collection_item->field_name == "field_event_participants") { $role = $field_collection_item->content['field_event_participants_role']; $profile_id = $field_collection_item->field_event_participants_profile[LANGUAGE_NONE][0]['target_id']; // Remove Participant Role from previous location in field collection output unset($field_collection_item->content['field_event_participants_role']); // Insert Participant Role into the Participant Profile output $field_collection_item->content['field_event_participants_profile'][0]['node'][$profile_id]['field_event_participants_role'] = $role; } } 1 2 3 4 5 6 7 8 9 10 11 12 functionhook_event_field_collection_item_view($field_collection_item,$view_mode,$langcode){  if($field_collection_item->field_name=="field_event_participants"){    $role=$field_collection_item->content['field_event_participants_role'];    $profile_id=$field_collection_item->field_event_participants_profile[LANGUAGE_NONE][0]['target_id'];       // Remove Participant Role from previous location in field collection output    unset($field_collection_item->content['field_event_participants_role']);       // Insert Participant Role into the Participant Profile output    $field_collection_item->content['field_event_participants_profile'][0]['node'][$profile_id]['field_event_participants_role']=$role;  }}

Through several quick, small, iterations an elegant, straight-forward solution was found to what at first sounded like a complicated feature request. Now users viewing an Event can see a list of Participants, with each Participant displayed as a single item. Check out more on field collections on the Phase2 blog!

Dec 17 2014
Dec 17

Chris Ohmstede, Skilled Drupal UserThis post is part of an ongoing series detailing the new personas that have been drawn up as part of our Drupal.org user research.

Chris Ohmstede is based out of Los Angeles, California. An experienced programmer, Chris is new to Drupal but already identifies as a skilled persona. Several months ago, Chris discovered Drupal as he was looking for solutions to build a website for hosting a program he wrote.

"I spent a number of years in the banking industry, and in that industry banks are constantly making connections to everything. I was always running in to problems when things couldn’t connect— it was always an issue trying to figure out what was actually going on. Over the years, I wrote a bunch of applets here and there to figure out what the problems were, and my program whycanticonnect is a conglomeration of those applets that work across operating systems, mobile— I’ve got cloud services approaching me about it, too.

Wanted: Custom Functionality

“Originally, I was hosting my project for download using a different CMS option, but things weren't working properly. Finally, I hit a point where I said “I’ve got to make a change.” I initially went to Drupal because I use Linux as my OS, and Drupal is one of the only CMSs available out of the repositories. I saw that, went and did some research, and Drupal looked big and well organized. One of the cool things that caught my attention is that I could Google around and see that everything I needed was already built. I never had to ask for any help — I was able to roll on through and put my site together easily."

“Also, one of the reasons why I selected Drupal was that I've released my product in 12 different languages, and Drupal looks like it has some decent translation services coming in. I want my site to be usable in all 12 languages I support."

After downloading Drupal, Chris relied on his technical knowledge to carry him through. Though he initially encountered some difficulty getting the modules to do what he wanted, he found after some searching that deriving his theme would fix his problems.

"Once I derived the theme, everything got easier— the way drupal derives themes is perfect. It’s very object oriented, which is fantastic. The nice thing about Drupal was that I could figure out what I wanted to mess with, and I could build variables on the local scope and carry them through the whole page and process, which is surprisingly difficult to do with other systems."

"It gives me everything I need"

“The great thing about Drupal is that you're not limited in how to inject code,” said Chris. "I like my download module because it gives me everything I need. It figures out intelligently what’s the best download to go with, it builds a link and launches the javalaunch, so it’ll work whether java is enabled or not— which was a problem I’d had before. I’m very happy with it, and very happy with Drupal."

For the time being, Chris doesn’t plan to put together more Drupal sites — rather, he’s going to focus on maintaining the one he has.

"I have no desire to become a web designer,” Chris said. “I built my site because I know I'll need to change it often, and with Drupal it doesn’t cost me anything. As far as giving back to the community, I’m mostly focused on working with my product. I need to write an Android, Mac, and iOS version. I’m planning on submitting some of my modules to Drupal.org once I’ve got them in shape, and I’ve made posts for people asking the same questions and encountering the same problem as I have. Maybe someday down the road I’ll have time to do more, though I certainly wouldn’t make any promises about it."

Dec 17 2014
Dec 17

Global #SprintWeekend will be 17-18 January 2015.

Please encourage contributors you know to host a small local sprint near them.

Post an announcement in your local meetup groups please; ask if anyone is interested in hosting a sprint and wants some support organizing it.

List sprints on the wiki: https://groups.drupal.org/node/447258
That wiki also has more information about organizing a sprint.

We will have a couple meetings to support organizers (doodle for organizer support meetings: http://doodle.com/uutcp2ge7gt3a8ed )
and also have mentors in irc over the sprint days (doodle for mentoring during the sprint: http://doodle.com/5vr54mpvxq4k7x29 ).

Pages