Upgrade Your Drupal Skills

We trained 1,000+ Drupal Developers over the last decade.

See Advanced Courses NAH, I know Enough
Nov 30 2016
Nov 30

Freitag logoOur latest site with Drupal Commerce 1.x went live in July 2016. It is Freitag. Since then we’ve been adding several new commerce related features. I feel it’s time to write a wrap-up. The site has several interesting solutions, this article will focus on commerce.

First a few words about the architecture. platform.sh hosts the site. The stack is Linux + nginx +  MySQL + PHP, the CMS is Drupal 7. Fastly caches http responses for anonymous users and also for authenticated users having no additional role (that is, logged-in customers). Authcache module takes care of lazy-loading the personalized parts (like the user menu and the shopping cart). Freitag has an ERP system to which we connect using the OCI8 PHP library. We write Behat and simpletest tests for QA.

We use the highly flexible Drupal Commerce suite. 23 of the enabled Freitag contrib modules have a name starting with ‘commerce’. We applied around 45 patches on them. Most of the patches are authored by us and 15 of them have already been committed. Even with this commitment to solve everything we could in an open-source way we wrote 30.000+ lines of commerce-related custom code. Still, in March 2016 Freitag was the 3rd largest Drupal customer contributor.

The words ‘product’ and ‘product variation’  I’ll be using throughout the article correspond to ‘product display node’ and ‘product’ in Drupal Commerce lingo.

ERP

ERP is the source of all products and product variations. We import this data into Drupal on a regular basis using Feeds. (Now I would use Migrate instead, it’s better supported and easier to maintain.) ERP also lets Drupal know about order status changes, sends the shipping tracking information and informs Drupal about products sent back to Freitag by customers.

There is data flowing in the opposite direction as well. ERP needs to know about all Drupal orders. Also, we create coupons in Drupal and send them to ERP too for accounting and other reasons.

Emails

We send commerce-related emails using the Message stack. This way we can have order-related tokens in our mails and we can manage and translate them outside the Rules UI. Mandrill takes care of the mail delivery.

Payment gateway

It was a client requirement to use the Swiss Datatrans payment gateway. However, at the time of starting  the project, Commerce Datatrans (the connector module on drupal.org) was in dev state and lacked several features we needed. Pressed for time we opted for buying a Datatrans Drupal module from a company offering this solution. It turned out to be a bad choice. When we discovered that the purchased module still does not cover all our needs and looked at the code we found that it was obfuscated and pretty much impossible to change. Also, the module could be used only on one site instance which made it impossible to use it on our staging sites.

We ended up submitting patches to the Commerce Datatrans module hosted on drupal.org. The module maintainer, Sascha Grossenbacher (the well-known Drupal 8 core contributor) helped us solving several issues and feature requests by reviewing our patches. This process has lead to a stable release of Commerce Datatrans with a dozen of feature improvements and bugfixes.

Additional to Datatrans we use Commerce Custom Offline Payments to enable offline store purchases by store staff and bank transfer payments.

Currencies

The site works with 7 different currencies, some of them having two different prices depending on the shipping country. Prices come from ERP and we store them in a field collection field on the product. We do not use the commerce_price field on the product variation.

Tax

Freitag ships to countries all around the world. VAT calculations are performed for EU, Switzerland, UK, Japan, South Korea and Singapore. To implement this functionality our choice fell on the commerce_vat module. Adding commerce_eu_vat and commerce_ch_vat released us from having to maintain VAT rates for EU and Switzerland ourselves. For the 3 Asian countries we implemented our own hook_commerce_vat_rate_info().

We have two different VAT rates for most of the countries. This is because usually a lower VAT rate applies to books. Drupal imports the appropriate VAT rate from the ERP with the product variation data. This information is handled by price calculation rules in Drupal.

Shipping

Freitag delivers its products using several shipping providers (like UPS, Swiss Post) all around the world. Most shipping providers have many shipping rates depending on the destination country, speed and shipped quantity (weight or volume). On checkout the customer can choose from a list of shipping services. This list needs to be compatible with the order.

We used rules to implement the shipping services in Drupal based on the Commerce Flat Rate module. For this end we trained our client to set up and maintain these rules themselves. It was not easy: shipping rules are daunting even for experienced commerce developers. First we needed to set up the “Profile Address” rules components. Then we configured the “Place of Supply” components. We applied these in turn in the condition part of the shipping rules components themselves.

The weakest point of any  implementation based on Rules is the maintenance. It’s not easy to find a specific rule after you created it. Having 250 rules components for only shipping made this feeling stronger.

The shipping line item receives the VAT rate of the product with the highest VAT rate in the order.

Coupons

Freitag has 6 different coupon types. They differ in who can create them, who and where (online/offline) can redeem them, whether partial redemption is possible, whether it’s a fixed amount or percentage discount and whether Freitag accounting needs to know about them or not.

Based on these criteria we came up with a solution featuring Commerce Coupon. Coupons can be discount coupons or giftcards. Giftcard coupons can only have a fixed value. Discount based coupons can also apply a percentage discount. The main difference between them is that customers can partially redeem giftcards, while discount-based coupons are for one-time use.

To make coupons work with VAT was quite tricky. (To make things simpler we only allowed one coupon per order.) Some coupon types work as money which means that from an accounting point of view they do not actually decrease the order total (and thus the VAT) but work as a payment method. Other coupon types however do decrease the order total (and thus the VAT). At the same time Drupal handles all coupons as line items with a negative price and the Drupal order total does decrease in either case.

The solution we found was to use Commerce proportional VATAxel Rutz maintains this brilliant little module and he does this in a very helpful and responsive manner. All the module does is adding negative VAT price components to coupon line items to account for VAT decrease. It decreases the order total VAT amounts correctly even if we have several different VAT rates inside the order.

Conclusion

Although there’s always room for increasing the complexity of the commerce part of the site (let’s find some use case for recurring payments!), it’s already the most complicated commerce site I’ve worked on.  For this Drupal Commerce provided a solid foundation that is pleasant to work with. In the end, Drupal enabled us to deliver a system that tightly integrates content and commerce.

I also would like to say thanks to Bojan Živanović from Commerce Guys who provided me with valuable insights on the legal aspects of tax calculation.

Aug 17 2016
Aug 17

Last week Drupalaton 2016 took place. With about 150 registrations this was the largest Drupalaton so far. The organizers did an amazing job in coping with this mass. There were two session threads and a sprint room. Of the many interesting presentations I would like to mention Fabian Bircher’s “Configuration Management: theory and practice” (a must for everyone who gets lost while trying to work in a team on a Drupal8 project) , Pieter Frenssen’s “Working with REST APIs”  (it was good to see how simple it is in Drupal8) and “Drupal 8 Media” from Pónya Péter, Szanyi Tamás and Rubén Teijeiro (seems we have a huge step forward in media handling since Drupal7!). I held a session on caching in Drupal 8 which was the shortened version the one I did on Drupal Developer Days in Milan.

Liip was a silver sponsor of the event.

Finally, some pictures on the Friday ship cruise. Thanks to Brainsum for sponsoring it!

Jun 27 2016
Jun 27

DDD is mostly for – surprise! – Drupal developers. This year it took place between 21 and 26 of June in Milan. People were on code sprints all week long and on Thursday, Friday and Saturday there were sessions and workshops as well.

I went to 2 sessions. The keynote of Bojan Živanović was about building reusable php libraries. Bojan is the architect behind Drupal Commerce 2 which is a prominent example of adopting the “leave the Drupal island” principle. They are not only advocating the usage of external solutions in Drupal but also creating libraries that are usable outside Drupal.

The session of Major Zsófi about organizing Drupal events could not have come from a more authentic source. She shared her experience about the practical aspects of building a community and the importance of providing coffee.

All session recordings are or will be available online.

I attended three workshops. A really excellent one by Florian Loretan was about the trending search solution, elasticsearch. Pieter Frenssen had a workshop about Automated testing in Drupal 8. For me this proved to be the most valuable one since I could not keep up with the changes in this field since Drupal 7 and I need it in my contrib work. All my respects to Pieter who was able to present for 3.5 hours in a way that noone fell asleep even though we were just after lunch.

The third workshop I attended was my own 2 hours workshop about Caching in Drupal 8. I learnt a lot about this important topic during preparation and since only around one person left the room it might have been useful for the audience as well.

In the sprint room I joined the Commerce team. The team seemed to have been cursed. A laptop was stolen from the sprint site on Wednesday. Then on Thrusday night Bojan’s MacBook got also stolen from a restaurant with days of uncommitted work. Fortunately, with the effective help of the organizers (which among others included providing the victims a spare laptop and taking them to the police to file a report) they could participate in the sprints only with a minimal amount of delay. As a result we could finish several issues in the Commerce, Commerce Migrate, Token and Address modules.

Sightseeing with drupalists

Sightseeing with drupalists

But the most important part of DDD was the social aspects. I met old friends and got to know new interesting people. Wednesday evening there was a quantitywise challenging dinner for speakers. On other nights we visited several parts of the beautiful city of Milano. Huge thanks to all the organisers, you did an amazing job! Hope to see you next year!

May 11 2016
May 11

tl;dr Keep your site modules up-to-date.

Drupal is famous for its security and it also does not miss a chance to boast about it. However, security does not come automatically, steps need to be taken to ensure it. One of the most important of these steps is keeping site modules and core updated. Failing to do so can lead to incidents like the recent Panama papers incident where an outdated WordPress and Drupal site might have played a role in the data leak.

So what does Drupal do for you to make security easier?

The Drupal Security Team was set up in 2005. It has around 40 security experts from all around the globe who communicate through private channels.

When a security issue is discovered in Drupal (let it be a contributed module, a theme or core itself) an issue is created in the security issue tracker. The issue is visible only for a small group of people (usually the security team, the maintainer of the affected module and the reporter of the issue) to prevent the vulnerability to be exploited before a fix is created. When a fix is ready, the security team issues a public Security Advisory that has informations on the affected module, the security risk level and the solution for the issue (which is usually updating the module).

Security issues are reported almost daily to the security team but some of these are non valid. For example, only modules with a stable release (i.e. non-dev/alpha/beta/rc) are considered by the Security Team. Still, 2015 saw 160 security advisories. The most frequent issues are related to XSS.

Security updates are released on Wednesdays. For core that’s usually the third Wednesday of the month, for contrib it can be any Wednesday. This does not mean that a security release appears on every Wednesday, only that site administrators should look out for them.

In Drupal 8 there are several security improvements. One of them is Twig autoreplacing which drastically decreases the chances for a piece of code to have a XSS vulnerability. Another source of insecurities, the PHP filter module has been removed from core. Also, the routing system now has support for protection against CSRF attacks by providing tokens to urls.

After learning what Drupal does for security, it’s time to see what site administrators should make sure of. Keeping the following 3 things in mind you as site admin should be fine for 95% of the cases.   (These are only the Drupal-specific aspects, we won’t go into general security principles.)

  1. To make sure you have an up-to-date site follow at least one of the security news channels. There are some RSS feeds, a twitter account and also a newsletter. Update your site as soon as a security update is released.
  2. There are several modules improving security or helping in finding security issues. A few of these are Security review, Paranoia and Two factor authentication.
  3. A Drupal-specific hosting provider can also have its benefits. For example, in the case of the infamous 2014 Drupalgeddon security advisory Pantheon and Acquia Cloud sites were protected against attacks without any action taken by the site administrator.

If you have not done it yet go and check your module update status page right now.

This blog post is heavily based on the Lullabot podcast on Drupal Security.

For further links we recommend the Barcelona presentation of scor and klausi.

Apr 21 2016
Apr 21

The two biggest players in the Drupal 7 webshop field are Drupal Commerce (also known as DC1) and Übercart. DC1 actually started as an Übercart rewrite to make use of Drupal 7 APIs. After the split Übercart was ported to Drupal 7 too but it was still using Drupal 6 technologies.

Although still very much in development, it seems something similar will be true for Drupal 8 as well. The developers of DC2 (the Drupal 8 version of Drupal Commerce), lead by Bojan Živanović rewrote the whole system from scratch to make use of the huge changes in Drupal 8. They are active members of the Drupal developer community so they not only know but also form the actual best practices. While working on DC2 they have fixed many dozens of Drupal 8 core issues and much more in other contributed modules (such as Entity, Inline Entity Form, Profile).

A great realisation when rewriting Commerce was that several components of a webshop could be reused by other (not even necessarily webshop or Drupal) systems. Some typical examples are address formats, currencies and taxes. These components are usually a huge pain to maintain because of the small differences from country to country. So they have created standalone PHP libraries usually using authorative third party datasets such as CLDR for currency or Google’s dataset for address formats. Some of them are already used by other webshop solutions like Foxycart and developers even outside the Drupal community are giving feedback which makes maintaining them easier.

In the DC2 development process UI and UX has got a big emphasis already from the beginning. Based on research of existing webshop solutions the shop administration and checkout process has been redesigned by UX specialists. For example, the product creation process is quite confusing in DC1 and there’s not even a recommended way to do it. In DC2 this happens now in one single form which makes it super easy.

A new concept in DC2 is Stores. Stores represent billing locations and products can belong to one ore more stores. One use case is the need for different billing for customers from different countries. Another one is having a shop where sellers can open an account and sell their own products. In this case each seller has their own store.

There are many other new features and improvements like a new and flexible tax system (you can say things like: “from Jan 1st 2014 the VAT changes from 21% to 19%”), a redesigned checkout flow, different workflows for different order types etc.

DC2 is still in alpha phase and is not recommended for production use yet. Beta releases will already have upgrade paths between them and so can be considered for starting real sites with. Beta1 is expected for May.

Drupal Commerce is the most popular e-commerce solution for Drupal 7. Given the high quality code and responsiveness to developer, shop maintainer and customer needs I do not expect this to change in Drupal 8 either.

Sources:
Drupal Commerce 2 blog
Modules Unraveled podcast on Commerce

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web