Upgrade Your Drupal Skills

We trained 1,000+ Drupal Developers over the last decade.

See Advanced Courses NAH, I know Enough
Jan 14 2022
Jan 14

in has been intense over the last 4 months, since we first presented this new to the community. Let's catch up with all the features and ideas, that already made it into it.

Who is working on it

In addition to the code maintainers Daniel, Jürgen, Max and Richard we also have supporting team members like Nico who contributed a nice logo, Ralf who helps us to get the word out and provides feedback - and a few others who already started using ECA and report back in the issue queue with issues and feature requests. This helped us a lot to stay focused and move towards a first stable release in such a short period of time.

How we moved forward

Although still in early stage, we optimized the core components by changing how ECA stores its config entities, as well as the caching mechanism to further reduce the performance impact when using ECA in production environments. The overhead ECA brings to your live site is negligible, it only consumes resources if something needs to be done as a result of configured events that your ECA models want to react upon.

The code structure got an overhaul too. There is now ECA Core which provides the config entity handling, the ECA processor, all required plugin managers and numerous interfaces, traits and base classes used by the now 10 submodules that comes with ECA:

  • ECA Base: provides access to context, cron, custom events, state system and tokens
  • ECA Config: integrates with Drupal's CMI
  • ECA Content: supports content entities, fields API and more tokens
  • ECA Form: integrates with Drupal's form API
  • ECA Migrate: integrates with events of the migrate module
  • ECA Misc: make available events from Symfony's kernel, Drupal core and the routing system
  • ECA Queue: allows to enqueue tasks for asynchronous or scheduled processing
  • ECA User: integrates with user account related events and actions like login, logout, cancel, account switching and roles
  • ECA Views: allows processing of views with further processing or exporting of the result set
  • ECA : integrates into Drupal's content moderation framework

Note that all of ECA depends solely on Drupal core and the new remarkable Context Stack module from Max, which we extracted from the ECA code base simply because its functionality is useful for other tools too. This is intentional so that you can easily onboard ECA without the requirement to have any other components being installed or enabled.

General features built into ECA core

With all the submodules enumerated above, you already get comprehensive lists of events, conditions and actions from either ECA or from Drupal core. Additionally, ECA has baked-in support for crucial APIs and functionality, that needs highlighting:

  • Context Stacks: each process triggered by any event as well as any action of such processes get executed within their own (stackable) context, so that ECA can restore the context after execution to the state it was right before processing. This not only makes each process predictable, but also allows to forward the current context into the queue, should a model want to process actions asynchronously.
  • Caching: all ECA models get cached in a special way such that ECA can determine, with almost no overhead, whether any of the triggered events in your Drupal site needed to be processed by ECA.
  • Loops: by default, actions in an ECA model operate on some data object, which often is something like a content entity, a route, an HTTP request or any other arbitrary value. If you provide a list of objects instead of a single instance, then ECA automatically loops over each of them with the subsequent actions in the process model.
  • Logging: everything in ECA is built to provide detailed log information, which is written to Drupal's logger service. It depends on your own setup, if that logging information is being written to watchdog, syslog or elsewhere. To avoid overwhelming logs just from ECA, you can configure the level of log information coming from ECA with a single setting, defaulting to error messages and exceptions only.
  • States: reading and writing from and to Drupal's state system helps to persist some data for later use, even if this is required over a longer timeframe.
  • Tokens: Drupal core's token environment is already pretty powerful. ECA builds on top of it and enhances it by introducing DTOs (Data Transfer Objects). This mechanism allows any event, condition or action in an ECA model to store any data into the token environment so that subsequent components of that model can access that data for further processing if needed.
  • TypedData: one of the most significant features and presumably the most used conditions and actions are around entity fields. Those fields can be more complex than just storing individual values, they can even store values with any cardinality. To allow ECA models generic access to any component of such fields, the TypedData API is the ideal tool provided by Drupal core, which has been utilized by ECA.
  • Recursion prevention: it may happen unintentionally, but ECA models, like any other rule definition, can quickly end up in a recursion. The simplest use case might be an entity update, which results in a process that makes additional modifications to that entity - being another entity update that already starts a recursion. As this would always lead towards an infinite loop, something you prefer not to experience on your production site, ECA inevitably prevents that from happening. Even at the cost of preventing you from building certain models. However, there should always be a way around that - and the issue queue on drupal.org or this Slack channel are great places to seek help in such situations.

Working on test coverage

We've already started building tests, from simple unit test to kernel and functional tests, but also a powerful framework to test any ECA model regardless how complex they are. This helps us to keep moving with ECA development while ensuring that existing functionality remains intact. Having said that, there's a lot that needs to be done to bring the test coverage to the necessary level. This is what we're focussing on in the weeks to come.

Integrating with other modules

Now that ECA core with all its integrated submodules stabilizes its own APIs, the time is right to start working with other maintainers of popular and useful modules, to make their functionality available to ECA and its models. The list of modules, we would like to integrate with, is being tracked in this issue and if anyone wanted to add more modules to the list, this is the right place to get the ball rolling.

Talking about the UI provided by modellers

As powerful and feature rich the ECA framework already is, it still requires the UI component which allows users to create and maintain their models. This is a separate task and completely decoupled from ECA as the backend processor. The component we're talking about here is called a modeller. And there can be more than just one. BPMN as one of the popular standards to describe business processes is supported by a variety of applications that allow to work with such models, two of which have already been integrated with ECA: Camunda and BPMN.iO. But others, even such that are not based on BPMN, can be integrated as well, and we hope to see initiatives who want to get started on such modeller development.

However, what matters to all the existing and future modeller UIs, is a great user experience (UX). This is what we haven't worked on yet for the existing integrations, and that's one of the areas where we have to spend more resources - but we also need help for that. The existing ECA team is busy with all the processing and background tasks. To move more quickly with the UI too, we're seeking for help.

Conclusion

A lot has been achieved! And I mean, A LOT. The team is really proud of it and feedback we receive is very positive too. We even consider ECA very close to being feature-complete for its 1.0 release soon. Yes, we admit, there is a way to go before it will be usable by "mainstream" users. The UI components mentioned above as well as comprehensive documentation are just the 2 main tasks that come to mind.

Having said that, it turns out that ECA not only accomplished its original objective to provide a "Rules" replacement for Drupal 9 and beyond, it also delivers benefits above and beyond that mission with 2 additional wins:

  1. Reduce the likelihood of writing a custom module for a client, only because you required a hook or a form alter function, that can now also been done with ECA.
  2. Eliminate the requirement for a number of helper modules that provide simple functionality like redirects after login or validating, stripping, trimming or otherwise manipulating certain fields.

In other words, the feature set provided by ECA is huge, yet it can reduce the complexity of any Drupal installation out there. And while doing it this way, ECA also improves transparency and maintainability by storing all the rules in one place, where it can be visualized, reviewed and even moderated so that all stakeholders get a better understanding of what's being implemented and configured at any given time. It also increases accessibility for users who understand the business logic implemented into a Drupal site but wouldn't ever be able or willing to dive deep into Drupal logic or even code base.

Let's move on with ECA, it's got that far that we're all very excited and motivated to move it over the finish line. Watch this space to stay up to date - or reach out on drupal.org or in Slack, if you either have ideas, feature requests or resources to help us get all this done.

Jan 04 2022
Jan 04

bitegra Solutions becomes LakeDrops!

With this merger, we focus even more on technology and . With the so called "LakeDropper" Richard Papp and Matthias Walti we additionally strengthen the Drupal expertise and allow an even broader foundation of high quality and long term service around Drupal.

Under the new name LakeDrops GmbH & Co. KG we focus even more intensively on Drupal as a platform. The solution portfolio around the already established portal framework will be continuously expanded. We already maintain and develop a large number of modules on Drupal.org. A visit to our company page there is worthwhile. Just have a look. 

We still love . In the future, we will make all our developments publicly available. In the next months, we plan to release more information about our portal framework. You can be very curious about many interesting components. We will of course keep you informed.

History

LakeDrops exists since 2011 as a agency and has made a name for itself in the Drupal community with Jürgen Haas, Richard Papp and Matthias Walti. With increasing project size, which could no longer be managed with freelancers, Jürgen Haas, Harald Geisser and Daniel Speicher founded bitegra Solutions GmbH & Co. KG. In January 2022, bitegra Solutions was transferred to LakeDrops and is now strengthened by Richard Papp and Matthias Walti.

Harald Geisser, previously on the management board of bitegra Solutions GmbH & Co. KG, left the company in January 2022.

Note for customers, suppliers & partners

Please also note our new company address as of 01.01.2022: LakeDrops GmbH & Co. KG, Am Breitenstein 27, D-78244 Gottmadingen, Germany

Sep 17 2021
Sep 17

Our new suite intends to replace the popular 7 Rules module for Drupal 8+. But not only Rules. Actually, it also provides features from other modules (e.g. Automatic Entity Label) and for form validation.

Rules allows site administrators to define conditionally executed actions based on occurring events (known as reactive or rules). You can e.g. display a message to the user or send an email when updating content.

But the Drupal 8+ version is still in alpha release stage and needs significant work. There is a module tr_rulez to fill the gaps, but it is only available as a version. Business rules is only minimally maintained and not stable, either. So if you wanted to do something similar in Drupal 8+, you would have been left out in the cold, so far.

ECA

In order to upgrade Drupal 7 applications that heavily use rules we looked for an alternative solution for Drupal 9+. Based on earlier discussions on Twitter we came up with an idea for an Event – Condition – Action processor that can handle business process models stored as Drupal config entities.

Sat, 12/18/2021 - 14:01

Twitter - Replace Rules

The models should be created with modellers integrated by plugins either as stand-alone applications or as part of the Drupal user interface. Hence anyone could easily add its own modeller if desired.

Our aim was to reuse existing Drupal core components as far as possible and to rely on Drupal API for new components. Out-of-the-box Drupal provides events, event subscribers and actions. There is also a plugin API that allows to add more components. This way components can be reused anywhere in Drupal, not only in the context of our module.

The only thing we had to add to the processor was a plugin manager for conditions and apply a small trick to chain entity related actions together.

Modeller

Think of a modeller as the user interface for ECA config entities. It can be integrated as a @EcaModeler plugin, that stores models as config entities. The plugin manager is provided by the ECA module.

2 modelers are already integrated. We relied on BPMN modellers so far because non technical users might be accustomed to flowcharts. Similar to Behat, BPMN modellers allow to define and to document business processes in a business-readable language, at once. No need to write extra documentation and a diagram is sometimes worth a thousand words.

Documentation

Dec 26 2020
Dec 26

After bitegra solutions won the Splash Award Switzerland in several categories in the fall of 2020 and even celebrated the overall winning across all categories, it was time for the redesign and the visual and user-friendly upgrade of our website.

The challenge

At the time, our existing website was created without the help of a designer. As a result, the information was presented without a specific visual style or structure. The potential to improve the look and feel of our website and make navigation easier and more accessible was very high.

From old to new, virtually no stone was left unturned and we started completely from scratch as far as the conception of our website is concerned. In the following the executed steps are shown and explained, that you too can make a successful redesign of your website. bitegra solutions is an expert in the field of digitalization and in the construction and operation of digital solutions. Therefore, please do not hesitate to contact us.

Design Principles

At the beginning we analyzed the previous state of the website and fixed some potentials that we want to use in the redesign. Inspired by Don Norman’s Principles of Interaction Design and Dieter Rams timeless commandments for good design we envisioned our own design principles. This part of the design process has to be done in a very disciplined way, because in the following everything will be derived from this analysis. So we focused on the following aspects in particular:

Conception & Wireframing

With these guidelines in mind, we designed a first prototype, which was then tested by several user testings of selected personas. We incorporated the feedback collected in the user into our prototype until we had designed a first alpha version of the website.

With the extensive findings from the alpha version, we then created a beta version of the design, which was again validated through user testing. Through several rounds of feedback, we approached the final beta design, which met user needs exceptionally well.

Visual design

Based on the very mature results of the beta design, this design draft was then fine-tuned through minimal improvements and selected as the final design in a selection process based on the guidelines defined above.

Design implementation and iteration based on the Figma design tool allowed us to move collaboratively and quickly into implementation and to make decisions efficiently based on a unified decision-making basis. In addition, Figma provides an optimal designer - developer that involves the developer in the design processes.

Site Building and

The old 8 site was updated to Drupal 9 with regard to the redesign, which was fully automated. The strength of Drupal was again confirmed in this task, as the update went completely smoothly and the complete content structure of the old site could be adopted 1:1. Here, countless components are used, which extend the basic functionality of Drupal, such as the Gin Admin Theme or the Twig Tweak .

With the very extensively designed pages and components of the website, an Atomic Design Pattern was then implemented for the styling including a style guide, which allows for high maintainability and at the same time scaling without any problems. With the help of the newly released Bootstrap v5, we used state-of-the-art frontend architecture and were able to implement our design faster than usual.

We are proud to start 2021 with this beautiful redesign and wish all our readers, customers and colleagues a happy new year. We look forward to hearing from you and helping you with your project.

Nov 27 2020
Nov 27

The Splash Awards Switzerland have been held as an online event in 2020, still it has been an entertaining and always worthwhile event for all participants. For the team at bitegra Solutions this has been extra special, not only because we have been newcomers to this event. We had submitted one of our current projects, the "Integrated tenant portal for Mattenhof, Luzern" (see our Case Study) for 3 categories and got nominated for all three of them.

The outcome from this year's Splash Awards has been like this:

  • Nominee in the Tools & Apps category
  • Runner-Up in the Design & UX category
  • Winner in the Technology category
  • Award as the Project of the Year 2020

This of course if extra special to everybody involved, for our whole team as well as for our client Mobimo who has been a great partner throughout the whole project, and still is. You can watch the full ceremony of the Splash Awards Switzerland 2020 in the video below. This video is the edited version of the Awards ceremony as of Thursday, 26th November 2020 which is worth watching in full length. For the bitegra Solutions project parts, you may want to focus on these segments:

  • Intro and Award for category Desing/UX: 00:17 – 00:26
  • Award for category Technology: 1:03 – 1:14
  • Award for category Tools & Apps: 1:14 – 1:26
  • Award Project of the Year: 1:33 – 1:37

[embedded content]

Aug 03 2020
Aug 03

Almost all websites have one requirement in common: getting notified when something specific is happening. Whether that's a new blog post, a new comment, a modified version of a node, an exception in the logs or a deleted user account, either the site admin or website visitors would like to know about some or all of these.

While this requirement is pretty common, each and every individual has very different views on how they want to get notified. Without getting into any discussion on effectiveness of emails, it seems enough to just think about all these other available channels like Slack, mobile push, ticket systems or even SMS.

Building websites with makes you wonder why the solution stack to those requirements is that fragmented, as it has been for over a decade. For Drupal 8 and 9 alone, you'll find over 100 modules for the "notify" search term. At least 40 of them should be considered serious solutions, as they have stable releases and security advisory coverage. But each of them only ever addresses one single aspect, i.e. only nodes or only comments or states or exceptions or what ever. And then, most of them only offer email notifications for each of the event types they've been built for.

As this is not appropriate for an enterprise grade web application framework like Drupal, it's been time to sit down and address this space in a generic, scalable, configurable and maintainable fashion such that the community can build upon it and be ready for current and future requirements - because the subscription and notification requirements are under permanent pressure to adjust and improve, because the world around us is changing all the time as well.

The team at LakeDrops (edited: was "bitegra Solutions" before) in Germany spent almost 2 years in problem analysis, solution design, software architecture and to make available the ultimate solution for Drupal. We've decided to publish all available components on drupal.org and let the community participate in the benefits but also future maintenance and development of existing and additional features and notification channels for free.

This is part 1 of a blog post series which introduces the idea and implementation and how you can use them yourselves on Drupal projects of any size:

  • Part 1 (you are here): Introduction and system architecture
  • Part 2: Getting started with subscriptions and notifications
  • Part 3: How to add new notification channels
  • Part 4: Covering more events for being notified
  • Part 5: Customizing the notification output

System architecture

Let's get started with a list of requirements:

  • Support everything notable:
    • Content, i.e. entities:
      • Create, update, delete and publish/unpublish
      • Every entity type and bundle
      • Each bundle with individual and detailed settings
    • Configuration: track all changes to the site's configuration
    • Logger: track all log entries above a configured threshold
    • External events: allow other services in the enterprise network to notify the same audience
    • Allow developers to provide new plugins to create notifications
  • Be flexible with recipients
    • Authenticated users should be able to subscribe/unsubscribe to/from any available event configured on the site and made available for subscription
    • Site configuration should allow pre-selection of recipients for each available event type
    • Content editors with appropriate permission should be able to overwrite notification defaults per entity action (create, update, delete, publish, unpublish)
    • Allow developers to apply custom rules on recipient selection
  • Support multiple notification channels
    • Allow site builders to activate any number of different notification channels at once
    • Allow users to define their own preferences on channel priority
    • Make sure that each user only gets notified once for each event
    • Ensure and trace notification delivery
    • Provide good defaults for notification rendering per channel
    • Allow site builders to modify the notification output
    • Allow developers to add new channels

While collecting and structuring the requirements, it became obvious that there are 3 isolated phases that need to be addressed separately:

  1. Event capturing
  2. Recipient selection and notification creation
  3. Notification delivery

Yes, phase 2 contains two steps: for each notable event all relevant recipients need to be determined, and for each of them a notification entity gets created in order to further handle that notification for each recipient individually.

Sat, 12/18/2021 - 13:56

DANSE 1

This led to another major decision in the solution design: capturing events and selecting recipients for notifications is one major building block, and delivering those notifications to users over a number of channels is another one. Both building blocks can operate independently, and hence we broke this down into 2 separate modules:

DANSE: Drupal - Audit - Notification - Subscription - Event

The DANSE module addresses the first two phases: event capturing and recipient selection. Both phases are implemented with their own plugin managers and hence can easily be enhanced to support additional requirements.

Out of the box, DANSE supports event capturing for content entities, configuration, logging and webhooks (external events). Each of those event types are implemented in separate submodules so that site builders can enable just those actually needed.

The recipient selection plugin manager allows implementing your own business logic but brings 2 plugins already with it: "all admin users" and "all active users". For the content entity event capturing plugin above, site builders can also configure on a per-bundle basis, which user roles are allowed to subscribe to supported event types - available subscriptions can be recognized as a recipient selection plugin in its own right.

DANSE comes with 3 entity types to handle its duties:

  • Events (danse_event): each notable event gets stored with its creation timestamp and further details depending on the event type
  • Notification (danse_notification): for each recipient of each event, a single notification gets stored
  • Action (danse_notification_action): to track the activity around that notification, including an API to store delivery attempts and the receipt flag

As a result, this module allows you to audit the activity on your website by reviewing all these entities for events, notifications and actions so that you can easily tell what happened when and why. Of course, it also stores who got notified how and when. So, that made the final piece of the module name, just in cased you asked yourself: audit.

An extra benefit is the notification widget that comes with DANSE. Without even delivering notifications out to the recipients, there is a huge value in telling your users what's new and changed on the site. The stored notifications allow you to do that easily. DANSE comes with a view for unseen notifications, and you can place that view as a block on the site for authenticated users.

With this in mind, DANSE can be used on its own for any existing Drupal 8 and Drupal 9 site.

Push Framework

In order to also send notifications out to their recipients, the Push Framework module comes in handy. It hooks into the API provided by DANSE but can also be used together with any other source of information that should be delivered. To make this possible, the push framework comes with a source plugin manager, and DANSE is just the first module which implemented a plugin for this. But of course, other sources can be provided easily like e.g. a newsletter module.

The push framework module handles all the logic of collecting the source content which is due for delivery, determining the appropriate channel for delivery which depends on available channels and user preferences, and then delivers the notification and reports back to the source on the delivery success. If delivery fails, each channel can support any number of retries and continue with another channel, if all retries failed.

The output channels are made available through the channel plugin manager, which allows developing additional channels very easily. Currently, supported (or planned) channels are:

  • Email
  • OneSignal
  • Alerta
  • Slack (planned)
  • Mattermost (planned)
  • Twilio (planned)

Diagram of the overall solution

Both of the above modules make intensive use of queues, i.e. all the heavy lifting is operated asynchronously. That helps to ensure reliability of both event capturing and recipient selection, as well as delivery - without any impact on the website's performance.

Sat, 12/18/2021 - 13:56

DANSE 2

Both DANSE and Push Framework have been carefully designed to provide a rock solid solution for all Drupal sites of any size. It comes with a common feature set for (a) event types which create notifications and (b) notification channels. But we are well aware that lots of sites have their own business logic for most of this solution space - even our own first customer project, where we used this, had such a specific requirement - which can still be satisfied by developing custom plugins or public modules, if the extra feature were of general interest.

The subsequent parts of this blog post series will give you all the insight into what's required to use these modules and how to enhance them towards your own requirements.

Update 11.8.2020: Due to an issue with the old module name, we've changed the name and updated this blog post accordingly.

Aug 03 2020
Aug 03

Current situation

Almost all websites have one requirement in common: getting notified when something specific is happening. Whether that's a new blog post, a new comment, a modified version of a node, an exception in the logs or a deleted user account, either the site admin or website visitors would like to know about some or all of these.

While this requirement is pretty common, each and every individual has very different views on how they want to get notified. Without getting into any discussion on effectiveness of emails, it seems enough to just think about all these other available channels like Slack, mobile push, ticket systems or even SMS.

Building websites with makes you wonder, why the solution stack to those requirements is that fragmented as it has been for over a decade. For Drupal 8 and 9 alone you'll find over 100 modules for the "notify" search term. At least 40 of them should be considered serious solutions as they have stable releases and security advisory coverage. But each of them only ever addresses one single aspect, i.e. only nodes or only comments or states or exceptions or what ever. And then, most of them only offer email notifications for each of the event types they've been built for.

Our solution

As this is not appropriate for an enterprise grade web application framework like Drupal, it's been time to sit down and address this space in a generic, scalable, configurable and maintainable fashion such that the community can build upon it and be ready for current and future requirements - because the subscription and notification requirements are under permanent pressure to adjust and improve, because the world around us is changing all the time as well.

The team at bitegra Solutions in Germany spent almost 2 years in problem analysis, solution design, software architecture and to make available the ultimate solution for Drupal. We've decided to publish all available components on drupal.org and let the community participate in the benefits but also future maintenance and development of existing and additional features and notification channels for free.

We take you on a journey with us

This is part 1 of a blog post series which introduces the idea and implementation and how you can use them yourselves on Drupal projects of any size:

  • Part 1 (you are here): Introduction and system architecture
  • Part 2: Getting started with subscriptions and notifications
  • Part 3: How to add new notification channels
  • Part 4: Covering more events for being notified
  • Part 5: Customizing the notification output

System architecture

Let's get started with a list of requirements:

  • Support everything notable:
    • Content, i.e. entities:
      • Create, update, delete and pusblish/unpublish
      • Every entity type and bundle
      • Each bundle with individual and detailed settings
    • Configuration: track all changes to the site's configuration
    • Logger: track all log entries above a configured threashold
    • External events: allow other services in the enterprise network to notify the same audience
    • Allow developers to provide new plugins to create notifications
  • Be flexible with recipients
    • Authenticated users should be able to subscribe/unsubscribe to/from any available event configured on the site and made available for subscription
    • Site configuration should allow pre-selection of recipients for each available event type
    • Content editors with appropriate permission should be able to overwrite notification defaults per entity action (create, update, delete, publish, unpublish)
    • Allow developers to apply custom rules on recipient selection
  • Support multiple notification channels
    • Allow site builders to activate any number of different notification channels at once
    • Allow users to define their own preferences on channel priority
    • Make sure that each user only gets notified once for each event
    • Ensure and trace notification delivery
    • Provide good defaults for notification rendering per channel
    • Allow site builders to modify the notification output
    • Allow developers to add new channels

While collecting and structuring the requirements, it became obvious that there are 3 isolated phases that need to be addressed separately:

  1. Event capturing
  2. Recipient selection and notification creation
  3. Notification delivery

Yes, phase 2 contains two steps: for each notable event all relevant recipients need to be determined, and for each of them a notification entity gets created in order to further handle that notification for each recipient individually. 

This led to another major decision in the solution design: capturing events and selecting recipients for notifications is one major building block, and delivering those notifications to users over a number of channels is another one. Both building blocks can operate independently and hence we broke this down into 2 separate modules:

The DANSE module (DANSE: Drupal - Audit - Notification - Subscription - Event) addresses the first two phases: event capturing and recipient selection. Both phases are implemented with their own plugin managers and hence can easily be enhanced to support additional requirements.

Out of the box, DANSE supports event capturing for content entities, configuration, logging and webhooks (external events). Each of those event types are implemented in separate submodules so that site builders can enable just those actually needed.

The recipient selection plugin manager allows to implement your own business logic but brings 2 plugins already with it: "all admin users" and "all active users". For the content entity event capturing plugin above, site builders can also configure on a per-bundle basis, which user roles are allowed to subscribe to supported event types - available subscriptions can be recognized as a recipient selection plugin in its own right.

DANSE comes with 3 entity types to handle its duties:

  • Events (danse_event): each notable event gets stored with its creation timestamp and further details depending on the event type
  • Notification (danse_notification): for each recipient of each event a single notification gets stored
  • Action (danse_notification_action): to track the activity around that notification including an API to store delivery attempts and the receipt flag

As a result, this module allows you to audit the activity on your website by reviewing all these entities for events, notifications and actions so that you can easily tell what happened when and why. Of course, it also stores who got notified how and when. So, that made the final piece of the module name, just in cased you asked yourself: audit.

An extra benefit is the notification widget that comes with DANSE. Without even delivering notifications out to the recipients, there is a huge value in telling your users what's new and changed on the site. The stored notifications allow you to do that easily. DANSE comes with a view for unseen notifications and you can place that view as a block on the site for authenticated users.

With this in mind, DANSE can be used on its own for any existing Drupal 8 and Drupal 9 site.

In order to also send notifications out to their recipients, the Push Framework module comes in handy. It hooks into the API provided by DANSE but can also be used together with any other source of information that should be delivered. To make this possible, the push framework comes with a source plugin manager and DANSE is just the first module which implemented a plugin for this. But of course, other sources can be provided easily like e.g. a newsletter module.

The push framework module handles all the logic of collecting the source content which is due for delivery, determining the appropriate channel for delivery which depends on available channels and user preferences, and then delivers the notification and reports back to the source on the delivery success. If delivery fails, each channel can support any number of retries and continue with another channel, if all retries failed.

The output channels are made available through the channel plugin manager which allows developing additional channels very easily. Currently supported (or planned) channels are:

  • Email
  • OneSignal
  • Alerta
  • Slack (planned)
  • Mattermost (planned)
  • Twilio (planned)

  

Diagram of the overall solution

Both of the above modules make intensive use of queues, i.e. all the heavy lifting is operated asynchronously. That helps to ensure reliability of both event capturing and recipient selection as well as delivery - without any impact on the websites performance.

Both DANSE and Push Framework have been carefully designed to provide a rock solid solution for all Drupal sites of any size. It comes with a common feature set for (a) event types which create notifications and (b) notification channels. But we are well aware that lots of sites have their own business logic for most of this solution space - even our own first customer project, where we used this, had such a specific requirement - which can still be satisfied by developing custom plugins or public modules, if the extra feature were of general interest.

The subsequent parts of this blog post series will give you all the insight into what's required to use these modules and how to enhance them towards your own requirements.

Update 8/11/2020: Due to an issue with the old module name, we've changed the name and updated this blog post accordingly.

Jun 27 2019
Jun 27

8 already provides the necessary configuration files and PHP plugins to migrate files from Drupal 6. But if you are using the Drupal 8 core media for your files you need to add a second .

I assume that you are doing a Drupal 8 migration using the migrate tools and migrate plus modules.

File migration

Copy /core/modules/file/migrations/d6_file.yml to config/install/migrate_plus.migration.d6_file.yml in your custom module and set the source file path:

# Every migration that references a file by Drupal 6 fid should specify this
# migration as an optional dependency.
id: d6_file
label: Public files
audit: true
migration_group: d6
migration_tags:
  - Drupal 6
  - Content
source:
  plugin: d6_file
  constants:
    # The tool configuring this migration must set source_base_path. It
    # represents the fully qualified path relative to which URIs in the files
    # table are specified, and must end with a /. See source_full_path
    # configuration in this migration's process pipeline as an example.
    source_base_path: '/path/to/drupal6/files/to/migrate'
...

Drupal 8 refuses to install a module whose configuration has already been imported. To avoid such error add a dependency on your custom module to the migration file. If you uninstall the module, Drupal 8 will also remove the configuration:

dependencies:
  enforced:
    module:
      - custom_migrate_module

Media entity migration

Copy /core/modules/file/migrations/d6_file.yml to config/install/migrate_plus.migration.d6_media_entity.yml in your custom module.

Change plugin id and label, process and destination configuration in order to create media entities:

Before (excerpt)

id: d6_file
label: Public files
...
process:
    # If you are using both this migration and d6_user_picture_file in a custom
    # migration and executing migrations incrementally, it is strongly
    # recommended that you remove the fid mapping to avoid potential ID
    # conflicts. For that reason, this mapping is commented out by default.
    # fid: fid
  filename: filename
  source_full_path:
    -
      plugin: concat
      delimiter: /
      source:
        - constants/source_base_path
        - filepath
    -
      plugin: urlencode
  destination_full_path:
    plugin: file_uri
    source:
      - filepath
      - file_directory_path
      - temp_directory_path
      - is_public
  uri:
    plugin: file_copy
    source:
      - '@source_full_path'
      - '@destination_full_path'
  filemime: filemime
  # No need to migrate filesize, it is computed when file entities are saved.
  # filesize: filesize
  status: status
  changed: timestamp
  uid: uid
destination:
  plugin: entity:file

After (excerpt)

id: d6_media_entity
label: Media entities
migration_group: d6
...
process:
  langcode:
    plugin: default_value
    default_value: 'en'
  field_media_file/target_id:
    plugin: migration_lookup
    migration: d6_file
    source: fid
  field_media_file/display:
    plugin: default_value
    default_value: 0
  field_media_file/description:
    plugin: default_value
    default_value: ''
  status: status
  changed: timestamp
  uid:
    plugin: migration_lookup
    migration: d6_user
    source: uid
destination:
  plugin: entity:media
  default_bundle: file

Or if you migrate images:

id: d6_media_entity
label: Media entities
migration_group: d6
...
process:
  langcode:
    plugin: default_value
    default_value: 'en'
  field_media_image/target_id:
    plugin: migration_lookup
    migration: d6_file
    source: fid
  field_media_image/alt: filename
  field_media_image/title:
    plugin: default_value
    default_value: ''
  status: status
  changed: timestamp
  uid:
    plugin: migration_lookup
    migration: d6_user
    source: uid
destination:
  plugin: entity:media
  default_bundle: image

At last, add the migration dependencies to config/install/migrate_plus.migration.d6_media_entity.yml:

migration_dependencies:
  required:
    - d6_user
    - d6_file
dependencies:
  enforced:
    module:
      - custom_migrate_module

Running these 2 migrations will copy your files from the Drupal 6 to the Drupal 8 file directory and create a media entity for each of them.

Assign media entities to nodes

In order to assign media entities to media fields on nodes, you have to assign the media entity id (instead of the file id) to the field, e.g.:

id: d6_node
label: Nodes
audit: true
migration_group: d6
migration_tags:
  - Drupal 6
  - Content
deriver: Drupal\node\Plugin\migrate\D6NodeDeriver
source:
  plugin: d6_node
process:
  field_media:
    plugin: sub_process
    source: uploads
    process:
      target_id:
        plugin: migration_lookup
        migration: d6_media_entity
        source: fid
        no_stub: true
      display: list
      description: description
...
destination:
  plugin: entity:node

You may have to create a custom source plugin in order to retrieve the id of the file that is attached to a node.

Dec 09 2018
Dec 09

Thanks to a tutorial from @swentel my attention got back to webmentions again and I gave the first release candidate for another try. What's still confusing to me that I have to enter a link-class and a publishing target manually. The former might be a task for a link widget and the latter may be some lack of understanding the API so far.

The Drupal looks pretty solid and feature rich. Webmentions - the new thing on the web - makes me hope that the dependencies on "social" networks like Twitter, Facebook, Reddit or others could be massively reduced if not eliminated. Sounds like another major step towards de-centralising the web. Thanks a lot to all the contributors to Webmentions and the Drupal integration.

May 31 2018
May 31

 Updated again in February 2021 with even more tasks and tools that got replaced.

Updated in January 2020 with significant improvements since originally posted.

My personal #gdpr today, May 25th 2018: completed my project to get back all my data from @Google, @evernote et al and host it all by myself with @Nextclouders, #joplin and dozens of other @OpenSourceOrg tools that come with the same convenience but with real privacy. Check!

Following this Twitter post I got asked about more details and how I actually achieved that goal. Here is a table of tools that I'm using now instead of the old tools that I consider harmful to some extent:

* Plugin for Thunderbird

All the red ones are gone completely - and that feels really good. The green marked SpiderOak ONE - although it falls into the old tool category as it is proprietary - I'm planning to keep that in the tool set, simply because it is powerful, affordable, and it well protects my data because everything is encrypted prior to uploading it to their facilities. SpiderOak ONE is now gone also because their CLI support has always been weak and since they clearly stated in multiple support issues, they won't be improving that, it got time to drop this too and BorgBackup as a replacement turned out to be even stronger.

That leaves the yellow ones. Those are the tools I would love to replace but haven't found an appropriate solution yet:

  • Feedly and Pocket: The news app in Nextcloud aims to provide the same sort of functionality but lacks some features at this point. Let's hope this is going to improve over time. But as the content being managed, there is collected from internet sources and already public, this is not one of the most urgent tasks in my view. The news app is now good enough, and I'm happy with it.
  • LastPass: this is my go-to password manager, and it does work really well not only because it integrates almost perfectly on all the platforms I'm using day in and day out. However, since they've been acquired by LogMeIn Inc. my confidence dropped significantly and I'd replace it rather sooner than later. There are lots of alternatives, and it feels like I've tried them all, but none of them is mature enough yet and I have to keep watching out. January 2020: switched to KeePassXC and couldn't be more excited. I didn't really expect better integration in browser and Android, but all the tools around KeePassXC are better than everything I had seen before (including LastPass).
  • Authy: the fact that all the sites where I'm using 2FA are registered and stored on a third-party facility is a fairly big worry to myself. As the protocol for 2FA is public and the algorithm well documented, this is my number one candidate for a new app on Nextcloud so that I could be hosting all those keys in my own private cloud. January 2020: successfully switched to andOTP which stores all its data on my Nextcloud instance and hence I can use multiple devices or switch to a new one without having to re-setup 2FA everywhere. February 2021: now dropped andOTP as well as all the required features are included in all my KeePassXC clients too.
  • IntelliJ IDEA: as a freelancer, I heavily rely on a code writing tool with lots of integrations and supporting features. There is the open source alternative known as Eclipse, but it is far less capable and lacks performance. As a result, IntelliJ IDEA is one of the last tools that I pay an annual fee for with no regret.
  • Toggl: this is equally important for freelancers - no billing without meaningful time tracking. As Toggl integrates very well into almost any task and/or project management system on the planet, it is very convenient and works without extra time being spent. That's the final reason to stick with them, otherwise I'd switch to self-hosted open-source alternatives.

Of course, while not yet living on an island, there are frequent scenarios where exchange with others is important, either for work or pleasure. That's why a number of communication tools (WhatsApp, Messenger, Hangouts, Slack, GoToMeeting, Citrix), networking platforms (LinkedIn, Xing) or social networks (Facebook, Twitter) are still being used daily. Don't think we are close to the point where we could consider closing them down, but even positive things are happening unexpectedly now and then - so let's keep hoping for the better. January 2020: most of the above is not true any more: cancelled my Xing subscription, haven't used Facebook for months and don't use the communication tools mentioned above any longer. Just Twitter is left in the toolset, hoping to replace that with Mastodon when my peers are going to move also. Voice and video sessions are on a paid Zoom account now which is a commercial tool but at least offers end-to-end encryption. February 2021: Not sure about my judgement regarding Zoom any longer. Alternatives like Jitsi and BigBlueButton are getting there, and I should be replacing that in the next phase too. Facebook and WhatsApp accounts are deleted, the Google account will be next to be deleted.

Especially the communication tools are a main concern. While Skype screwed up completely and friends and customers are all using the wide range of tools listed above, I'd really love to consolidate most if not all on a single platform. Nextcloud Talk is pretty good already, but Zoom is the best platform to date (screen sharing on a multi screen desktop in particular) and Mattermost is amazing, feature rich and can be self-hosted. But for all three of them it's hard to get other on board as well which leaves you talking to yourself, which doesn't make sense all day.

Conclusion

Nextcloud managed to evolve into the central hub of all my personal and business data which is well protected with a 3-2-1 backup strategy and synchronized across all desktop and mobile devices when used together with add-ons and apps from various sources almost all of which are Open-source products. Of course, these are moving targets as new platforms, gadgets and ideas arise all the time. But so do these platforms, and they often integrate faster with new APIs than most of the proprietary tools. I am controlling my data to a very high degree without giving up significant convenience.

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web