Feeds

Author

Upgrade Your Drupal Skills

We trained 1,000+ Drupal Developers over the last decade.

See Advanced Courses NAH, I know Enough
Oct 21 2021
Oct 21

In a previous article on Drupal personalization I covered content personalization trends in terms of search frequency for the term "content personalization". We saw that interest in the term has been on the increase since around 2015 with possibly a peak being hit around 2020. Whilst it is too early to say that interest in the subject has collapsed, it certainly has taken a breather. In its place it appears that interest in "digital experience platform" (DXP) has picked up over the last couple of years. The chart below (Google Trends) shows the relative rise of DXP as a search term in contrast to personalization.

Red: DXP, Blue: Personalization

The rise in interest is DXP can be seen as an extension in the interest initially shown in content personalization. Attention is now moving from the concept of what it is to how it might be achieved. Personalization cannot be achieved by an isolated system. It requires the coordination of a number of processes including data gathering, management and processing to determine the next best piece of content to deliver to the user. Traditional content management systems (CMS) have been good at managing and delivering the content but not so good at orchestrating the delivery of the experience. Campaign management software has been effective at pushing content to users but has suffered from not having a view into what their interests are. And finally CRMs store content on known users but miss out on insights to anonymous users who probably make up the bulk of traffic to the website and other platforms. Digital Experience Platforms offer the promise of being able to bring the pieces of the puzzle together.

Customer Data Platforms (CDPs) potentially play a role in this ecosystem. They offer capabilities to form unified user profiles based on behaviour and the aggregation from a number of sources. They also offer user segmentation tools which can help drive the delivery of personalized content. In a similar way to DXPs, there has been a rise in interest in them as well over the last few years.

Red: DXP, Blue: Personalization, Yellow: CDP

In conclusion there is increased interest around technology to better understand users and how personalized content can be delivered to them across the various touchpoints.

But what is a DXP?

It has been said that there is no such thing as a DXP. By its very nature a DXP is a collection of different tools acting in unison with the ultimate aim of delivering digital experiences to users. As we know, the marketing technology landscape is vast with countless options available for almost any problem. Buyers will look for solutions which fit their needs, capabilities and budgets. This will inevitably lead to buyers working with a mix of technologies which may not be well integrated. The problem can then be seen as to how to coordinate these tools to act in unison. 

Gartner identifies a number of different capabilities for a DXP:

  • Content management
  • Account services
  • Personalization and context awareness
  • Analytics and optimization
  • Customer journey mapping
  • Customer data management
  • Presentation, delivery and orchestration
  • Search, navigation and insight
  • Collaboration and knowledge sharing
  • Security and access control
  • Artificial intelligence (AI)
  • Cloud capabilities
  • Architecture and platform design
  • Integration, interoperability and extensibility
  • Multiexperience support

The list is long and covers a wide range of requirements. In the DXP space vendors have been scrambling to build out platforms which tick the various boxes for what makes a DXP. The individual pieces of these solutions will have their strengths and weaknesses and may not be right for every customer. Tightly integrated solutions offer the advantage of enabling the flow of data and easier management, however, they may well offer the wrong type of solution for the customer. In all practicality it is not feasible to buy a single solution off the shelf and have success.

The rise of the composable DXP

Whilst DXP vendors have been making acquisitions to build out their monolithic platforms, a counter idea has emerged, that of the "composable" DXP. The main idea behind the composable DXP is the recognition that a monolithic solution is not going to be right for everyone. With the monolith comes vendor lock in and the inability to pick and choose "best of breed" solutions. In a fast moving world businesses need to be able to act in an agile manner to the changing landscape and demands.

In the Gartner report on the matter, the following recommendations were made:

  • Modernization of the DXP technology stack around the principles of composable business.
  • Improvements in operational agility by replacing the DXP monolith with a composable architecture.
  • Implementation of task-oriented packaged business capabilities

The ideas of the composable DXP has taken strong hold with many vendors now using the term in their marketing. This is particularly the case for smaller vendors who may not have developed their own monolith. Composable DXPs require coordination and integration. The focus is now on how well the tools work together and what the effort and cost is in making this happen. For buyers considering their platform, the concept of a composable DXP offers a way to reconceptualise the problem to focussing on business needs and processes, rather than a single silver bullet.

The open DXP

The rise the composable DXP with its emphasis on integration has necessarily lead to an evaluation capabilities of the components to interoperate. Open systems will fare better because they are outward looking and better able to work with other systems. Open source systems, with their collaborative nature, are at an advantage as they will more likely have a breadth of solutions available. Where there is a gap, new functionality can be easily incorporated. The architectures of open system will also make extensions easier to implement and incorporate.

"Openness" can be seen in two ways. The first refers to the licensing of the code. Open source systems will necessarily be open in this sense. However, in a wider sense, openness also refers to the ability for the system to be easily implemented and extended. Is the system open to being extended and improved. Solutions claiming to be open need to be evaluated in this light. 

Drupal's place in the DXP world

Drupal has earned its place as a leading enterprise CMS which excels at content management and integration. These have always been its strong points and as such it is extremely well placed to form a key part of a DXP. The Drupal ecosystem (hosting, code, support, knowhow) is capable of ticking many of the boxes for what makes up a DXP. 

Looking back at the Gartner list, it is apparent that there are key pieces missing:

  • Personalization and context awareness
  • Customer journey mapping
  • Customer data management
  • Orchestration
  • Architecture and platform design

These missing pieces should be seen as an opportunity for the Drupal community as they point the way to how Drupal, as a product, can be improved to remain future proof.

Interestingly, Drupal is namechecked by the composable DXP article:

Currently, most DXP products are too large and not task-oriented. This prevents organizations from managing them easily, or changing them quickly. DXP capabilities are not easily decomposable or replaceable. However, several DXPs do have extensibility built in, via robust APIs, extension frameworks (such as Drupal’s module framework) and/or app marketplaces. These can be the starting point on the journey toward composability.

Drupal is therefore seen as being well placed to take a central role in the composable DXP ecosystem due to its APIs and extensibility.

The future

There will no doubt continue to be increasied interest in DXPs from businesses looking to build integrated platforms. The rise of the composable DXP as a concept should see many smaller vendors with more open platforms starting to make an impact in the space. For Drupal, improvements in the areas of personalization and experience orchestration should see it build on the strong foundations it already has. This should see a range of DXP options becoming available as activity increases in the area.

Oct 19 2021
Oct 19

In a previous article on Drupal personalization I covered content personalization trends in terms of search frequency for the term "content personalization". We saw that interest in the term has been on the increase since around 2015 with possibly a peak being hit around 2020. Whilst it is too early to say that interest in the subject has collapsed, it certainly has taken a breather. In its place it appears that interest in "digital experience platform" (DXP) has picked up over the last couple of years. The chart below (Google Trends) shows the relative rise of DXP as a search term in contrast to personalization.

Red: DXP, Blue: Personalization

The rise in interest is DXP can be seen as an extension in the interest initially shown in content personalization. Attention is now moving from the concept of what it is to how it might be achieved. Personalization cannot be achieved by an isolated system. It requires the coordination of a number of processes including data gathering, management and processing to determine the next best piece of content to deliver to the user. Traditional content management systems (CMS) have been good at managing and delivering the content but not so good at orchestrating the delivery of the experience. Campaign management software has been effective at pushing content to users but has suffered from not having a view into what their interests are. And finally CRMs store content on known users but miss out on insights to anonymous users who probably make up the bulk of traffic to the website and other platforms. Digital Experience Platforms offer the promise of being able to bring the pieces of the puzzle together.

Customer Data Platforms (CDPs) potentially play a role in this ecosystem. They offer capabilities to form unified user profiles based on behaviour and the aggregation from a number of sources. They also offer user segmentation tools which can help drive the delivery of personalized content. In a similar way to DXPs, there has been a rise in interest in them as well over the last few years.

Red: DXP, Blue: Personalization, Yellow: CDP

In conclusion there is increased interest around technology to better understand users and how personalized content can be delivered to them across the various touchpoints.

But what is a DXP?

It has been said that there is no such thing as a DXP. By its very nature a DXP is a collection of different tools acting in unison with the ultimate aim of delivering digital experiences to users. As we know, the marketing technology landscape is vast with countless options available for almost any problem. Buyers will look for solutions which fit their needs, capabilities and budgets. This will inevitably lead to buyers working with a mix of technologies which may not be well integrated. The problem can then be seen as to how to coordinate these tools to act in unison. 

Gartner identifies a number of different capabilities for a DXP:

  • Content management
  • Account services
  • Personalization and context awareness
  • Analytics and optimization
  • Customer journey mapping
  • Customer data management
  • Presentation, delivery and orchestration
  • Search, navigation and insight
  • Collaboration and knowledge sharing
  • Security and access control
  • Artificial intelligence (AI)
  • Cloud capabilities
  • Architecture and platform design
  • Integration, interoperability and extensibility
  • Multiexperience support

The list is long and covers a wide range of requirements. In the DXP space vendors have been scrambling to build out platforms which tick the various boxes for what makes a DXP. The individual pieces of these solutions will have their strengths and weaknesses and may not be right for every customer. Tightly integrated solutions offer the advantage of enabling the flow of data and easier management, however, they may well offer the wrong type of solution for the customer. In all practicality it is not feasible to buy a single solution off the shelf and have success.

The rise of the composable DXP

Whilst DXP vendors have been making acquisitions to build out their monolithic platforms, a counter idea has emerged, that of the "composable" DXP. The main idea behind the composable DXP is the recognition that a monolithic solution is not going to be right for everyone. With the monolith comes vendor lock in and the inability to pick and choose "best of breed" solutions. In a fast moving world businesses need to be able to act in an agile manner to the changing landscape and demands.

In the Gartner report on the matter, the following recommendations were made:

  • Modernization of the DXP technology stack around the principles of composable business.
  • Improvements in operational agility by replacing the DXP monolith with a composable architecture.
  • Implementation of task-oriented packaged business capabilities

The ideas of the composable DXP has taken strong hold with many vendors now using the term in their marketing. This is particularly the case for smaller vendors who may not have developed their own monolith. Composable DXPs require coordination and integration. The focus is now on how well the tools work together and what the effort and cost is in making this happen. For buyers considering their platform, the concept of a composable DXP offers a way to reconceptualise the problem to focussing on business needs and processes, rather than a single silver bullet.

The open DXP

The rise the composable DXP with its emphasis on integration has necessarily lead to an evaluation capabilities of the components to interoperate. Open systems will fare better because they are outward looking and better able to work with other systems. Open source systems, with their collaborative nature, are at an advantage as they will more likely have a breadth of solutions available. Where there is a gap, new functionality can be easily incorporated. The architectures of open system will also make extensions easier to implement and incorporate.

"Openness" can be seen in two ways. The first refers to the licensing of the code. Open source systems will necessarily be open in this sense. However, in a wider sense, openness also refers to the ability for the system to be easily implemented and extended. Is the system open to being extended and improved. Solutions claiming to be open need to be evaluated in this light. 

Drupal's place in the DXP world

Drupal has earned its place as a leading enterprise CMS which excels at content management and integration. These have always been its strong points and as such it is extremely well placed to form a key part of a DXP. The Drupal ecosystem (hosting, code, support, knowhow) is capable of ticking many of the boxes for what makes up a DXP. 

Looking back at the Gartner list, it is apparent that there are key pieces missing:

  • Personalization and context awareness
  • Customer journey mapping
  • Customer data management
  • Orchestration
  • Architecture and platform design

These missing pieces should be seen as an opportunity for the Drupal community as they point the way to how Drupal, as a product, can be improved to remain future proof.

Interestingly, Drupal is namechecked by the composable DXP article:

Currently, most DXP products are too large and not task-oriented. This prevents organizations from managing them easily, or changing them quickly. DXP capabilities are not easily decomposable or replaceable. However, several DXPs do have extensibility built in, via robust APIs, extension frameworks (such as Drupal’s module framework) and/or app marketplaces. These can be the starting point on the journey toward composability.

Drupal is therefore seen as being well placed to take a central role in the composable DXP ecosystem due to its APIs and extensibility.

The future

There will no doubt continue to be increasied interest in DXPs from businesses looking to build integrated platforms. The rise of the composable DXP as a concept should see many smaller vendors with more open platforms starting to make an impact in the space. For Drupal, improvements in the areas of personalization and experience orchestration should see it build on the strong foundations it already has. This should see a range of DXP options becoming available as activity increases in the area.

Oct 11 2021
Oct 11

This has seen a drive to add personalization features to automated marketing campaigns as well as in the content management system (CMS). CMSs have therefore evolved to accommodate this need, moving from platforms to manage and serve content to also service digital experiences which are personalised. This has seen the emergence of digital experience platforms (DXP) which need to orchestrate the personalization process. Related buzzwords in this space include "Content as a Service" (CaaS) and "headless" or "decoupled". 

Personalization trends

A look at Google Trends for "content personalization" emerged from a low base around 2013 to hit a peak around 2020. Interest in the term is still elevated today, however it does seem to have reached its peak. This perhaps represents attention moving to other hotter topics such as DXP and CDP.

This trend has also been seen in the Drupal community. The concept of "Web Experience Management" first appeared in the Drupal community around 2011 and was popularised by Dries in his keynote at DrupalCon Portland in 2013.

This marked the beginnings of the shift in the community. On particularly visionary presentation was given by Dave Ingram, Delivering Hyper Personal Digital Experiences Within Drupal, where he set out a recipe for personalization which covers all of the bases for personalization techniques as well as a sensible approach for iterating and improving over time. 

The concepts of segmentation, engagement and personalization were beginning to take hold. However, prior to 2015 there was only the occasional article or presentation on the specific subject of personalization. From 2015 interest certainly picked up, with more Drupal personalization presentations being given at Drupal conferences and camps. In 2020 in was a very hot topic at DrupalCon, with many presentation given from a variety of Drupal agencies.

Early personalization efforts were based around tracking a logged in user and serving a personalized experience to them based on their properties. Users could signal their interests by explicitly selecting them on their user profile or signing up to various groups. CRM systems could also be integrated with Drupal to sync user profiles of known users. In this light personalization is relatively easy as the user is known and already a customer. Recent developments have been around personalizing for anonymous users and this has lead the need for a more decentralised approach to tracking and the delivery of the personalised experience. This need to personalize for unknown users could be considered as the main driver for the activity in the space since 2015.

A lot of this interest has been driven in the emergence of Acquia Lift (now Acquia Personalization). This product saw a lot of promotional activity from Acquia which, whilst being targetted at potential customers, was also helpful in educating the Drupal community around the need for personalization and also some of the approaches that could be taken to achieve that. This lead to the development of modules based around user context and the conditional serving of content.

That is not to say the rest of the community has sat idly by. There are a number of Drupal agencies which have picked the concept up and integrated it into their practices, driving the way they communicate with clients about the needs for personalization and how it can be achieved. In the presentations below there is a fairly even split between strategy and technology. It is refreshing to see that the problem of personalization has been seen as multifaceted with a strong emphasis on research, planning and design, as well as the technology to achieve it.

This has lead to the development of a number of open source projects in the Drupal space which can be used for personalization projects. Some of these are simple client side solutions, sometimes adopting a decoupled approach to serving the personalized experiences. We have also seen increased integrations with CRMs, marketing tools and CDPs to serve the personalized content.

Conclusion

This article is a roundup of most of the events and presentations which have helped shape the way Drupal practitioners approach the conceptual and technical solutions required for Drupal personalization. If you have any items you think I have missed in the roundup below, please add them to the comments and I will look to incorporate them in. 

What is next for personalization in Drupal? Time will tell.

Oct 02 2021
Oct 02

The Personified module provides a flexible way to utilise the strengths of Drupal to deliver a personalized experience for anonymous users of Drupal websites. It is able to take data from user profile context, stored in localstorage, and use this to query a JSON endpoint for data. That data is then transformed using a clientside templating language, such as Handlebars, to present the content to the user. 

In non technical language, Personified provides a way to personalise Drupal web pages for anonymous users. It is very much akin to the Smart Content module, except here we query for data, rather than showing individual blocks.

The requirement

At Morpht, we developed Personified as part of our efforts to uplift the personalization capabilities of Drupal. We were impressed with the design of the Smart Content module, allowing for client-side context to be used to drive the user experience. However, we wanted the option of being able to query data from  Drupal (or elsewhere) to retrieve the data. We wanted to use Drupal as a content hub for serving personalized experiences and to deliver those experiences based on a query rather than 'if, then, else' logic.

The solution

In order to make this work the following ingredients were needed:

  1. The storage of user profile data in localstorage
    • The generation of that data (not covered in this article)
  2. A client side solution for:
    • retrieving the context
    • querying an endpoint with the context (with a default fallback)
    • transforming the data with Javascript
    • displaying the data back to the user.

We created Personified to handle the second part of this problem, handling the state, query, transform and display.

Along the way, we needed to solve the transformation part of the problem with a client-side solution. In order to find a pluggable solution, we developed the JSON Template module which is able to transform JSON to HTML using a variety of backends. We implemented a Handlebars transformer as the initial plugin as this provided us with a simple and well-known solution.

How does it work

The best way to demonstrate this is with a screenshot of what the block edit screen looks like:

The screenshot demonstrates the various ingredients which were mentioned above:

  • Data is gathered from localstorage
  • An enpoint is defined with a querystring parameter
  • A default fallback option is available for when localstorage is empty
  • A JSON Template is selected to transform the data.

In this example, the user's 'season' is used to delete a season promo from the Drupal CMS.

You can see all of this in action on our Convivial Demo site. I encourage you to explore that site to see how Drupal can be personalized for anonymous users. (We built this site with Convivial CMS.)

May 16 2021
May 16

A little while back, almost two years ago, Dries Buytaert wrote an interesting thought piece on the sustainability of open source projects such as Drupal. He reviewed the ways different actors engage with open source projects, dividing them into two camps, the Makers and the Takers. The makers build and create, providing benefit to the wider community and society. The takers are able to benefit from this creative process whilst not suffering any of the costs of the creative process, allowing them to gain a competitive advantage.

The difference between Makers and Takers is not always 100% clear, but as a rule of thumb, Makers directly invest in growing both their business and the Open Source project. Takers are solely focused on growing their business and let others take care of the Open Source project they rely on.

In order to demonstrate the difference in outcomes for makers and takers, a payoff matrix was provided. It shows that if everyone contributes there are shared advantages, however, if "takers" decide not to contribute, they will win out as they do not bare the costs of contributing to the project.

  Company A contributes Company A doesn't contribute Company B contributes

A makes $50
B makes $50

A makes $60
B makes $20 Company B doesn't contribute A makes $20
B makes $60 A makes $10
B makes $10


At the time the article did have an impact on me as it was an attempt by Dries to bring outside concepts to help understand the Drupal project and where it might be heading. Thinking such as this leads to informed ways of conceiving a future for projects such as Drupal and how they might shape themselves to thrive. 

The article examined concepts such as the Prisoners’ dilemma, public goods, common goods, free riders, the tragedy of the commons. Following on from conclusions by Hardin and Olson, the core problem for Dries was that “groups don't act on their shared interests”. How can a group organise to avoid the 'free rider' problem? Dries focused on a conclusion from Ostrom who writes “For any appropriator to have a minimal interest in coordinating patterns of appropriation and provision, some set of appropriators must be able to exclude others from access and appropriation rights.” The conclusion was that “Takers will be Takers until they have an incentive to become Makers.” These ideas have driven some changes being implemented at the Drupal Association, such as contribution credits and organisation accreditation and membership.

These thoughts have also been influential at Morpht, the Drupal agency where I work. We have adopted a set of foundation principles for the company. One of the key concepts is that we are Makers and Creators and value contributing to the Drupal project and the community. In practice, we have built internal systems to incentivise and reward everyone in the company to contribute back where they can. Outcomes of the process include a rise in commits to the project and a more open approach to how we share our code. We are also financial contributors to the project, supporting the Drupal Association as a supporting partner.


Green bearded altruism

Recently an intriguing video popped up in my stream “Simulating Green Beard Altruism” by Justin Helps. It is expertly researched, explained and visualized. It really is worth watching, so go on, I’ll give you a few minutes to take a look.

[embedded content]

For the uninitiated, myself included, Richard Dawkins coined green bearded altruism in his book The Selfish Gene. It represents a way for one actor to signal to another that they are altruistic. If altruists can recognise other altruists, they are able to direct their altruism at them and increase their chances of survival.

"I have a green beard and I will be altruistic to anyone else with green beard". 
Richard Dawkins, The Selfish Gene

The video took this concept and ran some simulations on how Altruists and Cowards fare under different conditions and rules. This setup relates closely to Dries' post around the sustainability of open source ecosystems.

The Makers (Altruists) build the system and positively benefit the whole. The Takers (Cowards) benefit more because they do not suffer the costs of maintaining the system. In this scenario, the Takers (Cowards) win out and thrive.

But what happens when the Makers (Altruists) only share the benefit with other Makers? They succeed and the Takers (Cowards) are less successful. The simulation, as set up, provides a possible way forward for open source projects such as Dupal. If you are a good actor and only reward other good actors, positive results will flow and will continue to flow.

The final simulation in video throws in the curveball of actors being dishonest. Sometimes a Coward will pretend to be an Altruist, tricking them into helping them. What do we find?

  • Altruists who do not signal their altruism tend to die out in a system where Altruists are rewarded. They are labelled as Suckers - doing useful things but not being recognised to their detriment.
  • Cowards who masquerade as Altruists are successful, reaping the benefits and suffering none of the costs. 

And most concerningly, in a world where actors can hide their true identity, even the Cowards, acting as Cowards, have success. The Altruists in their various forms cannot compete. This final outcome is depressing. What is the point of being altruistic in a world where others can just take advantage? Being altruistic is not enough if actors are gaming the system against you.

In a wider context what could we learn from these simulations? If an individual or organisation is indeed a Maker, they should signal this to others and be rewarded or recognised. This runs against the desire to be modest, but it does appear to be a sensible thing to do. Conversely, those gaming the system should be called out and somehow excluded.

A 2021 update

Dries has recently returned to the Maker and Takers concept in the Q&A following the Dries note at DrupalCon Global 2021. The video is yet to be released to the public but will be added here once available. Those of you with access to the video on Vimeo can take a look at 7:12 - 9:06. Dries says:

"Open source is a public good, meaning that everyone can use it. If I use a public good it doesn’t exclude you from using it either… One interesting thing is that leads which are essential to business are not a public good. It is actually a common good, meaning that there is a scarcity to it. If I win a deal, you can’t win that deal. So one of the things that we can do is make sure that the leads, the potential customers, …  go to those who contribute. 

There is something there that I really believe strongly. If we can route leads to organisations that contribute we will maximise the number of contributions Drupal will get and I believe that these organisations are often better serviced too because they get help from those organisations that actually help build Drupal. Its kind of a win win.

Sometimes I feel that we are afraid to talk about these topics because they may be somewhat controversial, but there is so much more that we can do."

These comments take the thinking to the next step. There is a recognition that if payoff for altruistic behaviour is financial, this will lead to further contributions. The way to achieve this is through the “routing of leads”.

Applying it all to Drupal

The main takeaways from the above appear to be:

  • We should be encouraging altruistic behaviour because it benefits the project.
  • Altruists can still benefit if they receive the benefit from other altruists.
  • Real financial benefits need to flow to the altruists if they are to be motivated.

So what is happening in the Drupal space?

The Marketplace

In recent times the Drupal project has reorganized itself to encourage more good actors. This has largely been done through the mechanism of recording contributions and promoting those who have contributed the most. Drupal agencies have been encouraged to support staff in contributing. The results are reflected in the Drupal Marketplace. The system gamifies contributions, motivating Drupal service providers to contribute more to move up the leaderboard.

It appears that following aspects have been valued:

  • Commit credits to core, contrib and other issues.
  • Publishing of case studies.
  • Financial support of the Drupal Association.

There are ongoing efforts to broaden this out and to further incentivise contributions from individuals and organisations.

Increase the exposure

The marketplace system does represent a huge step forward in demonstrating the contributions made by the various providers. It is like an X-ray into who is doing what in the community. It does suffer from a number of shortcomings:

  • What exposure does the marketplace have to potential clients?
  • When a client is looking to engage a Drupal agency, are they referring to the marketplace? If they are not, then the real financial benefits may not flow to the agencies. Then, the system is  just a game between the players. It benefits Drupal for sure, but does it benefit the players?

In order to be effective, the marketplace needs more exposure to end clients so that the 'routing of leads' can be improved.

The little guy

The Drupal Marketplace currently ranks organisations on an absolute scale according to credits in absolute terms. To my knowledge the rankings are not normalised by organisation size.

It would be interesting to see what the results would be if the rankings were normalised by employee count. We would then be able to see who the biggest altruists were, dollar for dollar. This would give more incentive to smaller organisations to contribute so they could better signal their altruism. 

Advertisements on drupal.org

It is possible to advertise on drupal.org in a variety of positions. You may be familiar with the prominently displayed ads for private companies which are displayed on the bottom of pages. These ads are visible across the whole of drupal.org and benefit from more exposure than being on the marketplace.

Open up the ad space

This form of promotion is obviously much more 'private' in nature, designed to promote the interests of the advertiser rather than that of the project. It is a way for actors to promote their own self interests against others and the interests of the project as a whole.

The advertising space is currently a vehicle to promote the interests of a selected few, rather than all of the altruists in the system. It creates a feeling of 'them vs. us' in the community to have certain players promoted in this way and not others. The Drupal Association should consider how this space could support all contributors. I would suggest that revenue for this space could still be maintained whilst opening it up proportionally based on contributions.

Sponsorship and visibility

Supporters of the Drupal Association receive promotion at Drupal conferences and in other ways. A sponsorship entitles a service provider to a variety of advantages, the main one of which is the promotion through badges and logo display at conferences. 

Continue the drive for more members

Supporting the project in this way can be more appealing than one-off sponsorship at individual conferences. Supporters do get quite a good level of exposure at conferences and this is a good way to signal altruism to other members of the community.

This system appears to be working reasonably well. Supporting the project is a good thing and sponsorship is a direct way to do it. The big challenge here is to encourage the non-subscribers to jump on board. If all individuals and companies did this in just a small way, the financial security of the Drupal Association would be assured. This has always been the case. As a community we should be encouraging this where we can. So if you are not yet a member or sponsor, you know what do do :).

Burnout and community funding

It is not uncommon for certain prolific or influential contributors to leave the community. A common reason would be burnout because of the stress of sustaining an important codebase in their spare time. It is not sustainable for them to do so, especially when there are many demands for support of features.

Most recently we have seen Jacob Rockowitz, maintainer of the Webform module, post several articles discussing this situation. The result was the decision to move to a sponsored approach using the Open Collective platform. This model encouraged users of the Webform module, the Takers to help contribute and become Makers by continuing the development of the module.

Closing the altruism loop

The way this message has been communicated has, in my opinion, been done in a very positive way. If you look at the Webform module page, there is a call for support through code, patches and reviews. And for those who cannot do that, financial options exist. This is a direct move to increase the altruism in the community and to close the loop between altruists helping other altruists.

Who sponsors Drupal?

In his blog post: Who Sponsors Drupal, Dries makes the point that companies or smaller agencies support most Drupal development. The bulk of the codebase is supported by actors who no doubt have an active interest in open source and Drupal, as well as the financial and technical ability to help support the codebase.

Deep pockets and shallow expertise

What can be done to broaden this out further, so that we can get more Makers and few Takers?

Larger entities with lower technical capability should have an easy way to fund development of code. Once the altruism feedback loop is strengthened, there should be a bigger drive for Takers to become Makers. We need a path for this to take place.

This is not the same as sponsoring the Drupal Association, i.e. infrastructure, promotion and governance. This is about funding code development, strengthening the code and functionality of the project and making it more attractive as a technological proposition.

In order for this to take place, two things need to happen:

  1. The Drupal Association, or some other body, needs to be ready to take on this responsibility.
  2. A method of dividing the resources needs to be determined.

At the moment, the efforts in this area have been ad hoc. There are notable examples of large companies contributing to initiatives which push the project forward. However, in order for this to be scalable, it needs to be done in a more systemic manner. It may be that the Drupal Association doesn't want to take this responsibility on - that is fair enough. If so, how might it be done?

The Webform module has turned to Open Collective. A recent article from Rachel Norfolk, makes a similar suggestion. Maybe the DA is taking a look at what is going on with Webform?

Shallow pockets and deep expertise

And what of the individual developer? The one who loves Drupal, loves open source and dedicates their time to improving the project. What if they are not supported financially by a larger organisation? These people are the lifeblood of the project and they need to be supported. It makes a heap of sense to harness their creative energies and support them financially to progress the project. 

If we can find a way to put these Makers together with those with the funds and the desire, great strides will be taken.

I would therefore suggest that this seems to be the most practical approach for closing the altruism loop and progressing the project.

Conclusion

While I make some suggestions, I believe Drupal is in an excellent position. The community is broad and deep and there is a lot of desire to keep the momentum going. There are also some excellent systems in place to help reinforce this such as recognition for contribution and the marketplace.

I have argued for the following:

  • Continuing with the credit system and the marketplace;
  • Increase the prominence of the Drupal Marketplace to outsiders;
  • Promote organisations who are punching above their weight in terms of contributions per employee;
  • Continue the sponsorship approach with the community encouraging membership to the Drupal Association;
  • Reconsider the advertising space on the Drupal Association as something for Makers rather than being private for a select few;
  • Develop a system to bring Makers together with Takers who have deep pockets.

The biggest challenge, where we can make the most gains, lies in bringing the big Takers into the fold and supporting talented individuals who do not otherwise have support, and with that closing the altruism feedback loop and increasing the chances for the project to grow and improve.
 

Mar 26 2021
Mar 26

It seems that with each passing year there is a new paradigm for how content can be arranged and organised in Drupal. Over the years a number of approaches have moved in and out of being in vogue: Panels, Displya Suite, IPE, Bricks and Paragraphs to name a few. Some change has been positive, providing leaps forward in flexibility or control. Other developments have not lived up to their promise.

In February 2021 I presented a new module, Layout Paragraphs, to the Sydney Meetup. The slides and video have been provided below. This presentation demonstrates Layout Paragraphs in action and how offers some advanced layout options for Paragraphs. Conceptually it is similar to Layout Builder in many respects, however, it performs its magic on the Node Edit page, integrating with the natural content editing environment for site editors.

Layout Paragraphs offers a new way forward for the following reasons:

  • Editing happens on node edit, rather than the layouts page. Better for editors.
  • Paragraphs can be placed into Layout regions to bring more flexibility to Paragraphs. This is similar to what Bricks was doing.
  • Nicer UI for Paragraph selection.
  • Nicer UI for Paragraph display - no need for Preview view mode any more.

A bit of history

It is worth reviewing a little history to see where Layout Paragraphs fits in. The presentation takes a look at some of the popular combinations over the years and gives them over all scores, weighted by functionality and editor experience. Here is a spoiler of what is covered in the video:

Recipe Year Score Pure template 2010 65 Display Suite 2010 58 Panelizer 2012 69 Panelizer and Paragraphs 2014 73 Panelizer and IPE 2016 39 Panelizer, Bricks and Paragraphs 2017 63 Layout Builder and Blocks 2018 70 Layout Builder and Paragraphs 2019 78 Layout Builder, Layout Paragraphs, Paragraphs 2021 81

The scores were calculated from a weighted average of various aspects of the techniques: flexibility, control, editor experience, etc. Watch the video for the details.

Conclusion

You can see that Layout Paragraphs is the latest in the line of approaches and that it is scoring quite well. A recioe based around Lout Builder, Layout Paragraphs and Paragraphs seems to work quite will. Layout Builder remains the domain of the sitebuilder, using it to define the basic layouts for the page. With Layout Paragraphs, a new set of simpler layouts can be used by the editor for their paragraphs.

I think that the approach holds a lot of promise moving forward and it is good enough for Morpht to be considering it as a standard part of our editor toolkit. All up we have found the module to be usable and a definite improvement on editor experience. We are adopting it into projects where we can.

Watch the video and let us know what you think in the comments below.

Watch the video

Feb 26 2021
Feb 26

How does it stack up

Those of you who work with Drupal, you are probably familiar with the combination of using Search API with a search backend such as MySQL or Solr. A pluggable architecture makes Search API a good choice for indexing content in Drupal.

For a long time MySQL and Solr were the popular choices. MySQL was an easy choice as performance was good and results were OK. For those working with large datasets and many concurrent facets, Solr made more sense. Most Drupal hosting companies provide it as a service for this reason. As the search market has matured, other backends have become available, including one for Sajari.

The table below compares these three options and highlights the strengths and weaknesses of each.

Feature Database Solr Sajari

Separate service

No

Built into Drupal.

Yes

Drupal hosting companies provide a Solr as SaaS.

Yes

Sajari is available as a SaaS.

Full text search

Yes

Yes

Yes

Facets

Yes

Yes

Yes

More like this

No

Yes

A useful feature for providing item recommendations based on similarity.

No

Result quality

OK

Good

Very good

Performant

Partial

Slow with many filters over large datasets with facets.

Yes

Yes

Easy install

No

Requires a module such as Search API Database to push data across to Solr.

No

Requires a module such as Search API Solr to push data across to Solr.

Yes

We can configure Sajari in the Sajari UI to run from metadata on the page. Sajari provides an embeddable widget.

We recommend the Search API Sajari module approach.

Search API Integration

Yes

Search API Database module

Yes

Search API Solr module

Yes

Search API Sajari module

Federation

No

No

Yes

A site parameter can be passed into the index for easy filtering.

ReactJS components

No

No

Yes

Interface is faster than Search API as server round trips are not needed.

Result tracking

No

No

Yes

Built-in metrics understand page trends and poorly performing keywords to help you see what searches led your users to individual pages, or which content visitors are searching for but can’t find.

Reporting

No

Reports can be set up in analytics software.

No

Reports can be set up in analytics software.

Yes

Sajari provides logs and charts of search requests.

Autocomplete - suggestions

Yes

Extra module can be installed.

Yes

Extra module can be installed.

Yes

Synonyms

No

No

Yes

Libraries of synonyms can be uploaded via Sajari UI.

Typos

No

No

Yes

Support for misspelled words.

Boosting

Limited

Limited

Yes

Advanced rules can be defined on certain plans.

Machine learning

No

No

Yes

Sajari will learn which results are more or less relevant, promoting the best results to the top.

Pricing

Free

Database comes with Drupal hosting.

Included

Solr server comes built in with typical Drupal hosting.

Free and up

Starts free for smaller sites and then increases.

https://www.sajari.com/pricing

Summary

An easy, low cost search solution.

A more scalable solution with handy features such as “more like this”.

A fast system with smart results helpful for those looking for synonyms, results boosting, tracking and reporting.

Sajari is a viable alternative for clients who are looking for more insights into how their audience use the search on their site and more control over the delivery of the results. This is the case for content driven sites as well as for ecommerce configurations where preferences play a big role.

Integrating Sajari with Drupal

The Sajari Widgets

It is possible to implement Sajari search into any website without the need for the addition of modules or custom code in the backend. Sajari provides a set of widgets which will allow search to operate without the need for much technical knowledge.

Firstly, a Javascript tracking code will allow for “instant indexing”. When a user visits a page, the code fires up and tells Sajari about the page. Sajari can then visit and index the page to update its index. This approach is simple to set up but has its downsides - freshly updated or deleted content will not make it into the index immediately. If this is a concern, then using Search API Sajari, below, would be an alternative.

Secondly, Sajari offers a tool in the admin UI to define a search form and results. It covers things such as the search query, filters, tabs, result counts and result display. It is very easy to configure. The result is a snippet we can embed onto your search page. A set of ReactJS components drive the search and return results in lightning speed, leading to a good experience for users.

Drupal Module: Search API Sajari

For those looking for a tighter integration between their Drupal site and Sajari, it is possible to use their API to push updated content across. The Search API Sajari module , authored by the developers at Morpht, provides a backend to the venerable Search API module  This will update Sajari when content is updated on your Drupal site.

The main advantages of this approach are:

  • Content is indexed instantly, even when no one views it;
  • Deleted content is removed from the index immediately;
  • The tools within Search API allow for the fine tuning of the various fields;
  • There is support for sending a site name across in the result, allowing for federation of results.

Drupal Module: Sajari

The widgets provided by Sajari offer a quick way to get up and running with a search page. However, there are some limitations in the way they work. At the time of writing (early 2021) the widgets did not support the definition of facets.

In order to overcome this shortcoming, Morpht developed a ReactJS library which sits on top of the components provided by Sajari. It has quite a number of configuration options for queries, result counts, filters, tabs and facets. It even has the ability to customise the results through the provision of a callback function which can convert the JSON result to HTML. This code is available at Sajari Configuartor.

The Sajari module makes use of Sajari Configuartor to power the way search is implemented. The module provides a block for defining how the search will operate. The configuration is then passed through to the Sajari Configurator and the UI and results are then shown.

The Sajari module also makes use of the JSON Template module which allows for different handlebars templates to be defined by the themer. These templates can then be selected by an editor during the block creation process. The select template then forms the basis for the callback which is passed into the Sajari Configuartor. The result is that editors can select how to show results. There is no need to alter the ReactJS templates which are in the library.

A recipe

If you are looking to get up and running with Sajari, we recommend this process:

  • Sign up for a free account at Sajari;
  • Set up an initial collection in Sajari, but add no fields;
  • Install JSON Template, Sajari and Search API Sajari;
  • Configure Search API Sajari with your collection details in a new Server;
  • Define your Node Index and assign it to the Sajari server you have just created. The schema will be updated automatically in Sajari with the changes you make in Drupal;
  • Confirm that content is being indexed properly;
  • Add a Sajari search block to your search page and configure it. Be sure to use the correct pipeline and get the field names right;
  • Test the search and confirm it is working.

Conclusion

Sajari is an up-and-coming search provider offering a new breed of search which can utilise human behaviour to improve the results it shows. It's useful for content heavy and ecommerce sites which have a strong need for good search results. There are now integration modules for Drupal to get you up and running with Sajari easily.

Is Sajari right for you?

If you currently have a Drupal site based on a different engine and are interested in what Sajari can offer you, please get in touch with us to discuss it further.

Aug 10 2020
Aug 10

At Morpht, we have been busy experimenting and building proof-of-concept personalisation features on Drupal 8. We started off with content recommendations as one of the cogs in a personalisation machine. Recombee stood out as a great AI-powered content recommendations engine and we decided to develop some contributed modules to integrate it with Drupal.

We have implemented three modules which work together:

  • Search API Recombee indexes content and pushes it across to Recombee.
  • Recombee module tracks users and displays recommendations.
  • JSON Template module defines a plugin system for transformers and templates and controls how JSON can be transformed client side.

The video below is a demonstration of these three modules and how they combine to deliver a power recommendations system capable of providing content to anonymous and logged in users alike. 

[embedded content]

Download the slides from the presentation
(PDF 785KB)

Let's talk

Find out how personalisation can help you increase audience engagement and lift user experience.

Contact us

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web