Upgrade Your Drupal Skills

We trained 1,000+ Drupal Developers over the last decade.

See Advanced Courses NAH, I know Enough
Sep 10 2020
Sep 10

Let's decouple Drupal - Well, what exactly?

When it comes to decoupling, it turns out there are many options on how to decouple. Not only are there many technology choices in choosing the right frontend framework and APIs involved, the questions become also more foundational: Which system should be in charge of handling different aspects of the application, e.g. routing, placing blocks or authentication?

So before moving on, we need to clarify what exactly we want to achieve:

Clarifying goals: Why to decouple?

For us, the main reasons are:

Independent frontend development

By separating the frontend from the backend, frontend development can happen completely independent of the backend. So frontend developers do not have to know Drupal development, but can focus solely or their part: the frontend. That way, it's much easier to find and onboard frontend developers.

Easier performance optimization

While efficiently lazy-loading CSS or JS assets of a page is quite hard to do with Drupal, modern frontend technologies handle this with breeze and simply lazy-load pre-built asset chunks as needed. Furthermore, the frontend can better optimize the page loading experience (and the assets needed per-page) since the frontend has the knowledge of what exactly is needed by which frontend component.

Re-usable frontend components

Re-usable frontend components - including asset dependency management - are a solved problem in the Javascript world and all the various Javascript frontend frameworks provide that. Additionally, tools like Storybook make it really easy to build up your own component library.

Modern frontend and UX

By building upon modern frontend tooling we can provide a more app-like UX to our users, handle client-side route changes and only reload and repaint what's needed when navigating pages. The web page can become and feel more app-like, or even be turned into a Progressive Web App that does more than providing an offline copy of the site!

Challenges faced

This sounds all good - but there are lots of challenges that must be well-considered:

Re-inventing the wheel

By throwing away Drupal's frontend, you also throw away all the existing integration Drupal provides with its own frontend. Besides the more obvious, like handling forms and logins, features like contextual edit links, previews or the layout builder just do not work out of the box anymore. So one has to develop alternative solutions for all those features - if needed. Depending on the solutions choosen, this can be a lot of work.

Server-Side-Rendering and Hosting

For delivering a decent user experience and being friendly to robots (Google) some sort of pre-rendering is needed on the server. So this needs extra thought and comes with new infrastructure requirements (Node.js) that, come at a cost, and must be considered when choosing where to host.

Cold cache performance

When the Drupal backend is connected via JSON API or GraphQL a first, uncached page load typically needs various kind of data, quickly ending up in multiple API requests to the backend. Since all of those need to be resolved on a cold-cache page hit requests possibly end up really slow and lead to slow first page loads.

Coupling of the frontend to the backend

When the frontend talks to Drupal via its default JSON or GraphQL APIs, the frontend needs to know about Drupal's data structure. Field names, their types and entity relationships are no Drupal interna anymore, but become part of the contract between the backend and the frontend, limiting the possibilities to develop them separately. One could implement custom JSON endpoints or GraphQL Schema to mitigate and fine-tune that.

Our solution: Rendering into Custom Elements markup

We figured that in order to reach the mentioned goals, we can use a soft approach to decoupled Drupal in order to keep more of Drupal's features available. The basic idea is that Drupal keeps rendering pages by providing the main page content as custom elements (markup). Then, this simplified markup can be picked up by client-side frameworks like Vue.js to handle rendering the elements client-side.

An example of custom element would be the following markup for a Twitter paragraph:

<pg-twitter src="https://twitter.com/bmann/status/1283090375742091264">
  <h3 slot="title">#Driesnote suggesting to ship an official component for React and Vue for managing menus</h3>
</pg-twitter>

Note that we use pg as abbreviation for paragraph here. Or a quote paragraph:

<pg-quote>
  <h1 slot="title">a quote from twitter...</h1>
  <p>#Driesnote suggesting to ship an official component for React and Vue for managing menus<br>Ship it in NPM as those devs expect.</p>
  <span slot="author">Boris Mann</span>
</pg-quote>

Generating this markup, is exactly what the Custom Elements module does. The module renders the data of all visible fields either as attribute to the custom element tag, or as nested tag with a slot attribute.

As custom elements are part of Web components specification, web components provide a good fit for rendering the content client-side, but other client-side frameworks like Vue.js or React.js can pick up the data and render it as well - so there is plenty of choice for the best-suiting client-side solution.

So what about the page template?

Since the header and footer area is pretty static for most of the time on most of our sites, we figured they are best implemented by mostly static components in the frontend. Any dynamic parts, like meta tags or navigation elements can be complemented with data provided by (long-cached) API calls or even computed client-side (for example, the active menu item). If necessary, it's easy to re-render parts of it and client-side frameworks like Vue.js and React can handle updating only the necessary bits when navigating pages.
While the dynamic per-page content is provided by the backend in a single API response per page, the header and footer can be controlled by the frontend. The frontend application takes the custom element content and renders it, next to taking care of updating essential page metadata like meta tags.

Thus, for rendering individual pages, the frontend needs to fetch the main content - rendered as Custom Elements - as well as any page metadata that it needs for rendering the page from the backend. For that, we've implemented the Lupus Custom Elements Renderer module which turns Drupal into an API backend providing exactly that.

Summing up

By rendering the page shell and custom element content in a frontend framework of our choice, we achieve all of our goals stated. Moreover, the custom elements "format" provides a good way of decoupling the frontend of the backend, since any Drupalisms or backend complexities in the data model can easily by translated into a meaningful collection of custom elements. Those elements comprise a well-defined structure representing your site's content elements and transferring the necessary data to the frontend. In the frontend, the custom elements map nicely to Vue/React/Web/... components.

By keeping Drupal's page routing mechanism, we can keep using Drupal's path handling, including useful features like editor controlled path aliases or redirects. Since page responses are handled as whole by Drupal, we optimize and cache full-page responses and leverage Drupal's existing cache infrastructure and optimized cache tags for caching individual parts of the page.

Finally, the approach taken allows us to keep using some of Drupal's existing solutions like, cookie-based authentication for user-specific page responses, the layout builder or even form processing.

Following up

Since there is so many details more to talk about, I'll be following up with further blog posts in the coming weeks, covering the following topics:

  • Selecting a frontend framework. Server-Side-Rendering and hosting options
  • Architecture variants, Authentication, Custom Elements markup & json formats
  • A developer introduction: Creating components, adding Custom Element processors, the relationship to Drupal's render API, Custom routes and optimizing cache metadata.
  • Handling blocks & layout builder, content previews, forms, caching & performance optimizations.

Finally, I'm going to talk more about this stack at the Drupalcon Europe 2020 in my session "Custom Elements: An alternate Render API for decoupled Drupal" - so mark your calendars!

Sep 24 2017
Sep 24

While the Drupalcon webseite has a good few pointers to the well-known major tourist attractions, as locals we'd like to share our knowledge about some of our favourite places with you! So here a few recommendations:

Viennese Wine and Heurige

If you stay for the weekend after the Con, you can join the Vienna Wine Hiking day, which I can highly recommend. There are 3 possible easy hikes through the vineyards with lots of options to stop for tasting gorgeous wine directly from the producers. Furthermore you may enjoy great views of the city even if the wheather is not that great!

If you stay long enough, don't miss it! You can find details and options at https://www.wien.info/en/shopping-wining-dining/wine/wine-trail

If you cannot join the wine hiking day, be sure to visit some Viennese "Heurige" (wine taverns). Good options would be the Schreiberhaus or a little bit closer to the city-center Sissy-Huber.

Otto Wagner Buildings

The famous Viennese Jugendstil architect Otto Wagner (and friends) has left lots of traces back in the city. Apart from some of the subway stations (you won't be able to miss them) we'd recommend looking at the following buildings at least from the outside:

Cafés & Restaurants

Kaffee Alt Wien: An interesting mixture between a traditional Vienese Cafe and a "Beisl" (pub). The food can be recommended too, simple but authentice Viennese dishes, like Gulasch, Schnitzel and a variety of sausages. Although the Kaffee Alt Wien is mentioned in travel guides, it has not lost its athmosphere and is visited by tourists and locals alike.

Flatchers: Great steaks for a reasonable price. There are two restaurants in the same street: A French bistro with georgous French athmosphere and a larger one in American style.

Brunnenmarkt: A local market in one of the lesser known districts, lots of immigrants of south-eastern Europe and Turkey run market booths and Cafés around a nice plaza. You'll find great athmosphere and good food options: Kent, Cafe Ando, Cay Cafe am Yppenplatz

Barfly's: A cuban style cocktail bar with authentic athmosphere and music!

Wolfgang Ziegler

On drupal.org:

fago

Drupal evangelist since 2005, Drupal Core contributor & contrib maintainer, TU-Vienna graduate (Information & Knowledge Management)

Dec 23 2014
Dec 23

Last October Drupal Austria organized a full day long free workshop for getting started with Drupal 8 in Vienna. drunomics sponsored the event and I had the favour of presenting on Coding with Drupal 8. Meanwhile, Drupal Austria has got the recordings of the workshop up, the result: 4 hours of high-quality Drupal 8 training material in German:

[embedded content]

The available tracks are:

Intro - by Nico Grienauer (grienauer)
Sitebuilding Part 1 - by Philipp Melab (pmelab) & Sebastian Siemssen (fubhy)
Sitebuilding Part 2 - by Philipp Melab (pmelab) & Sebastian Siemssen (fubhy)
Theming - by Christian Ziegler (criz)
Coding - by Wolfgang Ziegler (fago)
Entwicklungsumgebung - by Philipp Melab (pmelab) & Sebastian Siemssen (fubhy)

You can find the playlist with all recordings on Youtube here.

Wolfgang Ziegler

On drupal.org:

fago

Drupal evangelist since 2005, Drupal Core contributor & contrib maintainer, TU-Vienna graduate (Information & Knowledge Management)

May 30 2014
May 30

Since the DrupalDevDays in Szeged Entity system constributors have worked a lot on getting the remaining beta blockers of the Entity system done, which are mostely centered around an unified field repository and entity storage improvements; i.e. automatic schema generation for your content entities!

We've been making great progress on the unified field repository front. There are new field info methods that are provided by the entity manager now, which cover the entity base fields as well as configurable or other module added fields. Those new methods replace the old field_info_fields() or FieldInfo methods, which we've been removed from Drupal 8 already. For more details, see this change record.

The work on automatic schema generation is not yet finished though, although we've a huge (300kb) RTBC patch which removes hook_schema() implementations and replaces them with automatically generated schema derived from the entity base field schema. Once that's in, we still have to take care of letting the system handle changes to the field definitions and consequently the schema (issue). Given that it will be possible to automatically generate an entity schema based on a site needs, i.e. we'll have entity storage capable of handling translations for multilingual sites. Actually, this is the primary reaon for getting this feature done that late in the cycle at all. However, besides that, I must say I'm really looking forward to not having to define the schema manually and I'm excited on how easy that will make it to define a new entity type!

I want to thank Acquia who sponsored some of my contribution time during the last weeks and thus enabled us to move on faster. Furthermore, I want to thank tstoeckler, plach, berdir and jessebeach who all did an amazing work on the mentioned issues!

Wolfgang Ziegler

On drupal.org:

fago

Drupal evangelist since 2005, Drupal Core contributor & contrib maintainer, TU-Vienna graduate (Information & Knowledge Management)

May 13 2014
May 13

Today, I've the pleasure to introduce the #d8rules initative, our combined effort to get the Rules module ported to Drupal 8 in time!

#d8rules logo

Drupal 8 is coming...

Fortunately, the first beta of Drupal 8 is coming closer so it's time to make sure all the modules are ported and ready for Drupal 8. Unfortunately, the Rules module - my long term number one contributed module - is about to be left behind. I've worked a lot on Drupal 8 to get critical Entity API improvements like the new Entity Field API (called Entity Property API at the time of the introduction) as well as the new Typed data API done. Of course, the work on core is time intensive, not "done" yet and won't stop - so there is not a lot contribution time left for my contributed modules. :/

So where is Rules for Drupal 8 now?

In short, it's not there yet. While quite a bit of work under the hood and planning has been done already, the main work - porting of the module itself - is still to be done. As Drupal 8 ships with quite a bunch of important improvements, this requires the rewrite of significant parts of the module. In Drupal 7, the Rules module had to solve a lot of hard problems like handling its configuration, configuration translation, dependency tracking, integrity checks and plugins, which all have solutions in Drupal 8 core now. That's awesome, but it requires us to adapt the previous Drupal 7 solutions to work inline with Drupal 8 best practices. However, more than that - the foundational metadata Rules needs is already there as well!

The foundation is (mostly) there

In Drupal 7 the creation of the Entity API module, in particular its Entity Metadata wrappers and the backing Entity Property Information, was driven by the needs of the Rules module. While the Rules module makes it simple to work with data for site builders, the Entity module makes it simple for developers first. Subsequently, the Rules module can build upon the easy API and make it accessible via the rule model and its UI. For Drupal 8, the Entity Field API and its underlying Typed Data API are that easy API upon which the Rules module can build upon. That's great, as it means we have the foundation we can build upon in place - but again, it changed (improved) substantially and will require us to adapt a lot of what's there. However, having the Typed Data API and all the necessary metadata built-in means that the out of the box module and entity type support of Rules for Drupal 8 will be substantially better as well.

Actions and conditions in core

We've got an Actions and Conditions API in core already, so one might think another huge part has been taken care off. Unfortunately, no - those APIs have been created/ported with other use cases in mind, so they do not cater for all the more advanced features Rules users are used to. While I tried to make sure they fit Rules needs as far as possible when they were introduced/updated, they do not fit our needs yet and it might be impossible to make them fit without breaking those APIs. For Rules 8.x we plan to work on improving those APIs (from contrib) as needed first, so we can ensure they fit Rules' requirements. Once we are sure everything works out we'll know what we have to adapt and whether improvements can be contributed to core. Depending on how that works out, we'll see whether we can build up on the core Action and Conditions API or there will be Rules' variants of those APIs (again :(). For more details please see the related issues:

We have a plan

We've quite some work to do to get Rules ported to Drupal 8. klausi and me estimated the task to be additional 1050 hours work (from here). With us, working on it in our spare time besides our other contributions (Entity Field API, Rest module in core, ..) we figured the module won't be ready before sometime in 2015, not unlikely even 2016. That's obviously too late, so we'd love to invest more of our time and work on it during work hours as well, such that we can deliver a ported version in 2014. Our companys cannot afford taking that investment alone, but are up for supporting us and enable us to work on a community rate of € 45/h net cost for the project. You can find more details on the project plan and estimations on our initative site.

Rules needs your help!

If you think the Rules module is a valuable tool and helps you building sites faster, please consider supporting our iniatitive! There is a limited goodie for the 50 first supporters pledging >65$ - check it out. If you are going to Drupalcon Austin and you'd love to help, consider signing up for the #d8rules sprint! We'll get started porting either events, conditions or actions to the new API. Lastly, please help to spread the word! We've got supporter badges that you can embed on your site, and obviously our hash tag is #d8rules!

Video of #d8rules: https://www.youtube.com/watch?v=gEH291mq48Y

 

Resources

Iniative website: http://d8rules.org
drupalfund.us campaign: https://www.drupalfund.us/project/d8rules-support-rules-module-drupal-8
Project page: http://drupal.org/project/rules

External links

Sep 17 2013
Sep 17

Recently, the Drupal 7 Rules module has received some new functionality - mostly thanks to the development of the fluxkraft project. Quite some of them are under-the-hood improvements available to developers, but there is new stuff for site builders as well. So let's have a look at what's new for everyone using Rules and at what's in for developers also:

Configurable bundles for entity events & support for event settings

With Rules 2.4 events may define an event settings form via an optional event handler class. This might be useful in various scenarious, one of which Rules already implements: It's now possible to restrict entity related events to certain bundles immediately when adding an event. Compared to the previous way of adding a generic entity event plus a separate condition to check the bundle (e.g. "After saving new content" plus "Content is of type" condition) it's way nicer to just select the content type directly while adding the event:

Configuring the content type while adding the event

However, the new improvement is more than just a UX improvement - it performes better as well. That is, as when you use the event settings Rules only fires up when the configured event occurs - e.g. "After saving a new article". But if another content type is created Rules won't chime in at all. In most situations that's not a big improvement, but in certain performance critical situations it's quite valuable. That way it opens the door to the creation of useful new events (path-specific rendering events, anyone? :-)

Entity is of bundle condition

This is a new condition that is already available for a while, i.e. since Rules 2.3 - but as it relates to one of Rules' most confusing issues I'd like to highlight it here also. As probably most of you know, when you deal with an entity while conifguring a rule, Rules "sees" only all the properties and fields that it knows to be there later on. That is, if you have a content node, but you do not know the content type, you won't see any fields that are content-type specific. The way around this is that you do a check on the bundle, e.g. you use the "Content is of type" condition or the "Data comparison" condition to make sure the content type is e.g. "article". Given that Rules knows it's an article, you'll get to see the article related fields then.
However, with other entity types - e.g. taxonomy terms - there is no such neat "Term belongs to vocabulary" condition (patch welcome!), so you'd have to go with the generic "Data comparison" condition and know what you have to check for to make the fields appear (the bundle property, i.e. the vocabulary) - or you use the "Entity has field" condition for each field you want to use later on. Quite tedious. Now, the "Entity is of bundle condition" takes away the headache of figuring out what's the bundle of an entity and selecting this as part of a generic "Data comparison". Instead, just go with the "Entity is of bundle" condition, select your entity and bundle (e.g. the content node and "article") and you are good to go.

Event dispatches & task handlers

During the development of fluxkraft we faced the requirement to easily dispatch a bunch of events, depending on some configured events. We realized that this might be a common need for developers who want to support some neat event settings, so I worked together with Sebastien Siemssen (fubhy) to add this feature directly to Rules (actually it's fubhy who wrote all the code). So thanks to this improvements developers may make their event handler implement the RulesEventDispatcherInterface such that the handler gets notified whenever a Rule is configured (or "un-configured") with certain event settings.
Then, often you'd need to perform some sort of periodic checks in order to invoke the necessary events based upon provided settings, i.e. in one of our use cases we need to check twitter search results and invoke the respective event when new results are available ("polling"). The "Rules scheduler" module already solves executing that kind of periodic tasks, but it has been limited to invoke Rule components only. So the newly added support for Rules scheduler task handlers allows one to define custom task handler classes in order to make use of Rules scheduler's scheduling features to fire up custom code logic based upon a pre-defined schedule - such as checking for new twitter search results every 15 minutes. Combined, both features enable developers to implement configurable events like "A new tweet matches a search term" in a breeze.

Class based actions/conditions

In the fluxkraft project we've been using modern (object-orient, PHP 5.3, Drupal-8 style) programming practices in order to ease re-using existing PHP libraries and to simplify the update to Drupal 8 - so does the flux Services Integration project implement class based plugins similar to Drupal 8 (more on that another time). While doing that we realized it's quite a pain having to write old-style callback based Rules actions, conditions and events all over the place, and people new to fluxkraft would have to learn both approaches. Fortunately, I figured that Rules' object-oriented design already treats actions and condition plugin implementations as objects, thus supporting class-based actions and conditions in Rules was a question of adding just ~10 lines of code and some small related changes. (In fact, I'd have preferred class based plugins already when I wrote Rules 2.x initially - back in 2009, however unfortunately I felt the community would not be ready / like that approach at that time). Given that and the addition of a simple discover mechanism Rules allows you to provide new events / conditions / actions now :-) (requires 2.4 or later). For more information about that check the change record or look at a working example like the flux Twitter API module.

Not yet into Rules?

We've got nice new features, but you are not into Rules yet? Not a problem, there is a lot of free learning material around (get started with the handbook) or, if you attend Drupalcon Prague grab one of the remaining seats of our training and learn Rules in a day.

External links

Introducing fluxkraft - web automation simplified with the power of Rules!

Mar 03 2013
Mar 03
Dec 14 2012
Dec 14

As previously announced on the IKS blog I’ve been recently working together with Stéphane Corlosquet on integrating the tools provided by the IKS project to do semantic content enhancements in Drupal as part of the IKS early adopter program.

The Interactive Knowledge Stack (IKS) project is an open source community which got funded by the EU to build an open and flexible technology platform for semantically enhanced Content Management Systems. Thanks to IKS, open source projects like Apache Stanbol, VIE.js or Create.js got started. While VIE.js and Create.js are already on their way to Drupal 8 via the Spark initiative, our focus was on integrating Apache Stanbol with Drupal 7. In short, Apache Stanbol is a java web application that leverages tools like Apache Solr, Apache Tika or Apache OpenNLP to provide a set of reusable components for semantic content management via RESTful web services. On the front-end side, VIE.js (“Vienna IKS Editables”) is the JavaScript library for implementing decoupled Content Management Systems and semantic interaction in web applications.

For leveraging Apache Stanbol with Drupal we send Drupal’s data over to Apache Stanbol’s EntityHub component for indexing, such that it is available to Apache Stanbol’s content enhancer. That means, we can use VIE widgets like annotate.js to send pieces of text over to Apache Stanbol for auto-linking content items indexed to Stanbol, which by default includes DBpedia entities, but could be easily extended by any source providing data in RDF. Next, the VIE autocomplete widget allows for easy tagging based upon entities indexed with Apache Stanbol - regardless of whether they come from DBPedia or from one of the Drupal sites your organization runs!

Diagram

Initially, the plan was to use JSON-LD for sending Drupal’s content over to Apache Stanbol. However, while VIE.js makes heavy use of JSON-LD, Apache Stanbol does not (yet) support JSON-LD when indexing data in its EntityHub component (it does for sending back the text annotation results though). Thus, we used RDF/XML generated by the RDFx module to communicate with Apache Stanbol’s Entityhub. For easy indexing, we created the Search API Stanbol module, which leverages the (great!) Search API for that. Together, with an Apache Stanbol install that is pre-configured for Drupal’s use (link) you can get your own Apache Stanbol instance up and running with Drupal in a minute! Checkout scor’s screencast introducing the module!

[embedded content]

Next, we integrated VIE.js libraries and widgets into Drupal 7 such that one can easily use Apache Stanbol content enhancement capabilities from within Drupal. So we created the VIE.js module, which cares about adding the javascript library in Drupal. We used that to build an autocomplete widget for term reference fields that leverages the VIE.js autocomplete and creates according terms on the fly - while keeping a reference on the original resource. Moreover we integrated annotate.js with NicEdit which is available via the Wysiwyg module, such that text enhancements via Apache Stanbol are always just one click away!

Demo time

Enough with the talking, we’ve prepared a screencast demoing the Drupal integration:

[embedded content]

Of course, there is a demo available online also, so you can try it yourself! Head over to the demo page and give it a test!

Run it yourself!

While integrating all this nice stuff we made sure our work is easy to get up running yourself - so it’s easily re-usable and a good start for any further semantic content enhancement adventures with Drupal. So we’ve not only contributed the VIE.js and Apache Stanbol integration modules, but packaged the whole demo site as a Drupal distribution! You can download Drupal with all the modules needed and everything pre-configured out of the box (just as for Apache Stanbol) - just install Drupal as usual, run the pre-configured Apache Stanbol and you’ve the demo running.

There’s even a feature module which provides you with default content you can start over with. Check out the IKS Content Enhancement Demo distribution! Of course, you are welcome to use its issue queue for reporting problems or for collaborating on improvements or further features!

External links

Semantic content enhancements with Drupal, Apache Stanbol and VIE.js

Dec 14 2012
Dec 14

There must be Rules at the Drupalcon Munich

Aug 21 2012
Aug 21
Aug 08 2012
Aug 08

Why?

While the conversion to full CRUD for entities and classed objects for Drupal 8 made good progress, we’ve not yet reached the goals of fully translatable entities and having a default entity serialization for import/export, content staging and web services. For those points we need to know what’s in an entity! Yes we can look up the fields, but there are also base entity properties and further stuff modules put on entities, which both are no fields so we do not know anything about them. That is what the new Entity Property API is supposed to change: It’s about providing a unified interface to entity properties *and* fields, avoiding the split between fields and non-field properties. So fields will be entity properties working with the same API as well! Then, it’s about being able to instrospect what’s going to be in an entity - regardless whether it has been added by a module or by the user via field UI. But furthermore, with classed entity objects as the basis we can improve the developer facing API and make it easier to access property and field values (I’m looking at you, $entity->field_body[LANGUAGE_NONE][0][‘value’]).

What?

As we’ve outlined in the WSCCI Web Services Format Sprint report we want to have a modern OOP, interface based API for properties. The API should allow for introspection, so that it’s possible to look up the defined properties of an entity type in advance. If you know the Entity property information system of the Drupal 7 Entity module, it’s a bit similar to that, but instead of adding an extra layer above the existing entities it’s about supporting it natively. So there will be no need for entity wrappers - all the easy API and information will be directly available in $entity. See the Entity Property API meta issue for a more complete list of planned features.

The current state

Work on that is currently done in a sandbox, where we’ll flush out the API with a test entity type. Once this works out, the plan is to convert a first entity type and to post a patch for further review to the core queue (here). So let me shortly explain the overall architecture of what we currently have or plan for: The idea is that *every* entity has to follow the following data model: The drupal8 entity data model Thus, an entity consists of multiple entity properties, which consists of one or more property items (just as field API $items), which each has one or more values (just as field API’s columns). You may notice that no language is in there - the idea is that this all just works with a reasonable language default, while it’s possible to get properties in a certain language by specifying an optional language parameter in the property getter. In order to ease access to the properties (in default language) we plan to implement magic getters, so that e.g. the node’s image alt text could be accessed like that: echo $node->field_image[0]->alt; As all entity properties have to follow the structure, it works the same way for others like the node title: echo $node->title[0]->value; $node->title[0]->value = 'updated title'; However, we actually know there can be only one title yes? So you can also do: echo $node->title->value; $node->title->value = 'updated title'; If the array access is omitted, it will just default to the first property item. That makes it easier to deal with single-valued properties, while it makes sure code does not break if a single-valued field gets changed to be multi-valued later on (or vice versa). Next, dealing with entity references should become easier as well, such that the system cares about loading referenced entities for you. E.g. to access the node author’s user name could work like that: echo $node->user->entity->name->value; For that all to work the system maintains so called “property definitions", what are a bit similar to $field and $instance for fields but are there for all properties. The system defines methods to look them up: foreach ($entity->getPropertyDefinitions() as $name => $definition) { echo "Property label: {$definition['label']}"; echo "Property type: {$definition['type']}"; } It will be possible to look up those definitions up front, i.e. without having a concrete $entity with data. So you can use it to generate UI. Of course there will be some more keys in the entity property definitions and we’ll add more keys on the way as necessary. Then the idea is to use the same API for other levels in the entity as well: foreach ($propertyitem->getPropertyDefinitions() as $name => $definition) { echo "Property label: {$definition['label']}"; echo "Property type: {$definition['type']}"; } That’s roughly where we are so far - of course some things will be adapted and change on the way. There is some prototype code in the sandbox, but nothing is ready yet. If you’d like to get involved, contact me and/or dive into the sandbox’s issue queue. If you are Drupalcon, don’t miss the Entity Property API core conversation and join the entity API sprints!

The time is running...

Yes, Drupal 8 feature freeze is not far away (December), so the time is running out. Unfortunately, recently I’ve not been able to invest enough time into this all to move on as quick as it’s needed now, so thankfully Acquia is helping out and funding some of my development on this. This will speed up things now, so expect a patch ready to review before Drupalcon :-)

External links

Wolfgang Ziegler

On drupal.org:

fago

Drupal evangelist since 2005, Drupal Core contributor & contrib maintainer, TU-Vienna graduate (Information & Knowledge Management)

A New Entity Property API for Drupal 8

Aug 08 2012
Aug 08
Jun 03 2012
Jun 03

The last entity type just moved to classed objects, so all core entity types are classed objects using a defined interface now! See the change record for details!

It's done: Rules 2 is out!

Oct 10 2011
Oct 10

Publishing Linked Open Data of Austria at the Vienna Create Camp 11

Oct 09 2011
Oct 09

Nginx clean-URLs for Drupal installed in various sub-directories

Jul 15 2011
Jul 15

Drupal 8: Entities and data integration

Mar 11 2011
Mar 11

Drupal 8: Approaching Content and Configuration Management

Mar 09 2011
Mar 09
Mar 07 2011
Mar 07

eRecruiter 7.x-1.0 BETA 1 released!

Finally, we are happy to announce the first beta release of the eRecruiter Drupal 7 distribution - our e-recruitment solution for enterprises and publishers!

During the last half year we invested lots of time and resources in developing Drupal 7 stuff - like the Entity API, Profile2, Field-collection, the Search API, Rules, the Rules Autotagger module and a Term-level field. Given that developments we finally have everything in place to build our castle - the eRecruiter distritbution!

We have setup a demo site - drupaljobs.epiqo.com - which lists various Drupal jobs from g.d.o. and some random Drupal companies as well as some demo resumes. The jobs are automatically scraped from the original sites by a first version of our currently developed crawler service, and get automatically tagged and categorized based upon the pre-defined taxonomies with the Rules autotagger module. So the jobs are all real, but the resumes are mostly fictional. Feel free to test it out and let us know what you think about it.

It's great to see everything is called into action now to power an actually useful product. So for the eRecruiter we are using Profile2 and Field collection for building resumes, whereas both modules are based upon the Entity API. That said, the Entity API really plays a central role as Profile2, Field-collection, Rules, and the SearchAPI make use of its CRUD API, whereas Rules and the SearchAPI also rely on it to implement exportables and are therefor nicely integrated with Features. Thankfully, the Entity API also provides entity property information for all those entities, upon which Rules and the SearchAPI build in order to be able to seemlessly work with any entity type. With that in place, creating the distribution was simply a question of putting everything together and exporting the configuration in nicely bundled feature modules.

For the stable 1.0 release we plan to also include an application workflow as well as facebook-style "mini-feeds" in order to provide recruiters, applicants but also administrators of a stream of interesting updates. For that we are going to rely on the Message module - which Amitai has re-written for Drupal 7 to be fully based upon entities and fields. Together with using Rules and Views for display, it forms a flexible and powerful solution for logging updates on any event.

Read more about the eRecruiter at epiqo.com/recruiter.

Bookmark/Search this post with

Mar 04 2011
Mar 04

eRecruiter 7.x-1.0 BETA 1 released!

Finally, we are happy to announce the first beta release of the eRecruiter Drupal 7 distribution - our e-recruitment solution for enterprises and publishers!

During the last half year we invested lots of time and resources in developing Drupal 7 stuff - like the Entity API, Profile2, Field-collection, the Search API, Rules, the Rules Autotagger module and a Term-level field. Given that developments we finally have everything in place to build our castle - the eRecruiter distritbution!

We have setup a demo site - drupaljobs.epiqo.com - which lists various Drupal jobs from g.d.o. and some random Drupal companies as well as some demo resumes. The jobs are automatically scraped from the original sites by a first version of our currently developed crawler service, and get automatically tagged and categorized based upon the pre-defined taxonomies with the Rules autotagger module. So the jobs are all real, but the resumes are mostly fictional. Feel free to test it out and let us know what you think about it.

It's great to see everything is called into action now to power an actually useful product. So for the eRecruiter we are using Profile2 and Field collection for building resumes, whereas both modules are based upon the Entity API. That said, the Entity API really plays a central role as Profile2, Field-collection, Rules, and the SearchAPI make use of its CRUD API, whereas Rules and the SearchAPI also rely on it to implement exportables and are therefor nicely integrated with Features. Thankfully, the Entity API also provides entity property information for all those entities, upon which Rules and the SearchAPI build in order to be able to seemlessly work with any entity type. With that in place, creating the distribution was simply a question of putting everything together and exporting the configuration in nicely bundled feature modules.

For the stable 1.0 release we plan to also include an application workflow as well as facebook-style "mini-feeds" in order to provide recruiters, applicants but also administrators of a stream of interesting updates. For that we are going to rely on the Message module - which Amitai has re-written for Drupal 7 to be fully based upon entities and fields. Together with using Rules and Views for display, it forms a flexible and powerful solution for logging updates on any event.

Read more about the eRecruiter at epiqo.com/recruiter.

Restful web services in Drupal 7

Jan 31 2011
Jan 31
Jan 11 2011
Jan 11

I'd like to share some of my thoughts and long-term visions for Drupal 8 and beyond:

1. Full CRUD for the Entity API

In the long-term I want to see the Entity API becoming our main CRUD-API, on which modules may build upon. For D8 I do think for every entity should be based upon a class implementing the "EntityInterface", which provides some simple methods to easily access identifiers, revision ids, labels, uris as well as save(), delete(), .. methods.

2. Improved DX for fields

Now, we have two kind of entity properties: Fields and non-fields. So should one use an entity property or a field?
We have some nice APIs around fields, but they are not built for daily developer usage so programmatically re-using fields is no fun. Still, developers can go without a field for any custom data storage, but then we are loosing all the advantages fields come with - like flexible storage or the awesome module support (which I've tried to solve in D7 via hook_entity_property_info()).

Once we have improved DX for fields in place, developers can easily embrace it and benefit from its advantages.

3. Everything is a field

So why not adopt fields for any entity property? So we could make entity properties easily translatable via the field API and benefit from the improved out-of-the-box module integration and stuff already written for fields, like widgets and formatters . Of course, some fields need to be hidden from the UI then.

4. Storage APIs

With everything being a field, entity data would be scattered around in lots of db tables. Also, it should be possible to use the API to register any remote data object as entity. So we need to have entity-storage and field-storage backends, such that also the remote-data-entity can have fields stored in the local database. Thus, with everything being a field we need to allow developers to delegate field-storage to the entity.

5. Describing data

Also, as of now field types have to describe the db schema to be used for saving. However, the schema API is built for the database system so it has no notion of describing stuff beyond it, like that a timestamp is a date. So maybe the contract between the storage API and the field system should not be the db schema, but an actual description of the data to be saved. I.e. instead of telling the system to save an integer which will be used for saving a node id, tell it that it has to save a reference to a node.
Apart from that, the described data structure is what other APIs built around fields (widgets, formatters) have to use (or should use -> query), just as any developer working with fields. So simultaneously, modules making use of entities and their fields could rely on that information, e.g. to determine all entity references or just to get some data values of a certain type, e.g. textual values for token replacements.

6. Profile2 in core

With 1) in place, it should be rather easy to replace our old profile module with something new built upon entities and fields like profile2. We'll see how profile2 does for d7.

7. Rules in core

I'd really like to work on bringing a slightly simplified version of the foundational API of rules into core, thus re-placing the current action system. However with Rules 2.x the whole API is built around the way of describing data utilized for hook_entity_property_info() as well as on entities. Thus for Rules in core making sense, it would need something comparable in core - e.g. point 5).

Dec 01 2010
Dec 01

After about a year full of work around Drupal 7, Rules & Web services I recently graduated my master studies "Information & Knowledge Management" at the Vienna University of Technology. As finally my thesis will be presented officially with others at this year's epilog, I thought it would be a good time to share my thesis to the public.

The thesis title is "Event-Condition-Action rules for distributed content management", thus I've worked on a re-architectured version of Rules that is able to work across system boundaries, e.g. multiple Drupal installations. That resulted in Rules 2.0 as well as stuff like the Entity API being created - but better, it finally made it possible for me to fully concentrate on working on Drupal - awesome!

Thesis abstract:
For the popular open source Content Management System (CMS) Drupal the Rules
extension module makes it feasible for users to configure reactions on a high level
without requiring any programming expertise. To achieve that, the extension allows
for the specification of reactive rules - or more precisely it leverages Event-Condition-
Action rules, for which the actions are executed when a specified event occurs and
the conditions are met. Finally the ability to create custom reactions constitutes an
opportunity for users to rapidly adapt the behavior of Drupal based web applications.
In this thesis the existing Rules extension module is analyzed and revised in order
to obtain an extensible and reusable solution, for which all identified flaws have been
eliminated. Moreover the module is advanced to work across system boundaries,
such that it can be utilized for the rule-based invocation of web services as well as
for reacting on remotely occurring events. Therefore the solution obtains the ability
to work with arbitrary data structures with the help of metadata, so that the data of
remote systems can be seamlessly integrated based on metadata. Building upon this
capability we present the rule-based utilization of RESTful and WS* web services and
introduce Rules web hooks - a novel approach for the interaction of Drupal based
web applications that exploits reaction rules to enable custom near-instant reactions
on remotely occurring events.

At this point, let me thank my employer epiqo for sponsoring this work, as well as my academic advisors O.Univ.Prof. Dipl.-Ing. Dr.techn. A Min Tjoa and Mag. Dipl.-Ing. Dr. Amin Anjomshoaa of the Institute of Software Technology and Interactive Systems, who already did a great job supporting the Drupalcamp Vienna 2009.

Aug 04 2010
Aug 04

Update 10.01.2011: In the meantime the Entity metadata module got merged into the main "entity" API module.+

Drupal 7 modules working with entities often face the same problems:

  • How to create/save/delete an entity?
  • How to get referenced entities?
  • Which properties are there and how can they be accessed or modified?

This is, what Entity Metadata tries to solve for Drupal 7. It collects metadata from modules, such that it knows how this things can be done and provides API functions for that purpose. There are API functions for full entity CRUD, for determining access, as well as data wrappers that simplify dealing with entity properties.

Metadata for data properties, why that?

You might think, we have fields. Yes we have, but not everything is a field. There are also entity properties, like the node title and author, the term hierarchy and lots of others. Entity metadata collects information about all that available properties - regardless whether they are fields or not - and makes them accessible the same way. For that you have to provide property info via a hook, e.g. this is the info the module provides for books:

<?php
/**
* Implements hook_entity_property_info_alter() on top of book module.
* @see entity_metadata_entity_property_info_alter()
*/
function entity_metadata_book_entity_property_info_alter(&$info) {
 
// Add meta-data about the added node properties.
 
$properties = &$info['node']['properties']; $properties['book-id'] = array(
   
'label' => t("Book ID"),
   
'type' => 'integer',
   
'validation callback' => 'entity_metadata_validate_integer_positive',
   
'description' => t("If part of a book, the unique ID of this page's book."),
   
'getter callback' => 'entity_metadata_book_get_properties',
  );
 
$properties['book'] = array(
   
'label' => t("Book"),
   
'type' => 'node',
   
'description' => t("If part of a book, the book to which this book page belongs."),
   
'getter callback' => 'entity_metadata_book_get_properties',
  );
}
?>

The 'getter callback' is used to get the actual value of the property as described in the metadata - this is important to ensure the data exactly looks the way it is described. Consider a date value: Entity metadata defines that a date is represented as timestamp in UTC as documented, thus modules can rely on that. Similarly it defines data types like 'text', 'integer', 'decimal', 'boolean', 'date', 'duration', 'uri', as well as 'struct' for arbitrary data structures and any entity as registered to the system. Additionally it supports a generic 'list' construct, as well as lists of a specific type, e.g. data of type list<text> is represented as an numerically indexed array of strings.

Apart from getter callbacks there are other optional callbacks, like a 'setter callback' for writing back data, an 'access callback' or a 'setter permission' for establish property-level access control, etc.
Also note that Entity metadata automatically adds in information for any field, as long as it knows about the field type. Thus, if you add a new field type, tell it once how to handle it, and any instance will be supported automatically.

Metadata? I've my own module integration!

Yes, you might have and of course it works. But the point about metadata is to avoid re-inventing the wheel to provide basic information lots of modules need again and again. Instead let people define the information once, but use it a couple of times!
My need for metadata was Rules, however I decided to to go for a general, re-usable metadata format instead of doing it for Rules only. Why? To enable reuse and avoid duplication. That way other modules may benefit from the module as well, in turn Rules would benefit if more modules provide metadata.

The metadata butterfly.

As seen on the figure, many modules beside Rules could make use of the metadata. First off modules that want to export, import or transform data could easily make use of the metadata, but also search modules like the Search API or Services would be a natural fit. Oh, and apart from that Entity metadata also provides tokens for any textual property that has none yet.

At this point, let me mention that the originating idea is from my friend jpetso, who created the fieldtool for Drupal 6. I just took the idea to D7 and entities.

Metadata Wrappers

Entity metadata provides some wrapper objects you may use just to easily make use of the metadata. With the help of the wrappers you can access the metadata, loop over known properties, or just get/set the described data values, etc.

You can find some examples for using the wrappers in the README, but also in the tests.

Oh and the nice thing about the wrappers and the way of describing metadata is, it is not limited to entities at all! Indeed I use it for Rules to deal with basically any data, what is needed is just the appropriate metadata that describes the data structures. E.g. Sebastian has written support for dealing with XML and CSV data based upon the metadata as part of his Google summer of code project transformers. Next, we are going to use that for making remote data accessible too - but more on that at the Rules Drupalcon session in Copenhagen...

Apr 28 2010
Apr 28

Stay up to date, follow me on twitter or identi.ca!

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web