Apr 18 2016
Apr 18

email symbol on row of colourful envelopesWhat would a website be if it couldn’t send emails, even if just for password resets? Running your own mail server is a huge hassle, so many developers instead use a third party service to send transactional emails like password resets, new user welcome messages, and order summaries. One of the most popular services, in part because of their generous free tier, is Mandrill, owned by MailChimp.

In case you might have missed the announcement, MailChimp is changing Mandrill to be an add-on to paid MailChimp accounts, thus eliminating the generous free tier. We’re big fans of MailChimp and use its mailing list service for our own announcements, (hey, why not join that list if you’re not already on subscribed?) but a full MailChimp account isn’t going to be for everybody. They’ve already shut out the ability for new subscriptions, but if you’re a PHP developer who does things like put off your taxes until the last minute (American customers have three extra days this year, but that’s today), you’re probably sweating the April 27th deadline.

Many people also know Mandrill by reputation and will need options in the future. For you, we’ve put together this list of viable transactional email alternatives with PHP and major PHP application support. Joomla! and MODX support SMTP integration natively, so you’ll just need the SMTP configuration options from your chosen provider. If you want to use a provider’s web API, see the PHP options below.

Cal Evans did an unscientific Twitter survey to see what options people were migrating to:

If you are moving off of @mandrillapp, what are you moving to?

— Cal Evans (@CalEvans) March 29, 2016

SparkPost

MailChimp’s announcement notes that SparkPost has agreed to take on existing Mandrill users and honor Mandrill’s pricing for them. Fortunately, SparkPost has PHP users covered: there is an official PHP API library. There is also a Drupal module, but unfortunately it seems to be 7.x only at this writing and is only a sandbox project—you’ll have to install it via git. Drupal 8 users should be able to use the official API library with Composer. WordPress developers are in more luck: there is an official WordPress plugin. SparkPost provides a guide for Magento devs using the SMTP Pro extension. SparkPost also has one of the most generous plans we’ve seend, with 100,000 free emails per month, though you can not exceed that limit without upgrading ahead of time.

SendGrid

A long time option for PHP users has been SendGrid. (Full disclosure: SendGrid has sponsored our php[tek] conference in the past, but is not a current sponsor.) They have an official PHP API, installable via Composer. While there is a 7.x-only Drupal module, SendGrid recommends Drupal users use the SMTP Authentication Support or Swift Mailer modules in its documentation. Both the officially-recommended modules support Drupal 8 at least in the development releases of each module. Magento is also supported through the SMTP Pro extension. WordPress devs can install the official plugin. SendGrid doesn’t list a free tier on their pricing page, their “Essentials” plan start at $9.95 for 40,000 emails per month.

SendinBlue

Many devs I know have spoken highly of SendinBlue. They offer a WordPress plugin, (7.x only) Drupal module, and Magento extension. They also have an official PHP library. Their free tier is limited to 9,000 emails per month with no daily limits, however the messages will include SendinBlue branding.

Amazon SES

Amazon’s transactional email service is affordable but not as easy to install and configure for newbies. They have an official PHP library through the AWS PHP SDK. There is a third-party Drupal module for 7.x users. Similarly there’s an independent WordPress plugin. There is a USD 99 paid extension for Magento.

Mailjet

Mailjet offers a PHP API wrapper, a WordPress plugin, a 7.x-only Drupal plugin, a Joomla! extension, and a Magento plugin. The free tier is capped at 6,000 emails per month and 200 email per day. The first 30 days include a premium trial which allows users to explore segmentation, testing, and compare campaign performance.

Mailgun

Mailgun has a PHP SDK installable via Composer. There is also a WordPress plugin, a 7.x-only Drupal module,  and a Magento extension. The first 10,000 emails each month are free, after which you pay a tiered price based on monthly volume.

Postmark

Postmark offers a PHP API library, installable via Composer and available on Packagist. There is also an official WordPress plugin. There is a community-supported Drupal module (you guessed it, 7.x only) and Magento extension. There are also many other community modules for PHP frameworks. If you sign up to try it, the first 25,000 emails are free. After that, you can buy credits to send emails starting at $1.50 per thousand emails.

Conclusion

Which of these services you use depends on your needs, price sensitivity, and how much specific support you want for your platform. If I’ve missed any services with good PHP support, please let us know in the comments!

Image Credit: RaHuL Rodriguez on Flickr

Mar 02 2016
Mar 02

drupal-graphic

With over 200 new features, Drupal 8 is officially here! Drupal is one of the world’s favorite open source content management platform.. and it just got even better. Here are some of the ways that Drupal 8 will benefit various groups of people.

Developers:

  • Configuration management – In prior versions of Drupal, most of the configuration was stored in the database.  The problem with this is that it is very difficult to keep track of versions of the configuration when it changes.  The only way to get configuration out of the database was to use a combination of modules such as strongarm and features to export things from the database into code.  This was often time-consuming and error prone.  Now with Drupal 8, configuration management is built-in so that carrying over configuration from development to production is a breeze.
  • Web services – Drupal 8 can now be used as a data source to output content as structured data such as XML or JSON.  This means that Drupal 8 can strictly be used as a back-end while the front-end could be developed completely separate with a framework such as AngularJS or Ember.  In other words “Headless Drupal” capabilities are now built-in instead of requiring various addon modules and lots of custom development.  

Content Editors:

  • Bundled WYSIWYG editor – Drupal 8 is the first version of Drupal to come with a bundled WYSIWYG editor.  Previously it was possible to add one of many different editors into Drupal but the setup was often time consuming and confusing.  Additionally there were so many choices that some users felt lost about which one to choose.  Over time CKEditor has become the most popular WYSIWYG editor for Drupal and now it is included out of the box.
  • In place editing – In addition to having CKEditor bundled in with Drupal 8, the Spark initiative is taking WYSIWYG concept a step further with true in place editing.  This would give editors the ability to change content, menus, etc. directly from the front-end view of the site without having to navigate to an admin page on the back end.  More info about the Spark initiative can be found here:  http://buytaert.net/spark-update-in-line-editing-in-drupal

End Users:

  • Mobile First – Previous versions of Drupal allowed developers to create responsive themes.  However some modules were not 100% compatible with responsive layouts.  Now with Drupal 8 all themes are mobile first which means that all community modules will be compatible with responsive layouts.  Additionally the default Drupal admin theme will be mobile friendly which should improve the experience for editors who want to author content from mobile devices.

Accessibility and Languages – Drupal 8 now has extensive support for accessibility standards including the adoption of many WAI-ARIA practices.  This will make content structures easier to understand for people with disabilities.  In addition to the accessibility improvements Drupal 8 now has multi-lingual support included.  Drupal 8 has the capability to reach more users than any previous version of Drupal.

Oct 18 2012
Oct 18

For Drupal 8, we want to bake REST support directly into the core system. It's unclear if we'll be able to go full-on hypermedia by the time we ship, but it should be possible to add via contributed modules. For the base system, though, we want to at least follow REST/HTTP semantics properly.

One area we have questions about is PUT, in particular the details of its idempotence requirements. For that reason, I'm reaching out to the Interwebs to see what the consensus is. Details below.

For now, we're confining ourselves to RESTful access to entites, Drupal's main data object. Every entity has a "native" URI at http://www.example.com/$entity_type/$entity_id, such as /node/5. We're currently looking at JSON-LD as our primary supported serialization format.

Naturally for creating a new entity, we cannot use PUT since the entity ID is auto-generated. For that, POST to a /node/add page of some sort, which returns the URI of the created node. But what of updates?

Idempotence

At first blush, PUT /node/5 seems like an obvious thing to do. Simply PUT a JSON-LD representation of a node to an existing URI and it gets overwritten with the new version, no muss no fuss. The problem is that Drupal, being a highly extensible system, cannot always guarantee no-side-effects when that happens.

My understanding is that idempotence in HTTP is not absolute. For instance, GET, HEAD, and PUT are idempotent, but "incidental" side effects such as logging or statistics gathering are OK and not a violation of their idempotence. RFC 2616 has this to say on idempotence:

Methods can also have the property of "idempotence" in that (aside from error or expiration issues) the side-effects of N > 0 identical requests is the same as for a single request. The methods GET, HEAD, PUT and DELETE share this property. (RFC 2616 section 9.1.2)

I'm not clear on "the side effects are the same" qualifier. Does that mean it can be a repeat of the same side effect, or a net-0 effect?

There are two places where this becomes relevant for Drupal.

Versioning

Many types of entity in Drupal (hopefully all soon) support revisioning. That is, when saving the entity instead of overwriting the existing one a new draft is created, which, sometimes but not always, becomes the new "default" version. Previous versions are available at their own URIs. That can change at any time, however, subject to user configuration. Also, more recently we've been allowing forward revisions, that is, creating a new version that is not yet the default version, but will be.

How does that play into idempotence and PUT? If a new revision is created, then repeating the PUT is not a no-op. Rather, it would create yet another revision. The spec says:

A single resource MAY be identified by many different URIs. For example, an article might have a URI for identifying "the current version" which is separate from the URI identifying each particular version. In this case, a PUT request on a general URI might result in several other URIs being defined by the origin server. (RFC 2616 section 9.6)

That seems to imply that a PUT to create a new revision is OK. However, what of forward revisions? If you create a new revision, but don't set it live, it means that a PUT followed by a GET on the same URI will not return the value that was PUT. It would return the previously existing value.

Put another way:

PUT /node/5

{title: "Hello world"}

Results in:

GET /node/5

{title: "Hello world"}

and

GET /node/5/revision/8

{title: "Hello world"}

And that's totally fine, by my read of the spec. However, what of:

PUT /node/5

{title: "Bonjour le monde"}

Results in:

GET /node/5

{title: "Hello world"}

GET /node/5/revision/8

{title: "Hello world"}

GET /node/5/revision/9

{title: "Bonjour le monde"}

Is that still spec-valid behavior? And if not, does that mean that any system that uses a Create-Read-Archive-Purge (CRAP) model instead of CRUD, or that supports forward revisioning, is inherently not-PUT-compatible? (That would be very sad, if so.)

Hooks

The other concern is Drupal's extensibility. When an entity is saved, various hooks/events fire that allow other modules to respond to the fact that the node has been saved. Those hooks can do, well, anything. While in the vast majority of cases they will do the exact same thing every time a new update is made or a revision is saved, that's not a guarantee. They may take a different action depending on the values that were just saved for the entity. Or they may take a different action on different days. Or they may generate IO, such as sending an email or saving additional database information, or triggering a cache clear, or launching a nuclear warhead. (Unlikely, but the API allows for it!)

Since those hooks MAY do things that are not idempotent, does that mean that we MAY NOT use PUT, since it must be idempotent? Or does it mean that we simply document that hooks SHOULD NOT do non-idempontent things and call it a day?

In any event, that's our situation. We want to properly leverage the HTTP spec and REST principles here, but I fear that Drupal's very extensibility makes that semantically impossible. I am hoping I'm wrong, but in any event I turn the question out to the peanut gallery for consideration.

Can we PUT up with it?

Share with

Nov 01 2011
Nov 01

(At BADCamp, several people were asking me to explain what the heck the Web Services core initiative was trying to do, so I got to practice my elevator pitch. This is essentially that pitch in written form.)

Drupal today is very page-oriented. Every request that comes in is responded to with a full HTML page. It is possible to return something else, and finally in Drupal 7 there is, sort of, native support to do so with delivery callbacks, but by and large any non-page response is an after thought at best and a hack at worst. The entire system is built around the assumption that we're returning an HTML page. Why else would we still load the theme system and form system for an auto-complete callback?

In the past, that hasn't been a major issue. The web was a series of pages, in practice, and Drupal is one of if not the most flexible page-generating machine on the web today. Drupal 7 is, arguably, the pinnacle of this page-oriented world.

Just in time for that world to be fading fast.

Tomorrow's web

In case you've been living under a rock, the web is in the midst of a massive upheval. HTML and HTTP, tools built for linking static pages together, are transmogrifying (somewhat painfully and with lots of torn shirts along the way) into a layer of the Internet atop the Internet. Everything is not a page, destined for a known browser. Everything is a response to a request. That is all that can be assumed.

Responsive design and mobile-friendly sites may get all the press, but that's the least of our worries. That's "just" theming (he says as the themers get pitchforks). The web is changing in even more fundamental ways.

Consider a site using Varnish. That can do more than just serve a full page from a RAM cache. Edge-Side Includes (ESI) let it cache most of the page but call on the server to render just certain portions of the page (say, those that are customized to a given user) quickly and stitch them together on the fly. That's next year's high-volume web sites.

Consider FaceBook. They switched to parallel rendering of the page through sub-requests over a year ago. They call it Big Pipe. The general idea, though, is breaking the page up to generate and deliver in segments, in parallel. This is last year's high-volume web site.

Consider My.FCC.Gov. Watch the video. That is a Drupal site. (Can you believe it?) The page is actually being built in the browser, by the client, in Javascript. The individual chunks of the page are being retrieved by multiple parallel Ajax requests. That required a fair bit of custom code and Services work, which means lots of full bootstraps.

Consider Backbone.js, which moves nearly all logic into the browser and treats the server as simply a dumb data store. Panels in the client, anyone?

Consider Web Sockets. Create a persistent bidirectional connection between the client and server and push raw data back out to the browser, not on request but immediately as it becomes available, to be presented to the user in whatever way Javascript dictates.

Now, try to combine those. No pages, just HTML fragments, JSON strings, raw persistent data sockets, etc. In practice, the server may almost never generate an entire page from html tag to html tag.

That's the future. That's also not something Drupal today can handle.

Show some initiative

Supporting that sort of next-generation workflow requires changing the way we approach page handling. It requires treating an incoming request as just that, an abstract request, which may be returning a page, or a fragment, or JSON, or establishing a persistent connection, or any number of things we can't even imagine yet. It requires leveraging all of the HTTP protocol, not just the tiny subset that browser HTML lets us access.

Fortunately, work is already under way. We also can get outside help. One of the reasons I am so excited that we've begun adopting Symfony2 components in core is that the Symfony2 framework is built on exactly those principles. By leveraging the work of others, in good open source fashion, we can save time both in coding and in architecting basic components. I expect we will pull in more Symfony2 components as time goes on, as reinventing the wheel in these critical areas would accomplish nothing but ensure that we don't get to the real work.

Today, Drupal loads nearly all of its code base for every request. We need to allow most of the system to lazy-load on demand instead. Fortunately, we can now do that by converting code to PSR-0 compatible classes in core.

Today, Drupal figures out what function will handle a page based purely on the path alone. We need to enable it to look at the full HTTP request and then some to do the complex routing we need. Or perhaps this is another area we could punt to Symfony2 components to save time and debugging.

Today, Drupal figures out what sort of response it is going to return after it's already done the work. We need to make determining what the response will be before we've build a single render array element. That depends on not just the request itself, but on the Drupal-derived information from it. That is, the Context of the request.

Today, we lay out a page as a blob of content with smaller blobs of content arrayed around it. We need to move to a model where one block on a page is no more special than any other, and we approach the page outside in, not inside out. Fortunately we have the model of Panels to follow.

Today, when laying out a page in Panels we have to deal with the Panels UI, which is widely recognized to be not up to the task. Fortunately some smart people are already trying to think through how that could be re-imagined to make it more approachable for site builders and content editors alike. It will take a lot more directed thought and planning,though, to re-invision the UI.

Web sockets may not be straightforward to implement in PHP, but we're already trying to work out how we could enable them from Drupal. (If you've done any development with Web Sockets, please share your experience!)

See something you could help with? We'll see you in the issue queues.

Bookmark/Search this post with

Oct 17 2011
Oct 17

RESTful is the native API of web browsers. When you put some website’s address into a browser, that’s an implied REST expression called a “GET” of the resource at that address. In response to that GET request, the web server on the other end returns a web page. However, REST is much more than requesting the resource (data) at some address. Just like using any website, one is able to Create things, Retrieve them afterwards, perform Updates to them, and eventually Delete them. That Create -> Retrieve -> Update -> Delete cycle is called “doing CRUD” (really), and that in a nutshell is what creating and using a RESTful system is all about.

In the “early days” of the Internet, when someone wanted to make a printer or some other machine programmatically communicate over the Internet, more complex systems with names like SOAP, XMLRPC and AMF were used to handle that communication. Then around the year 2000, a smart guy named Roy Fielding pointed out that the web itself was an API and these complex systems were not only a bother to create and work through, but needless because what they were offering was already built into the web itself.

Now, Drupal is a content management framework whose essential purpose is to create a website of some sort. You are probably familiar with some websites including information from other websites, such as a Twitter feed or Facebook friend status. This including of other website’s information can be accomplished “the old, hard way” via scraping the page that normally shows this data, via SOAP/XMLRPC or that communicating of information can be accomplished “the new, shiny RESTful way” which takes less effort and by it’s nature is universally supported.

This is essentially machine-to-machine communications, and is how an iPhone/iPad/Android/game console/printer or virtually any other device communicates on the Internet. This is using REST.

The topic of our Developing RESTful Web Services and APIs class is how to use the new Services 3.0 module. Services 3.0 provides an API for Drupal module developers to create a REST API of their own design. Using Services it’s most basic level, one can install and enable a set of built in APIs that will allow remote programmatic administration of that website (thru secured authentication of course!) What I detail is the more ambitious creation of a series of programmatic resources, demonstrating how to create a useful API of the type that could support anything capable of programmatic control.

For example, you could have a site where you provide users “alerts” when items the users have shown interest become available. Additionally, those alerts can be seen on Facebook and in an iPhone app. Your Drupal site providing these alerts can use a custom module and Services to publish an example.com/alerts/uid API that the Facebook and iPhone apps use to manipulate that information. Using REST for this communication is more lightweight on your web server because the Facebook and iPhone app logic is able to request that information specifically, rather than an entire web page where they would scrape off the data they want, or get that information through the more complex SOAP or XMLRPC methods.

This is also how one could have a mobile and/or console game’s universal high scores and user community forums present in-game as well as simultaneously on the web. One could have a Drupal site using Services to publish an API for “doing CRUD” with high scores and interacting on the community forum. For the Drupal site, business as usual, but for the mobile and/or console game they are getting that data via a RESTful communication with the Drupal site. For the mobile and console game developers, this type of communication is easy. And through Services, it’s also easy for the Drupal developer.

Further, I use Services to create “custom on-demand digital products” at a Drupal/Ubercart store, with that on-demand creation taking place on a remote cloud server. I walk through how that is setup, and my architecture for scaling the environment should my custom digital products go viral.

And the best part, REST is how one creates Web Services. What are Web Services? They are the future of everything. Really. Remember up there where I mention machine to machine communication? Web Services are the creation and publishing of APIs to “do CRUD” with things that people care enough to pay real money for access. Such as access to commercially controlled data like music, movies, or even stock and bond research and trading. Web Services is taking REST and wrapping it in commercial activities. Some event venue could publish a ticketing API, and then charge ticket brokers for access. The list of possibilities are endless. And that list is expected to be how all commercial services in the future will be conducted. (Make your eyes really big when you read that last part :)

In summary, our Developing RESTful Web Services and APIs class covers how to create an API with Services 3.0, as well as how to support your API customers (who may not be using Drupal) (and who may be dumb as rocks) how to successfully use your site’s API.

At the class, I give out and walk through an API Shell, which took over a month to create. Next, students begin creating their own API with architectural guidance by myself and other Larks trainers. To facilitate this, an example API’s is step-by-step created, with time for the students to implement their component in their API as we go along. For individuals or groups with a specific project they are planning or have in development, a 3rd day of additional guidance and support is also available.

Oct 03 2011
Oct 03

To celebrate today’s release of Services 3.0 for Drupal 6 & 7, we sat down for an interview with Blake Senftner, a Services expert who is providing our Developing RESTful Services and Web APIs training in Los Angeles on November 3, 4 & 5.

We’re also offering 10% off this training: just use coupon code SERVICES10 at checkout. The discount code expires on October 15th.

Christefano: What was it that got you interested in Services?

Blake: Well, to be honest it’s because of Services and Drupal’s other APIs that I’m using Drupal at all. I come from a 3D animation background — I did both feature films and console video games — and I needed the ability to create Web APIs for a distributed computing environment for my own startup.

C: When was that?

B: I started working with Services 6.x and the XMLRPC Server, getting the first version of my distributed environment operating with that. It worked fine and I wasn’t looking forward to the move to RESTful until a buddy at Disney Interactive sat me down and explained REST to me.

With XMLRPC, you create remotely callable functions and the logic feels very “atomic” in that you’re doing one function at a time, with no “system” or architectural framework. Within a RESTful structure, though, you’re creating and working with “resources” — which are very much like objects in an object oriented sense. Where XMLRPC is working in data, REST works on “things” that have a complete CRUD lifecycle — create, review, update, and delete operations. Just that simple CRUD framework provides a structure that makes working in REST conceptually easier.

C: Give an example of how using REST makes things easier.

B: Okay, an example would be with my XMLRPC service, I had a function that could create a 3D model. That was all it did. The same thing in REST by default supports creating, editing, deleting and updating. Just because that comes with REST and is part of the concept of REST, you automatically think in lifecycle frames of references. With an XMLRPC, all you think of is “I just want this one item.” There’s no architecture in that. There’s no lifecycle in that.

Oh, I also had a client that saw my earlier XMLRPC API and wanted something exponentially more sophisticated. Envisioning that in XMLRPC was causing me to consider a CRUD framework for XMLRPC, but luckily my buddy at Disney had that talk with me. That’s why I switched to REST.

C: When did you start working with Services 3.x?

B: I was digging through the sources, examples and issue queue as soon as a usable 3.x version was available. That was probably around September or October of last year. There may have been working versions earlier than that, but that’s when I started. The maintainers of the Services project are amazing and overworked and I hope the training we’re doing helps alleviate their workload.

C: How long did it take you to get your first “hello world” working?

B: Oh, geez. [Blake checks his email.] It looks like it was just shy of 4 weeks before I had satisfactory handshaking and then another 6 weeks before I had a full CRUD resource working with Relationships, Actions, Targeted Actions, and Authentication. Of course, I was also developing my client’s project at the same time, but the Services work was a continual focus because we had so much riding on it working.

That’s a big reason behind my offering this training. I speak with Drupal developers all the time at my Droplabs co-working space, and very few of them have the time or clients with the vision to commit the time to learn Services. Services is the key behind offering “software as a service”, as well as backends for mobile apps and console games.

C: We’re really excited to be doing this training. What do developers need to know in advance, and what do they need to bring to the training when they sign up?

B: You probably need to know at minimum how to create a basic Drupal module. To make anything interesting, you probably want to know enough to create a Forms API-driven interaction. It could be creating a custom content type or anything that exposes forms from your module. If you know that, you have everything you need to jump into Services with gusto.

Bring a laptop with a local development installation or a way to remotely access a Drupal installation where you’re a server admin and can install and deploy modules. It can be Drupal 6 or Drupal 7. Your choice.

C: Thanks for answering all my questions!

B: Sure, I hope it’s helpful. I look forward to developing with you!

Exaltation of Larks is providing this 3-day training (2 days of classroom-style training with an optional third day of hands-on mentorship on student projects) on November 3, 4 & 5, 2011. If you have any questions, visit us at http://www.larks.la/training or contact us at trainings [at] larks [dot] la and we’ll be happy to talk with you. You can also call us at 888-LARKS-LA (855-527-5752) with any questions.

Sep 22 2011
Sep 22

Tomorrow is the last day of Summer but the Drupal training scene is as hot as ever. We’ve scheduled a number of trainings in Los Angeles this Fall that we’re excited to tell you about, and we’re happy to publicly announce our training assistance program.

First, though, we’re sending out discount codes on Twitter and Facebook. Follow @LarksLA on Twitter, like Exaltation of Larks on Facebook or sign up to our training newsletter at http://www.larks.la/training to get a 15% early bird discount* toward all our trainings!

Los Angeles Drupal trainings in October and November, 2011

Here are the trainings we’ve lined up. If you have any questions, visit us at http://www.larks.la/training or contact us at trainings [at] larks [dot] la and we’ll be happy to talk with you. You can also call us at 888-LARKS-LA (888-527-5752) with any questions.

Beginner trainings:

Intermediate training:

Advanced trainings:

All our trainings are $400 a day (1-day trainings are $400, 2-day trainings are $800, etc.). We’re excited about these trainings and hope you are, too. Here are some more details and descriptions.

Training details and descriptions

   Drupal Fundamentals
   October 31, 2011
   http://ex.tl/df7

Drupal Fundamentals is our introductory training that touches on nearly every aspect of the core Drupal framework and covers many must-have modules. By the end of the day, you’ll have created a Drupal site that looks and functions much like any you’ll see on the web today.

This training is for Drupal 7. For more information, visit http://ex.tl/sbd7

   Drupal Scalability and Performance
   October 31, 2011
   http://ex.tl/dsp1

In this advanced Drupal Scalability and Performance training, we’ll show you the best practices for running fast sites for a large volume of users. Starting with a blank Linux virtual server, we’ll work together through the setup, configuration and tuning of Drupal using Varnish, Pressflow, Apache, MySQL, Memcache and Apache Solr.

This training is for both Drupal 6 and Drupal 7. For more information, visit http://ex.tl/dsp1

   Drupal Architecture (Custom Content, Fields and Lists)
   November 1 & 2, 2011
   http://ex.tl/ccfl1

Drupal Architecture (Custom Content, Fields and Lists) is our intermediate training where we explore modules and configurations you can combine to build more customized systems using Drupal. You’ll create many examples of more advanced configurations and content displays using the popular Content Construction Kit (CCK) and Views modules.

This training is for Drupal 6. For more information, visit http://ex.tl/ccfl1

   Developing RESTful Web Services and APIs
   November 3, 4 & 5, 2011
   http://ex.tl/dwsa1

Offered for the first time in Southern California, Developing RESTful Web Services and APIs is an advanced 2-day training (with an optional third day of additional hands-on support) for those developers seeking accelerated understanding of exploiting Services 3.0 to its fullest. This is THE training you need if you’re using Drupal to create a backend for iPad, iPhone or Android applications.

This training covers both Drupal 6 and Drupal 7. For more information, visit
http://ex.tl/dwsa1

Training assistance program

In closing, we’d like to tell you about our training assistance program. For each class, we’re setting aside a limited number of seats for students, unemployed job seekers and people in need.

For more details about the program, contact us at trainings [at] larks [dot] la and we’ll be happy to talk with you. You can also call us at 888-LARKS-LA (888-527-5752) with any questions.

* Our early bird discount is not valid toward the Red Cross First Aid, CPR & AED training and 2-year certification that we’re organizing. It’s already being offered at nearly 33% off, so sign up today. You won’t regret it and you might even save someone’s life. ^

Apr 11 2011
Apr 11

At DrupalCon Chicago, Dries announced that the development process for Drupal 8 would be a bit different. Rather than a vast dog pile of efforts to improve Drupal in ways big and small, Drupal 8 will feature a number of major "core initiatives". These initiatives highlight major areas of work that represent not just a patch or three but major changes to Drupal's plumbing. Each initiative will have one or two initiative leads who have the ability to coordinate and make decisions relating to that initiative while working closely with Dries. In a large sense, it is a way for Dries to scale; Rather than Dries having to keep track of 1000 ongoing conversations himself, initiative owners can coordinate related changes while Dries coordinates the initiative owners. It also gives a clear indication of what work is happening and what to expect out of Drupal 8.

The first initiative for Drupal 8 has already been announced; Greg Dunlap will be leading the charge to overhaul Drupal's configuration system to provide more robust, performant, and deployable configuration and change management. That will be critical for Drupal's future as we push further into the corporate and enterprise sphere, as well as enabling more robust and unified configuration handling in the first place.

Today, I am pleased to announce Drupal's second core initiative: The Web Services and Context Core Initiative (WSCCI).

What web services means

Web services are an area that is exploding in recent years, and will only continue to become more important. What does that mean? In a nutshell, a web service is a web site responding to a request not with an HTML page but with data intended for another program to consume. That could be as mundane as an RSS feed, as complex as a headless SOAP application server, or anything in between.

While Drupal has had web services support for years via the Services module, it has been a bolt-on to the core system rather than a piece of core functionality. Drupal today, at its core, is a monolithic HTML-page-based CMS framework. Most parts of the system assume it will be delivering a blog-like HTML page, with a main content area and secondary content areas, to a human viewer on a desktop or laptop computer. Anything else is, architecturally, an afterthought.

Those afterthoughts, though, are rapidly becoming the norm. More and more pages now require asynchronous requests back to the server to request tiny bits of data or to submit forms without refreshing the page. Mobile devices 4" and smaller with Internet connections will soon outnumber desktops and laptops with big, wide monitors. Tablet computers are the fastest growing segment of the market, but have a wide variety of form factors and sizes. Extracting raw data from a site in order to analyze or mash it up is already a mandatory feature for many sites. Soon, HTML pages will be the least of Drupal's worries.

For Drupal to truly embrace the future web, we need to fundamentally rethink how Drupal response to an incoming HTTP request. We need to treat HTML pages as what they are: A particularly common form of REST response, but only one among many. To that end, Drupal needs to evolve, and quickly, from a first-class web CMS into a first-class REST server that includes a first-class web CMS.

Fortunately, we already have a roadmap to do just that.

The plan

Phase 1: Context

Drupal derives a lot of information out of the HTTP request. There is a lot of information there, too, and from that information we extract all sorts of information: What node is currently being viewed, what page callback to use, who the current user is, what the language of the site should be, and dozens of other important facts. However, there is no consistent way to do so. Every system and every module must invent their own way of doing so, usually with some global function or global variable. In order to build a more robust REST server within Drupal, we need to unify that contextual information in to a coherent and integrated system.

The solution here is to wrap the HTTP request into a single, extensible context object. That context object will act as a central gateway to information coming from the client (who may not be a user at all) as well as information we derive from it, such as the current node, current user, and so on.

By providing an integrated, extensible system for contextual information, we can build a more coherent, robust, flexible response system on top of it.

Phase 2: Plugins

Drupal is, by design, a very extensible system. Hooks galore allow additional code to tie into the system and add functionality in a myriad of places to control Drupal's actions and output.

What hooks do not do, however, is offer a way to exchange functionality, or change one implementation out for another. That is the realm of plugins, an area where Drupal has no existing standard. Core itself handles swappable, configurable components in a half dozen different ways. Various contrib modules have invented their own systems, with the most widely used being the Chaos Tools Suite but many other one-off implementations still in the wild.

The solution here is to develop a robust plugin system into core, and leverage it in core. Not only will that make core itself cleaner, more robust, and easier to learn and extend, it will provide a mechanism that contrib modules can leverage as well, just as they do with hooks. That in turn makes Drupal easier to develop for, easier to extend, and more consistent. And a single, good framework is easier to optimize and make faster than a dozen incompatible systems.

This is a well-understood problem space, so we will be trying to leverage existing work both in Drupal's own ctools suite and in other, similar open source projects.

Phase 3: REST Services

Both of those tools, context and plugins, are in a sense detours. They do not directly provide Drupal with web service capabilities. Rather, they are building blocks from which we will be able to build a new, robust underpinning for Drupal to support web services.

In the current approach, Drupal examines an incoming request URL and routes it to the appropriate callback function, then themes it as a page. In a REST server approach, Drupal should be able to examine the full HTTP request. A pluggable logic routine can then route the request to a pluggable response handler, which may be configured to respond with an HTML page but could just as easily respond with a JSON object, an Atom feed, or submitting a form and then firing off a redirect. All of those options become "first class citizens". Then, Drupal doesn't just support web services: Drupal is a web service.

Phase 4: Layouts

That doesn't mean that our HTML page generating system can't improve. In fact, those same underlying tools should make it possible to build a new, plugin-based, context-aware page rendering system as simply one response controller among many. We already have a solid conceptual model to follow here: Drupal's own Panels module, which has shown itself to be a very robust and effective way to build much more complex pages than Drupal natively supports. With a more context aware front-end system, we can also more easily tie into other important technologies such as ESI, or even implement them ourselves within Drupal.

The butler did it

If all of that sounds familiar, it should. It is essentially the Butler project renamed, an informal initiative I have been kicking around and trying to drum up support for since DrupalCon San Francisco and before, and for which we laid out a battle plan at DrupalCon Copenhagen. For that reason, I am happy, humbled, and more than a little scared to announce that Dries has asked me to head up the Web Services and Context initiative for Drupal 8.

The road ahead

All of that sounds like a lot of work, and it is. It's a ton of work. Fortunately we already have a head start, as many people have been working on designing these solutions over the past year. Work has already begun on the unified context system, for now still called "Butler", as a Drupal 7 module that we can forward-port to Drupal 8. Design work is already under way for the plugin system, with several volunteers researching 3rd party systems to see how they handle pluggable systems and the Drupal ctools team bringing their extensive experience to bear on building a robust, forward-looking solution. For phase 3, we have the experience and institutional knowledge of the Services module to draw on.

We also have the Configuration core initiative, which will be instrumental in enabling us to have access to the early-bootstrap configuration data we will need to do all of the above in a performant way.

Does this sound interesting? Want to be part of Drupal: The Next Generation? Read up on the architecture that has already been developed, join the conversation, and join us in the issue queues. Also watch this space for future updates as work progresses. I'll be posting a more complete tech reference in the g.d.o group in the next few days.

I'll see you in the issue queues.

Bookmark/Search this post with

Apr 19 2010
Apr 19

In the following week we've developed an android application to manage drupalcon oriented information.
Currently we only have the sessions info in place but our vision to the next drupalcons is to add twitter based activity streams, relevant map info and to become a one stop shop for your mobile experience using android.

To find the app search for drupalcon in the android market.

There is a very comprehensive mobile version of the drupalcon site which has more features built in - it's optimized for webkit and iphones yet it works well with android as well.

We're very excited about providing android native applications to our customers and we'd be if you (or your customers) need native android app support tailored to work with Drupal - try to find us in Drupalcon.

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web