WSCCI at Freeze

Drupal 8 Feature Freeze has come and gone. 2 years of hard work by dozens of people later, it's a good time to pause and take stock of where the Web Services and Context Core Initiative is, and what's left to do.

Short version: Holy crap we did a lot! Holy crap there's a lot left to do!

Drupal 8's first year of development didn't see much in the way of new features or radical changes. A lot of bugs were being fixed, a lot of groundwork being laid, but the really heavy lifting didn't really come in until February 2012. That's when the decision was made to fully adopt Symfony2 Components as the basis for Drupal 8's core pipeline. In hindsight, it was one of the biggest and, I believe, best decisions we've made in Drupal in years.

The work to date

WSCCI basically forked into SCOTCH and WSCCI at that point, and WSCCI rebooted. That means nearly everything that has been accomplished has, realistically, been done just in the past year. Let's see what that is:

1. Adopted the Symfony2 HttpKernel component and model for handling web requests.
2. Rebuilt the routing system to be mostly-decoupled from menus.
3. The new routing system supports routing by path, HTTP method, HTTP vs HTTPS, and basic mime-based routing, as well as pluggable additional filters. That's vastly more flexible than before.
4. The new routing system can very easily be swapped out for a non-SQL backend.
5. A new, more flexible access control system that supports multiple access checks on a single route.
6. A new serialization system to render any entity into a standard format, such as Atom, JSON, or HAL, and all without having to write support for every format, entity, and field combination in existence.
7. A new rest.module, which offers any new-style Entity in the system up as a RESTful resource using any available serialization format. (See Klausi's writeup for more detail on all of the fun things that entails.)
8. REST/serialization support for Views, to easily expose any View as a serialized resource collection.
9. Introduced the Guzzle library to core, a full-featured HTTP 1.1 client that is far more powerful and flexible than drupal_http_request().

Although not strictly part of WSCCI per se, we've also had a knock-on effect throughout Drupal of paying down gobs of code debt, refactoring many core systems to be cleanly injected services, introducing a Dependency Injection Container to help make code more unit testable, and lots of general API improvements. As a side effect, core is far more modular and it's now far easier to swap out entire swaths of core without hacking core. Score!

Some of those feature requests have been open in some form or another for years. (consider rendering into multiple formats, which was first opened in 2007.) The impact of these improvements is huge, and far reaching. Drupal 8 will be a viable Hypermedia server, something it never has been before. Combined with all of the other work that's gone into Drupal 8, and Drupal's long history as a CMS, we are looking at a platform that can take the web by storm.

Wait, who did that?

A lot of people have been involved in WSCCI to date. I couldn't list out everyone's name, because that would take a long time, but I especially want to highlight a couple of people:

And of course the ever-amazing Kat Bailey, who had the courage to dive into the depths of Drupal's bootstrap pipeline from which few return to tell the tale.

I also want to give a shout-out to Acquia, who has provided funding for both Klaus and Lin to work on the serialization and REST modules for Drupal 8 for the past several months. Without that, we would likely have ended up with a powerful new routing system and nothing to actually use it for, you know, web services. There's still work to do there, however, so if you would find it easier to help make Drupal 8 the best REST server on the market by sponsoring a developer than cracking open an IDE yourself, Klaus and Lin are still looking for additional funding to keep them on the job.

Awesome! So we're done, right? Right?

Eh, not hardly. Moving Drupal to a new foundation isn't a one-year job. We've built a new basement, poured the concrete, added plumbing, built the house framework, and installed windows. That's huge, but we still need to put down the new floors, plaster the new interior walls, paint, and buy furniture.

In short, it's time to port Drupal to Drupal 8.

The WSCCI team is still meeting weekly in IRC, every Tuesday at noon US Eastern Time in #Drupal-WSCCI. If you want to get involved, that's a great place to start. Our project manager, Ashleigh Thevenet, is posting regular updates on the WSCCI group with our current hit-list of high-priority items. A high-impact issue list is also available on the Core Initiatives page.

In short, you know what you need to do. :-)

In broad strokes, here's the major work that still needs to be done:

1. Finish separating Routes from Menus. We have an initial way forward that needs people to help with it, but will need to figure out how to go the rest of the way.
2. Fully implement HAL as our preferred format for Hypermedia resources.
3. Properly handle Entity References when importing an entity via REST, which would enable full round-tripping and therefore REST-based Deploy.module-like functionality. (All together now, "Oooo...")
4. Move our session handling system over to Symfony's, so we can eliminate tons of redundant code.
5. Finish the migration to a UrlGenerator, which unifies all of our outgoing-path logic into a single flexible, cachable, pluggable pipeline.
6. Finish porting our outgoing requests from drupal_http_request() to Guzzle.
7. The big one: Merge page caching and block caching to use HTTP caching and ESI, based on the new Symfony fragment pipeline that was developed in a large part in response to Drupal's needs.

And finally... move all of our page callbacks and routes over to the new routing system as controller classes. This is the most easily crowdsourced part, but we need to complete the first point above before we can really pounce on it.

So no, we're not done. Not by a long shot.

OK, so what's this got to do with me?

Everything! Drupal 8 is poised to kick ass. But it's not going to kick ass unless we close the deal. We need your help to help close the deal.

Come by the WSCCI channel any time to get up to speed. We'll also be working on documentation soon for converting page callbacks to controllers (stay tuned). And don't forget that there's a massive world-wide sprint coming up next weekend! Is there one in your area? Are you going to it? You should be. This is a great opportunity to get involved in core, and in WSCCI specifically. Several WSCCI regulars will be available from their home cities if you need help.

If you're coming to DrupalCon Portland (you are, aren't you?), there are also going to be pre- and post-conference sprints in which WSCCI is participating. Want to get your Drupal on for even longer? Sign up for Portland sprinting and let's crank out as many patches as we can!

Drupal 8 ain't finished until it's finished. Let's get stuff done.