Jan 14 2013
Jan 14

For Acquia, 2012 was a great year. In many ways, it's been our best year.

Last year, we saw more evidence of Drupal continuing to become a growing part of the mainstream. While this trend has been apparent for some time, in 2012 we were being adopted at a faster rate by more and more enterprise businesses and government agencies. Acquia, in many ways, has risen on the tide of this acceptance. Maybe we helped build this momentum. And along the way, as we've grown, we have worked to keep the philosophy of open source as the guiding philosophy of Acquia.

The Open Source Way

The concept of being guided by the philosophy of open source, which I call the Open Source Way, is reflected in Acquia's approach to our products and services. For example, we believe it is important to provide the capability to easily transfer data from one platform or solution to another, and not be shackled to proprietary vendors' platforms. The solutions we offer, whether PaaS or SaaS, allow innovation and agility by following the open source way, eliminating lock-in. We've coined the terms OpenSaaS and OpenPaas to refer to this.

This approach has resonated with enterprise business. This is reflected in our growth metrics for 2012. Our growth was reflected in our sales bookings, which grew at a record rate. We finished the year with 15 consecutive quarters of revenue growth, surpassing even our own aggressive goals.

Acquia grew by more than 160 employees last year, and now totals about 280 staff. In addition to Acquia's base in Burlington (Boston, MA), we have 28 employees in the UK office, 14 in our new Portland office, and 82 working remotely. Success poses many challenges. Hiring so many people is difficult. On one recent Monday, we have about 20 new staff undergoing orientation in our Burlington office. We've met the challenge of hiring, though, and we've assembled a staff of talented, passionate people. They are the reason for Acquia's success.

Our core strength is our ability to accomplish the aggressive goals we set for ourselves. This ability is the result of both the collaboration and the passion the Acquia staff brings to everything we do. Acquia's culture, in which collaboration and passion are key, also reflect the Open Source Way. We bring this passion and collaboration to our customers as well, and we work hard to ensure every customer's success. In 2012, the number of customers renewing with us was up, returning that commitment and loyalty.

Landmarks and trends

As we moved through 2012, we saw the growing acceptance of cloud computing. No longer was it "should we be on the cloud", but businesses asked "how best to move to the cloud". More often, the open, elastic cloud computing offered by Acquia was the answer. Platform as a Service (PaaS) and Software as a Service (SaaS) both continue to gain further acceptance and grow, again providing that ability to react to business needs rapidly, putting a larger portion of resources into building exactly what is needed when it is needed, rather than investing in expensive infrastructure and maintenance. The success of our cloud products means that Acquia will continue to invest and expand in this area in 2013, especially as we saw the trend last year that having many microsites, often one for each product or service, is quickly becoming the rule rather than the exception.

Other landmarks in 2012 were the growing number of health/pharma businesses moving to Drupal and the cloud, joining financial services companies and government agencies also making the move. Until recently, these industries were wary of open source and cloud-based services, fearing that these solutions weren't secure or reliable enough. The reality that the cloud can also be fault-tolerant and highly available, and that security and government compliance requirements can be met with confidence, opened up the cloud to more and more enterprise businesses in 2012. Their move to the cloud in 2012 reinforced the fact that freedom of innovation and agility of open solutions are driving factors for large-scale business as well as smaller organizations.

As the public moves rapidly to mobile platforms of all kinds, including smart phones and tablets, the need to provide a great user experience on these platforms is becoming increasingly important. UX also became important in 2012 as marketing rather than IT became the driving force behind more and more websites. Acquia responded with the creation of our Spark team, which took shape as a five-person team made up of some of the world's best Drupal experts.

Also in 2012, Acquia acquired Mollom, a company I created to address the challenge of managing social spam on websites. With the tremendous growth of user-generated content as part of the social media explosion, unwanted content has become a more important issue to take on. As a SaaS tool, Mollom fits in with Acquia's existing services.

Drupal community

In 2012, Acquia continued to invest in the worldwide Drupal community in a number of important ways. First, we sponsored over 82 Drupal events around the world in 2012. These events brought new people into Drupal and helped existing Drupal users learn new techniques. We employ more than 110 Drupal specialists, most of whom are significant contributors to the larger community. We've sent our Drupalists to more than 30 of these events (as well as hosted sprints ourselves at Acquia) to collaborate with others in the community on important problems for Drupal.

We also grew Acquia's Office of the Chief Technical Officer, or OCTO, in 2012. OCTO includes a dedicated team who work on Drupal full-time, on projects that include:

  • Drupal core architecture issues.
  • Authoring experience improvements via Spark.
  • Spearheading process changes that help the community work better together.

And finally, Acquia has sponsored other key contributors in the community to take on critical work, including the configuration management initiative, web services, and "Views in Core".

Looking forward

This year, like 2012, will be a key year for Acquia as we continue to develop products and services built on the open source philosophy.

Life-cyle management applications will be an increasing focus for Acquia in 2013. These applications will help craft great digital experiences by providing the tools to monitor and optimize digital content.

Of course, we'll continue to nurture and expand our vision of OpenSaaS and OpenPaaS. We'll continue to make the move to PaaS even easier, providing solutions that offer all of the functionality needed, but in a simplified package. We'll accomplish this by combining PaaS, Drupal services and Application Performance Management to produce comprehensive solutions that continue to make Acquia a no brainer when it comes to choosing a PaaS provider. PaaS platforms that embrace an open ecosystem provide faster business value, as many of our customers have discovered. We are working with our growing number of partners to help them build customer solutions on our open cloud platform.

As we start down the road of 2013, we enter the year just having raised $30 million in Series E financing, the single largest financing we have done to date. As we have grown and matured during 2012, these funds will assure sustained growth and success in 2013. No matter how rapidly we grow, or how large the Drupal community becomes, Acquia will put its open source philosophy at the core of all the work it does. In the end, the people of Acquia and the Drupal community, following this philosophy, are building the future of the digital experience. The Open Source way.

Aug 01 2012
Aug 01

As mentioned last month, on July 16 - 17, 2012, several community leaders in the Drupal project sat down with several community leaders from other open source projects and tried to hash out a governance structure to support the Drupal community's continued growth. "Governance" in this case, encompassing all the things we do to organize ourselves, make plans and decisions together, get things done, and resolve conflicts.

Here are the proposals we came up with, and we are actively seeking community feedback on the ideas within.

What problems are we trying to solve?

We began the sprint by brainstorming a list of problems we're hitting, given the scale of our community. This list included items such as:

  • No obvious answer to the question, "Who can make a decision here?" and as a result, important decisions can get stale-mated, or in the worst case leave smouldering craters where a healthy, functioning community used to be.
  • Because the Code of Conduct punts on the topic of conflict resolution, a handful of already swamped individuals get tapped on an ad-hoc basis to intervene in conflict situations. This both burns out those individuals, and also leaves the vast majority of the community who don't know who those individuals are with no conflict resolution path apart from "repeat what I already said, but with more volume".
  • While our "do-ocracy" model generally works well for our community, it can also lead to situations like "Tyranny of the person with the most time on their hands." We need ways for smart people who can't be on IRC 18 hours a day or read 300+ reply issues to participate and be respected as equals.

Over the course of two days, Drupal community members Dries Buytaert, Angela Byron, Randy Fay, Greg Dunlap, and David Strauss met and discussed a variety of these and other governance topics, and we also received input from Jono Bacon, community manager for the Ubuntu project, Jared Smith, former project lead of the Fedora project, and David Eaves.

Steve Edwards also recorded a podcast about the Governance Sprint; it provides more background on what problems we are trying to solve.

Overview of proposed changes

We propose the creation of a number of "working groups" that essentially make more explicit community structures that already exist. Each working group would consist of ~5 people, appointed by Dries, in charge of collaborating with the community in order to establish effective policy. Each working group will have one "lead" member (chair) who communicates major items and works with Dries. Some working groups will have a set duration (e.g. life cycle of a Drupal core release), others may have terms. Dries, as project lead, also reserves the ability to terminate a group at any time if it feels like they are overstepping their scope (charter).

The summary, in essence:

  • Establish clearly chartered working groups where currently loosely defined individuals are taking on things that are community-shaping in their scope.
  • Empower these working groups to make decisions, so important community governance decisions do not get stalemated. Keep the groups small enough that decision-making can take place efficiently, but large enough that a diversity of opinions are represented.
  • Make it more transparent and obvious, to newcomers and community insiders alike, where "the buck stops" with decision-making in our community in various areas, what the structure of the community is, what's expected of them, and who to turn to for help.

"Drupal" groups

The "Drupal" groups encompass areas that touch Drupal core or contrib, or the Drupal community itself. The ultimate "buck stops here" with these groups is with Dries Buytaert, the Drupal project lead.

Governance sprint community

Inspired by the Fedora Community Working Group, this group would be responsible for maintaining a friendly and welcoming community, and their charter will likely consist of items such as:

  • Maintaining and updating community-governing policies like the Drupal Code of Conduct.
  • Helping with mentorship and on-boarding of new community members.
  • Ensuring existing community members have their needs met.
  • Intervening as mediators in cases of conflict resolution (where the conflict cannot be worked out among individuals first) or burnout.
  • Issuing sanctions in cases of extreme policy violations.

In other words, this working group tries to make sure the "people" side of our community is functioning well. It doesn't set technical policy or intervene in any code-related matters; this is the role of the Technical Working Group. The ideal make-up of this group would be community-minded people with extreme amounts of patience, empathy, and diplomacy skills.

A corollary to the Community Working Group, this group would set and maintain policies around the technical aspects of our community, including:

  • Develop and maintain policies around things such as:
  • Establishing best practices and recommendations around bug/issue workflows; for example, strongly encouraging a workflow of idea -> architecture review -> implementation -> code style/clean-up.
  • Recommending to the Drupal Association tool changes that will help accelerate contribution.
  • Intervening as mediators in cases of technical conflict (where the conflict cannot be worked out among individuals first).

In other words, this working group tries to make sure the "technical" side of our community is working well. "People" problems would be escalated to the Community Working Group. Nevertheless, the ideal make-up of this group would be community-minded people who are also technical, known to be fair, and adverse to making new rules.

Drupal Core

A lot of time was devoted at the sprint to discussing Drupal Core, and how to address some of the challenges surrounding its development. For example, there is currently a lot of tapping of internal networks to move things along in core, and those without access to those networks can feel blocked out. It's also very difficult to get an answer as to whether or not something is "core-worthy" until far too late in its development process, making major feature development a risky affair.

The recommendation from the Governance Sprint is something like the following, which would not take effect until the Drupal 9 development cycle.

Drupal Core Initiative Working Group

This group works with Dries Buytaert, the Drupal project lead, in order to tackle strategically vital initiatives within Drupal core. Membership includes the initiative leads. This would entail a bit more formalized structure, including milestones and progress tracking, bi-weekly meetings among the various initiatives, and so on.

This would be essentially formalizing what already exists today with the Drupal 8 initiatives and initiative leads.

Contributions repository

At the Governance Sprint, we agreed to continue not to impose any additional governance structure on contrib, by design. This allows contrib to be an incubator not only for technical solutions, but also for governance itself.

The exception would be conflicts between maintainers or maintainers and their users which are not able to be resolved among the individuals. These would then go to either the Community Working Group or Technical Working Group, as appropriate.

Security and documentation teams

We have a few overall "Teams" that touch elements of the product, including the Documentation Team and Security Team (we also discussed the establishment of a Support Team). As part of the new governance model, we recommend creating charters for these teams that make it explicit to others what their roles and responsibilities are, how to join, and what is expected of them. It's likely these charters will be modeled after something like the Documentation Team and Leader Responsibilities page.

"Operations / Administration" groups

These groups act in support of the Drupal project and its community. The ultimate "buck stops here" with these groups is with the Drupal Association board instead of Dries. Many of these have a financial impact on the Drupal Association and greatly affect its ability to get things done in support of its mission.

Governance sprint operations

And what about Drupal.org?

Governance sprint drupal org

Next to Drupal core, this is probably where we spent the most time discussing. Drupal.org is special, in that it straddles both the community side of things, as well as the operations / support side of things. It functions through a combination of numerous volunteers as well as funding via the Drupal Association for support staff and development on major initiatives.

At the moment, the best place to put Drupal.org seems like it's at a halfway point between the "Drupal" and "Operations" sides of things, and for the charter of this working group to include the necessity to work with the Drupal Association and community members alike. Though eventually, for both legal and simplicity reasons, it would be better for this to be located under the purview of the Drupal Association board.

A few areas there was broad agreement on, however, were the following:

  • Split off a separate "Drupal.org content" working group from the "Drupal.org webmasters" working group; different skills/levels of trust are needed for managing the content on Drupal.org versus managing access and performing moderation of abuse.
  • Identify a much smaller subset of the Drupal.org webmasters group to form policy for this team. Currently, there are nearly 150 members with "site maintainer" privileges, and they often make and enforce policy on an ad-hoc, individual basis. Community members currently encounter very inconsistent experiences in the queue.
  • While the Drupal Association doesn't manage these groups, it's generally expected that the charters of these groups will include directives to collaborate with the DA in their policy-making decisions in order to ensure the financial sustainability of Drupal.org.

Next steps

We're very interested in community feedback on this direction, either in comments below or privately. We'll provide an update on the progress at DrupalCon Munich.

We encourage everyone to come and get involved in this discussion. As our community grows, it is essential that we come up with a governance structure that matches our core values and allows us our community to be more sustainable for the long haul.

Feb 16 2012
Feb 16

Last weekend, we held a sprint at the Acquia offices for the Web Services and Context Core (WSCCI) Initiative for Drupal 8. This was an important sprint for the future of Drupal. This blog post provides a high-level overview of what was discussed and agreed upon; the different sprint participants will be laying out more technical details in follow-up blog posts.

Overall, a wide range of experience levels, skill sets, and perspectives were brought to the table, with the goal of the sprint being to clearly define the initiative’s scope, get agreement on what we wanted to accomplish and why, and lay out a clear plan for how to accomplish this.

WSCCI sprint group photo

In attendance were:

Scope

The WSCCI initiative, as envisioned by Larry Garfield, was originally set to address Drupal's web services and flexible page layout capabilities. We discovered that both would require significant changes to Drupal core, and it was difficult to build consensus online, so we decided to get together for 3 days and to flesh out what we actually wanted to accomplish, and how.

At the sprint, we first attempted to articulate all of the problems that WSCCI was trying to solve, which included: multiple page layouts, better UI/UX to manage blocks, partial page rendering (ESI, AHAH), contextual blocks, different response types per delivery callback/URL, plugin system / swappable subsystems, lazy loading bootstrap (convert subsystems to PSR-0 classes), infrastructure for building web services, dependency injection, and so on.

We then did a round of voting where we could each choose 3 of those things in order to try to determine which of those were the most important.

WSCCI sprint Post-it notes

Two things became instantly clear during this exercise:

  1. The items encompassed under WSCCI really spanned at least 3 separate major areas: Web Services, more robust ESI-based layouts (think Panels only more powerful), and cleaning up our underlying toolset to be a more loosely-coupled framework.
  2. The underlying architecture to support RESTful calls to Drupal that makes all of the other things possible was deemed the most important thing to focus on.

Scope resolution

After a good chunk of discussions, all were in agreement to scale back the scope of the initiative to just the "Web Services" piece, and spin off the Layout/blocks/related-UI parts to a separate effort.

Furthermore, some efforts, such as PSR-0 and Unified Plugin system, were only semi-related to the WSCCI initiative in the first place, and just happened to become relevant for it. Work on those efforts will continue as part of the general Framework community efforts.

Architecture

Fabien was able to offer a tremendous number of insights as to how various Symfony2 components could help provide underlying structure for Drupal core to support Web Services out of the box. Essentially, most of what the WSCCI team had been trying to work toward, in the abstract, was already implemented within Symfony2. While some implementation details were different than what we had in mind, the end result is almost exactly what we have been trying to accomplish. We therefore agreed that the best way forward was to leverage several Symfony2 components within Drupal as a way to speed progress toward that end.

Benefits

Some of the concrete benefits that would come out of these changes are

  • An underlying framework that is a first-class REST citizen. That means developing web services becomes easier and more efficient, because the plumbing is already in place. An HTML page is a particular case of a REST service.
  • Because the core system will be fully REST-ified, we'll be able to improve existing APIs and expose Drupal content in a natively RESTful way. For example, our current AJAX system doesn't comply with REST standards, but with these changes, can be cleaned up to do so.
  • That in turn makes Drupal-to-Drupal communication, content staging, content sharing, and a host of other related tasks easier.
  • The use of widely used libraries and techniques makes Drupal more approachable to new developers.

Why does this matter?

As it has evolved into an increasingly powerful system, Drupal has gotten increasingly complex and is not as easy to start developing with as it once was. Many developers are nervous about continuing that trend. Managing complexity is a challenge faced by many software projects, and one approach is to use standardized patterns and components.

Due to its long support for PHP 4, as well as some unique needs, Drupal does not take full advantage of standardized patterns and components. The complexity of the custom code that’s used and the non-standard architecture combines to create a barrier to entry for developers new to Drupal (both experienced and novice developers alike).

Meanwhile, the web is constantly changing around us. Web services and mobile are more important than ever, and with that comes the need to have more flexible page and layout capabilities. Now feels like the right time to modernize Drupal’s capabilities to bring it to the forefront of modern PHP systems and web systems in general.

While changing Drupal's core plumbing to address these needs, it's also a good opportunity to do so using more widely understood and modern techniques and libraries. Leveraging the work of a fellow open source community lets us get a jump on these changes to build a more powerful, more flexible, and more easily learnable system in less time than it would take to go it our own.

While these changes may seem large, and some of them are, we believe that they will achieve the right balance between empowering Drupal's design and architecture, and moving toward more modern, standard, well-tested code and techniques to empower a new generation of developers. I hope you are as excited as we are!

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web