Feb 09 2017
tvn
Feb 09

I am happy and sad to share the news today that I am leaving the Drupal Association for an exciting new adventure.

I've been volunteering for Drupal.org for a while before I joined the Association as a staff member in April 2012. Before that I never actually thought you can be paid to do something you volunteer your free time for because you enjoy doing it so much.

It was also my first remote job. Back then it was not as common. I clearly remember first talking to Angie, and my initial response was "You are crazy. I am literally on the other side of the world." Which she didn't find to be of any consequence.

It's been a wonderful (almost) 5 years. I've learned a lot, and grew. I've traveled a lot and met lots of fantastic people. I watched the Association grow. Our engineering team grew as well from 2 to 11, which was exciting but also challenging to be a part of. We've done some great things on Drupal.org. Of course, there are so many many more I wish we'd done. :)

The time has come however to move on to the new challenges.

Thanks to all the volunteers who worked with me during all these years. You were so helpful and generous with your time.

Thanks to the staff members for being wonderful human beings, for your support, and laughs, for becoming a family.

I am incredibly sad to leave our staff, and especially the Engineering team. They are a smaller team now, and I know just how hard it will be to maneuver all the requests coming from all the different parts of the community with the limited resources they now have. So please be nice to them.

As for the next steps, I am looking forward to joining the DA alumni club, which includes some of my closest friends. And I will be around. You will probably see me at the next DrupalCon. Come say hi. And no, I will not fix your webmasters' queue issue. 
 

---
Photo from Flickr.

Aug 30 2016
tvn
Aug 30

One of the biggest content areas on Drupal.org—and one of the most important assets of any open source project—is documentation. Community-written Drupal documentation consists of about 10,000 pages. Preparations for the complete overhaul of the documentation tools were in the works for quite some time, and in the recent weeks we finally started to roll out the changes on the live site.

Background

Improving documentation on Drupal.org has been a part of a larger effort to restructure content on the site based on content strategy we developed.

The new section comes after a few we launched earlier in the year. It also uses our new visual system, which will slowly expand into other areas.

Goals and process

The overall goal for the new Documentation section is to increase the quality of the community documentation.

On a more tactical level, we want to:

  • Introduce the concept of "maintainers" for distinct parts of documentation
  • Flatten deep documentation hierarchy
  • Split documentation per major Drupal version
  • Notify people about edits or new documentation
  • Make comments more useful

To achieve those goals, we went through the following process:

First, we wrote a bunch of user stories based on our user research and the story map exercise we went through with the Documentation Working Group members. Those stories cover all kinds of things different types of users do while using documentation tools.

We then wireframed our ideas for how the new documentation system should look and work. We ran a number of remote and in person usability testing sessions on those wireframes.

Our next step was to incorporate the feedback, update our wireframes, and create actual designs. And then we tested them again, in person, during DrupalCamp London.
Incorporated feedback again, and started building.

The new system

So, how does the new documentation system work exactly? It is based on two new content types:

  1. Documentation guide: a container content type. It will group documentation pages on a specific topic, and provide an ability to assign 'maintainers' for this group of pages (similar to maintainers for contributed projects). Additionally, users will be able to follow the guide and receive notifications about new pages added or existing pages edited.
  2. Documentation page: a content type for the actual documentation content. These live inside of documentation guides.

Documentation guide screenshot
Example of a new documentation guide

All of the documentation is split per major Drupal version, which means every documentation guide or page lives inside of one of a few top level 'buckets', e.g. Drupal 7 documentation, Drupal 8 documentation.
It is also possible to connect guides and pages to each other via a 'Related content' field, which should make it easier to discover relevant information. One of our next to-do’s is to provide an easy way to connect documentation guides to projects, enabling 'official' project documentation functionality.

More information on various design decisions we made for the new documentation system, and the reasons behind them, can be found in our DrupalCon New Orleans session (slides).

Current status

Right now, we have the new content types and related tools ready on Drupal.org.
We are currently migrating existing documentation (all 10,000 pages!) into the new system. The first step is generic documentation (e.g. 'Structure Guide'), with contributed projects documentation to follow later.

While working on the migration, we are recruiting maintainers for the new guides. If you are interested in helping out, sign up in the issue. Please only sign up if you actually have some time to work on documentation in the near future.

There is a lot of work to be done post-migration (both by guide maintainers and regular readers/editors). The content is being migrated as-is, and it needs to be adapted for the new system. This means almost every single page needs to be edited. New fields (such as Summary) filled out with meaningful text (to replace text automatically generated by the migration script). A lot of pages include information for both Drupal 7 and Drupal 8, but this content needs to be split, with Drupal 8 information moved to pages in the appropriate version of the guide. These are just some of the steps that need to happen once the documentation has been migrated into the new system.

Next steps

As staff, we have a few follow-up tasks for minor improvements to the content types and tools. However, the bulk of the work is editing and improving the actual documentation, as I described above. This is in your hands now. Not only do we not have enough staff members to edit every single documentation page in a reasonable amount of time, we are also not subject matter experts for many of the topics, and so can't provide meaningful edits. The tools are ready, now it is up to the community to pick them up and write great documentation.

Documentation page screenshot
Example of a documentation page

Thank you

Lastly we want to say thanks.

Thanks to all the community volunteers who wrote those 10,000 pages over the years. Thanks to the Documentation Working Group members for their expertise, insight, and patience.

And, of course, thanks to staff. Unfortunately due to recent changes for the Engineering team, this will be the last section we'll have resources to work on for a while. This was a fun and important project to work on, and we are glad that we got to finish it. It is a beautiful legacy of the work we did together with some of our former colleagues: DyanneNova, japerry, and joshuami. Thank you!

Apr 06 2016
tvn
Apr 06

In this post I'd like to talk about one of our major projects for 2016, which comes as a follow up to the content strategy project of 2015.

Content restructure

Last year we presented our findings from the content strategy developed by Drupal Association staff in collaboration with Forum One. This year we're focusing on bringing as many of those ideas to life as we can. We call this implementation phase 'content restructure'. We'll look at one area of the site at a time, audit its content, change the way it is created and stored (content type) if needed, redesign the way it looks and reorganize it into a more usable and findable structure, improving content quality and giving content creators better tools to maintain it along the way.

The backbone of the new content structure are 'sections' or top level groupings of content. We created infrastructure to make those possible and have already launched the first few.

Together with sections we've been rolling out blogs to improve how we communicate about specific topics on Drupal.org. Recently, I talked in more detail how blogs and sections fit into our overall plan of making it easier to communicate important announcements and news to the Drupal community.

Our current focus is Documentation area of the site. We're working on a complete revamp that will change the way documentation looks and works, and will change the way users can navigate and improve documentation. We're working closely with the Documentation Working Group and performing rounds of usability testing to ensure the changes we are working on will improve the user experience across the board. More details on this can be found in the issue queue.

A big part of the content restructure plan is a content audit and migration. This is especially true for documentation revamp, where we have thousands of pages to migrate into the new system. We'll be turning to the community to help us with this effort. Not only because that's too many pages for a small team like ours to migrate on our own, but also because we need subject matter experts to look at a lot of the pages and evaluate how accurate they are, whether they should be migrated or archived, and so on.

More than just content

Along with the content restructure project, we'll be doing important work that complements and supports it, though each component is not a discrete project on its own.

Developing visual design system

The current Drupal.org design is based on branding and design work done in 2008 by Mark Boulton Design and Leisa Reichelt. They did a great job, but even the most beautiful site will age. Since more than seven years have elapsed since the last redesign, it's time to update the site for a more contemporary look.

Our quest towards updating Drupal.org visually started in 2014 with the user research project, which brought us user personas. The next big step was the content strategy project, which laid the groundwork for the content restructure work discussed above.

Building on what we learned about our users, and how to structure our content to best serve their needs, this year we'll be introducing the new visual design system for the site. There will not be a single, comprehensive launch, where you wake up and suddenly Drupal.org looks completely different. We'll do it iteratively, in parallel with the content restructure, by redesigning the specific area of the site we are focusing on at a given time. This approach lets us introduce visual changes sooner, and iteratively improve and refine them as we go. In fact, you've already seen some of the elements of the new visual system appear with the Drupal 8 launch.

Later this week, our designer DyanneNova will share a bit more details about the work we've already done towards the new system and our next steps.

Updating content style guide

Along with restructuring content we also want to improve the quality of the existing content during migrations, as well as the quality of the content that will be created in the future. To this end, we'll be taking a look at the content style guide, and plan to refresh and update it. We also anticipate expanding the guide to add information about specific content types and communication channels.

Capturing user engagement and contribution

Another aspect of our content restructure work will touch on user engagement and contribution. As we go area through area of the site, redesigning it and improving its content, we'll be looking at what type of user engagement and contributions happen in that area. We'll be looking for opportunities to better capture them, and subsequently better recognize and display those contributions. For example, right now 'documentation edits' count on user profiles show the number of edits user has done to 'book page' content type items, which may or may not be documentation. We'll make that calculation more precise to display the actual documentation edits. We'll also be able to display specific parts of documentation a user maintains, similar to projects they maintain.

Increasing sustainability

An ongoing challenge at the Drupal Association is ensuring we have sustainable revenue to support our work for the community. As we do this work, we will be looking into improving existing revenue opportunities and introducing new ones to make Drupal.org more sustainable. We will also work closely with partners who may be willing to sponsor specific improvements to Drupal.org on behalf of the community.

Other initiatives

The content restructure is not the only project we are working on right now. Some of the other initiatives will be described in future posts. Check our Roadmap to see all the things in progress.

And if you happen to be at DrupalCon New Orleans this May, come to our session to get further updates on some of the topics discussed in this blog.

Mar 23 2016
tvn
Mar 23

Last year, Drupal Association staff—in collaboration with Forum One—developed the content strategy for Drupal.org. One of the problems that surfaced early-on during that project was the problem of communicating with the community. Rolling out the new Sections on Drupal.org is part of how we are starting to address that.

Sharing important communications with the community as a whole was, and still is, a challenge, mainly due to how hard it can be to find clear, “official” communication channels on specific topics. Because Drupal.org has grown organically, an important announcement could be posted to a number of different places: to forums, to the homepage, to a group on groups.drupal.org, to the issue queue, or to the Drupal Association blog on assoc.drupal.org.

A community member might know about the existence of some of the channels, but not the others. For them, it is not clear which channel(s) to watch to make sure they don't miss important things. And when you are the one sharing important information, it is really hard to know if your message actually reached the people who needed to see it.

Many hours, a bunch of post-its, and quite a few Google docs were spent figuring out a better structure. Here are the communication channels we plan to create and consolidate.

Communication channels

We grouped all the various ways to send a message into groups based on the purpose.

Primary communication channels

These channels are the main source of all the news and announcements. They will include original content written specifically for these channels. The content will be published frequently, following an editorial calendar schedule in each channel.

News

  • ‘Official’ source of news on: Drupal (the software), Drupal.org (the website) and the Drupal Association.
  • Located in a dedicated News & Events section.
  • Curated content - mainly press releases, official announcements.
  • Comments are not enabled on this content.

Blogs

  • Blogs on a limited number of topics - aligned with top level Sections of the site. E.g. About section includes Drupal blog, Drupal.org section includes 'Drupal.org blog', Community section - 'Community blog', etc.
  • More frequent and less official updates and announcements.
  • Discussions happen here via comments.
  • Managed by maintainers of respective Sections.

Secondary communication channels

These channels will be used to support communication happening via primary channels. They also will include original content, but communication might be less frequent, aimed to support only the most important announcements.

DrupalCon sessions

  • In-person communication with the community.
  • Opportunity to get direct feedback.
  • Opportunity to communicate via live presentation.
  • Video recordings available on events.drupal.org.

Webcasts

  • An opportunity to do an extended presentation on the topic with slides and questions from the audience.
  • Video recordings available on Drupal.org.

Tertiary Communication Channels

These channels will be used to retranslate messages from the primary and secondary channels to a larger audience or different audience segments. They will provide the ability for the community to follow and consume information from primary and secondary channels in different ways. No original content will be written for these channels.

Drupal.org homepage

  • Highlight important information for Newcomers, Learners and Skilled users.
  • News and Blog posts can be promoted.

Planet Drupal

  • Provides ability to consume information from primary channels via RSS.
  • All primary channels will have feeds added to Planet.
  • There is no need to promote something to the homepage to get it on Planet anymore.

Newsletters

  • Provides ability to consume information from primary channels via email.
  • These are various newsletters sent by the Drupal Association staff.

Social media

  • Provides ability to consume information from primary channels via preferred social media.

Direct communication channels

Mailing lists

In some cases for community members with specific roles or duties, we use mailing lists for private two-way communication. E.g. mailing lists for working groups, infrastructure team, webmasters and other users with advanced access on the site.

Which existing channels will be replaced?

This restructure of the communication channels means that some of the existing ones might be replaced by new and better alternatives, including:

  • Announcement-only groups (e.g. core, governance) will eventually be migrated into blogs for appropriate sections on Drupal.org
  • News and announcements forum will be eventually replaced by a combination of topical blogs
  • Assoc.drupal.org blog will eventually be replaced by a blog in Drupal Association section on Drupal.org
  • Groups.drupal.org homepage will eventually be replaced by the landing page for groups area on Drupal.org.

The First Steps

It will take some time to put all these channels, tools and processes in place. But the first steps have already been taken, and we're working to bring the idea to life.

Blogs

We've decided to tackle primary communication channels first. The first primary channel we are implementing is the blog. Most of the top-level sections will have a topical blog inside of them. Typically, those blogs will be curated and only section maintainers will be able to publish posts in them. The following blogs already exist:

Drupal.org blog

As with the Sections, we wanted to try the concept out on a less visited area. So we created the Drupal.org Blog first. It is a main source of announcements and updates our team will share from now on instead of the assoc.drupal.org blog or Drupal.org forums. In fact, this post you are reading now is in that blog already.

Drupal blog

The Drupal blog is a part of 'About' section. It is a main source of important news and announcements about Drupal project. The editorial calendar for the blog is managed by the Drupal Association. And posts are typically written by staff or members of the Drupal Core committers team.

Keep up-to-date with blogs

At the moment, to follow new posts you can subscribe to an RSS feed for each blog. The ability to subscribe to new posts via email, or have them show up in your tracker, will be added soon.

All of the posts from both Drupal.org and Drupal blogs appear on Planet Drupal automatically, and the most important ones will also be promoted to the homepage for additional visibility.

The posts are typically also announced on Twitter (@drupal_org and @drupal respectively). And you can follow comments on individual posts to take part in discussions.

We are excited to take this first step towards improving community communications on Drupal.org. Watch this space for more news and updates.

Feb 29 2016
tvn
Feb 29

Last year Drupal Association staff—in collaboration with Forum One—analyzed the state of content on Drupal.org. We developed a content strategy aimed at improving its quality and findability. Various recommendations were made for content structure, organization, and governance.

One of the main recommendations was to restructure content around areas of user activity instead of the content type used to create the content. On Drupal.org, content about a topic is often scattered, because some content types are only available in certain areas. But we’d rather have a single place for content on a particular topic, no matter which content types are used to create that content.

User research gave us a number of general areas of user activity or tasks, which we used as a base for top level content grouping. We’ve called those groupings “sections.” Each section is based around a particular set of common user tasks and has a clear purpose and audience. We started implementing these sections a few weeks ago.

Each of them will also have a slightly different governance structure. Some of the sections will have more editorial/curated content, while others will be open to edits by everyone. We wanted the flexibility of having different user roles with different permissions in different sections. We also wanted the flexibility of being able to display a single piece of content in multiple sections if needed, and perhaps even use a different theme per section.

To meet those requirements we decided to use Organic Groups. The work on getting all the modules ready on Drupal.org began in August last year. After a few rounds of performance testing and configuration review, Organic Groups and a few accompanying modules were in place to enable us to work on sections. The first couple of them were launched simultaneously with the Drupal 8 release.

Drupal.org section

At first, we wanted to test out our ideas and assumptions on a less visible area of the site with lower traffic. So, the first section we created was about Drupal.org itself. It consists of various information about the website, aimed at those who follow or take part in Drupal.org development. Its content is mostly produced by our internal team.

About section

The first highly visible section we tackled was About. It is a source of general information about Drupal and promotional materials. The content is curated and aimed primarily at evaluators, and the Newcomer and Learner personas.

To create the section, we audited all the content in the old “About Drupal” area (which was using the old book page content type), rewrote most of it, and re-created it using the new content types. While the initial round of work on the section is complete, there are a few more things we want to do, so expect additions to the section throughout the year.

Because of the curated nature of the content, this section has tight edit permissions, and is managed by the Drupal Association staff. Feedback is always welcome, however, so if you do notice a problem please use the Content issue queue to report it.

A big part of the About section is talking about the features of the Drupal software. And specifically with the Drupal 8 launch, we wanted to do it well, which brings us to...

Drupal 8 section

It is the landing page for Drupal 8 release, and the main source of high level information about Drupal 8 and its features. And it is a section too, created using Organic Groups and located inside of the About section.

This one was created from scratch. All the content was written specifically for it, by the Drupal Association's communications team with a lot of help, review, and feedback from Core committers team.

For this section, we went one step further. Not only does it have unique content, it was also designed to look completely different from the rest of Drupal.org. To make it happen, we created a separate theme, based on Omega, and used og_theme module to make it possible to use the theme on only one particular section of the site. This worked really well.

Again, this section has curated content and edit permissions are locked down. If you do find a problem, please report via the Content issue queue.

What's next?

These new sections don’t only introduce a new governance model and navigation patterns. They also introduce a new way we create dynamic content. I will talk more about this, as well as the sections we are working on right now, in following posts.

Feb 12 2016
tvn
Feb 12

Drupal.org, the home of the Drupal community, has grown organically for many years. At some point it grew so large that a clear decision making structure became a necessity. The Drupal Association staff was not in the place to provide it at that time: our entire technology team for Drupal.org, including all its sub-sites and services, consisted of only two people, myself and Neil Drumm—so we turned to community for help.

In the summer of 2013, the three Drupal.org Working Groups were announced. Governance committees, consisting of community members and staff, created to act as a collective 'product owner' for the website. In the following two and a half years, with their guidance and feedback, we implemented many new features, performed user research, developed content strategy, and drastically improved the infrastructure behind Drupal.org.

At the same time the Drupal Association staff kept growing. We hired our first full-time infrastructure staff member, brought in the CTO and customer service coordinator a few months later, then a developer and two more infrastructure team members. And finally, we hired a project manager, a web designer, and one more Drupal developer. Our communications team grew, too: over the last two years, the Drupal Association brought in a content strategist and a dedicated writer. Overall, our capacity increased and a lot of gaps in skills and experience were filled.

Having skilled staff working full-time on Drupal.org, we were finally able to provide product direction, set a roadmap, and execute on it. We adopted Scrum as our project management methodology, with a new sprint starting every two weeks. This encourages iteration and pivoting based on the situation, instead of working against a 'set in stone' year long plan. As our staffing situation changed, we started to realize that the valuable time of dedicated community volunteers can be spent more efficiently than making them sit in countless planning and update meetings with staff.

At the end of last year, the Drupal Association Board, with the input of several Working Group members, made a decision that it is time for staff to work on Drupal.org improvements directly with the community. This means that the Drupal.org Working Groups will transition into an advisory group, with former Drupal.org Working Groups members available as advisors to provide feedback and input on specific initiatives the team is working on, relevant to their own skills and expertise.

The only requirement the Board and Drupal.org Working Groups themselves put out before the transition could happen is this: they asked that the Association staff create a clear process for community members to be able to suggest items on the Drupal.org roadmap, and provide a path for those community members to volunteer to help with implementation. With the input from the Working Groups and the Board, we created such a process. It was
launched last week.

As we reach the end of an era, I'd like to personally thank each member who served at various times on Drupal.org Working Groups over the past three years. Your time, skills, and experience you shared with us has been invaluable.

Gerhard Killesreiter / killes
Narayan Newton / nnewton
Melissa Anderson / eliza411
Angela Byron / webchick
Kim Pepper / kim.pepper
George DeMet / gdemet
Jeff Eaton / eaton
Roy Scholten / yoroy
David Hernandez / davidhernandez
Cathy Theys / YesCT

Thank you! It's been a pleasure to share all those moments, conversations, ideas, debates, and workshops.

While the role of these wonderful people is shifting to a less formal advisory one, we will still be calling on their expertise and help as we continue our work on making Drupal.org a better place.

--
Image by Roy Scholten.

Dec 03 2015
tvn
Dec 03
Posted by tvn on December 3, 2015 at 3:51pm

As our regular quarterly review and update of the Drupal.org Terms of Service and Privacy Policy, we made the following changes to the Privacy policy today:

1. Under 'Sub-sites' we added a paragraph to clarify which data is being sent to updates.drupal.org by Drupal sites.

Updates.drupal.org
Your Drupal site may send anonymous usage stats including your websites's ip address and information about your currently installed modules and their versions to updates.drupal.org when checking for available updates. Statistics may be aggregated so that the updates system could identify changes in the use of modules over the lifetime of each anonymized site. Those statistics do not contain personally identifiable information and are used for providing usage data on Drupal.org project pages, as well as anonymized reports shared via blog posts and other channels.

2. Under 'Service providers and partners' we added a paragraph about Distil, as it is now a part of our spam protection system.

We use Distil to analyse browser activity during registration to prevent malicious users from registering accounts. You can view Distil’s privacy policy for information on how they manage request data. Distil's profiling is only enabled on registration form (at register.drupal.org) and no user data is provided to Distil through other areas of Drupal.org.

3. Under 'Your Choices About Use and Disclosure of Your Information' we clarified that users can opt out of Google Analytics tracking by using browser addon.

You can opt out from Google Analytics tracking via your browser privacy settings or by using browser addon.

Jul 20 2015
tvn
Jul 20

Look for links to our Strategic Roadmap highlighting how this work falls into our priorities set by the Drupal Association Board and Drupal.org Working Groups.

Drupal.org

Iterative Changes to the Front Page of Drupal.org

The home page of Drupal.org has been changing in several small but important ways. The main focus of our design work in June was to provide new community metrics to replace the less meaningful and somewhat misleading metrics that were removed in the previous home page update.

We also updated the text of the Try Drupal button on Drupal.org, to better clarify the purpose of the feature. Try Drupal allows potential users to evaluate Drupal by using a highly polished demo hosted by our Supporting Partners. This gives Drupal newcomers and learners the chance to see examples of Drupal configured at its best, to encourage evaluators to choose Drupal for their needs. The program supports a core part of the mission of the Drupal Association: helping to promote Drupal and grow Drupal adoption.

Improving Drupal.org Performance with Advanced Aggregator

The Association has also been working to improve the performance of Drupal.org using Advanced CSS/JS aggregation, with the help of mikeytown2.

These configuration changes have been made carefully to ensure they don’t degrade the user experience for any user of the site - and are continuing into July.

The Plan Category for Drupal.org Issues

Another small deployment made in June was the addition of a Plan category to the Drupal.org issue queues.

The Plan category codifies the informal [meta] issues into a category selectable within the Issues UI.

This only scratches the surface of a long-buried issue in the Drupal.org issue queues - a lack of project management and prioritization tools. The larger Content Strategy work that the Drupal Association is beginning to implement will help to address this need further with a new Initiative content type to provide better hierarchy and prioritization tools.

Recent Issue Credits (3 months) now appear on individual and organization profiles.

Expanding upon the work the Association staff has done to create a system for credit and attributions in the Issue queues, the Association has begun displaying information about issue credits on user’s Drupal.org profiles.

Whenever a project maintainer has credited the user in an issue when marking the issue closed - the project will appear on the profile, along with a link to a list of the credited issues.

Additional improvements are planned for the crediting UI to allow credit attribution to users who did not comment directly on the issue. The Association will also begin to backfill historical credit data.

Organizations benefit from this change as well. When a maintainer closes an issue and assigns credit, if any of the users being credited have attributed the work to an organization - that issue credit will be displayed on the organization page. This change rewards those organizations that give their employees time to give back to Drupal.

In May, the long-running Content Strategy work culminated the presentation of the Drupal Association’s proposal for a new content model on Drupal.org. In June, after monitoring feedback from the Drupal community and the Working Groups following DrupalCon Los Angeles - we transitioned  from planning the new Information Architecture to planning the implementation details to make the new content model a reality on Drupal.org.

Implementation of the new content model and governance plan is going to involve quite a few changes to the modules on Drupal.org, so we want to approach the implementation iteratively and carefully.

Our plan developed in June calls for us to create the new ‘Why Drupal’ section of the site first. In June, we prototyped an implementation of this first section using Organic Groups and Panelizer and prepared a plan for performance profiling.

The Association Team is excited to implement our vision for new issue workspaces on Drupal.org - including a new spin on the implementation of pull requests.

Work on the Issue Workspaces is slated to begin once major work on DrupalCI is complete, and we are able to retire the PIFT/PIFR testing system without regressions.

However, we were able to remove a blocker to this work by updating our servers for our Git architecture (more below in sustaining support and maintenance.)

Community Initiatives

DrupalCI

DrupalCI was a major focus of the Drupal Association staff in June. In June the integration between DrupalCI and Drupal.org was enabled for the first time. DrupalCI is now running in parallel with the PIFT/PIFR testbots to provide us a reference frame to prove out the implementation.

A remaining MVP hit list was codified in June - representing the few remaining issues needed to meet the guidelines set out by the Drupal core developers. (Spoilers: Most of these issues are resolved at the time of this posting in July!).

Going into July the focus will be on providing testing for Contrib through DrupalCI, and then ensuring that there are no regressions in functionality or test result detail as compared to PIFT/PIFR so that the old test bots can be retired.

Once that is achieved, the Assocation’s work on DrupalCI will scale back to maintaining the system’s stability- and more development focus will be provided to our next initiatives.

Localize.Drupal.org

At the end of June we initiated a final round of community testing for the port of localize.drupal.org from Drupal 6 to Drupal 7.

That testing period has ended as of the first week of June, and we are now working together with the community to resolve the issues uncovered by this final testing before deployment.

Revenue-related projects (funding our work)

DrupalCons

Registration for DrupalCon Barcelona opened in June, with some small refinements to the registration process from lessons learned in DrupalCon Los Angeles.

Events.Drupal.org did receive one new front-facing feature, an opportunity in the registration process for ticket purchasers to purchase or renew their Individual or Organization memberships with the Drupal Association.

The Drupal Association is also working very closely with the DrupalCon Asia designer in preparation for the full site launch in the coming months.

Better Cart Management on Jobs.drupal.org

On Drupal Jobs we deployed a small update to the checkout workflow to make cart management easier - addressing the top support request that we receive from our users.

Future development for Drupal Jobs continues to be limited to the most high-impact bugs or features identified in the support requests we receive for job postings.

Signature Supporting Partners Page Launched

June also saw the launch of the Signature Supporting Partner program. This required a small update to our supporting partners page to support the new partner category.

Sustaining Support and Maintenance

Final work preparing for deployment of our new git servers was completed in June - but for scheduling reasons the maintenance window for replacing our existing Git infrastructure was scheduled to take place on 7/9/2015. (This deployment was successful!)

Our Fastly CDN deployment for updates traffic (updates.drupal.org) was also successful. Updates now use dynamic purging to reduce the number of requests served by our origin servers and decreases the latency between packaging a release and serving the update data from a number of minutes to a few seconds.

As part of the updates deployment with Fastly, we now have a *.drupal.org wildcard TLS/SSL certificate for https://updates.drupal.org and https://ftp.drupal.org. This enables HTTPS support on all of Drupal.org and its’ sub-sites for the first time.

---
As always, we’d like to say thanks to all volunteers who are working with us and to the Drupal Association Supporters, who made it possible for us to work on these projects.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra.

Jun 05 2015
tvn
Jun 05

Look for links to our Strategic Roadmap highlighting how this work falls into our priorities set by the Drupal Association Board and Drupal.org Working Groups.

Explicit Attribution Option for ‘I am A Volunteer’

As a part of our effort to recognize individual contributions to the Drupal ecosystem we’ve slightly adjusted the options available to a user when making an attribution in the issue queues. Instead of simply assuming that a comment made without an attribution to an organization or customer is done by a volunteer - we now allow volunteers to explicitly mark their work as such. Requiring a positive affirmation of the volunteer attribution should improve the accuracy of the data we are gathering about the Drupal ecosystem.

This now means a user can make issue comment attributions in the following ways:

  1. Without attribution
  2. As a volunteer
  3. On behalf of an organization and/or customer
  4. Both as a volunteer and on behalf of an organization and/or customer.

We are seeing a rate of around 30% of issue comments attributed to an organization, customer or as volunteer work. We hope to see that rate increase steadily.

To date, there have also been over 7,000 issue credits that have been awarded to over 2,300 users and 175 organizations. We are looking forward to displaying these credits on user and organization profiles in the month of June and beginning to find new ways to reward our top contributors.

Our collaboration with Forum One on developing content strategy for Drupal.org finished a few weeks ago. While recommendations were published in the issue queues earlier, we decided to use DrupalCon Los Angeles as an opportunity to present the work done and future plans in more detail, and get direct feedback from community members. Check out session slides or video if you want to know more on proposed changes to Drupal.org IA and content strategy.

Right now we are working on a few preparations steps before we can start implementing the changes. The first one of those steps would be a card sort exercise to validate our proposed IA and navigation with Drupal.org users. More blog posts and issues will follow as we move further.

The Drupal Association has been preparing a plan for a new issue workflow on Drupal.org - with some very exciting improvements planned to create a workflow that is both familiar to other repository hosts and yet unique to the needs of the Drupal community.

Perhaps the greatest value of the new Git workflow will be the presence of per-issue repositories and pull requests on Drupal.org issues without forking the issue conversations. Drupal.org will use git namespaces to provide every developer working on an issue with their own branch. Developers will be able to pull in the latest changes from HEAD, or changes from other users’ branches. Drupal.org will be able to summarize the commits, take the changeset and run tests, and help maintainers manage the merge process to push changes upstream.

This architecture will make additional features possible as well:

  • The patch based workflow will continue to work - behind the scenes Drupal.org will create commits on namespaced branches from these patches so that these code contributions will be first-class citizens with the new git workflow.
  • We will be able to provide an inline editor for code in issues - simplifying the workflow for contributions such as code style fixes, documentation, quick typo corrections, etc.
  • We can provide the option to compare any two changes in an issue, giving us automated interdiff functionality.
  • We can identify merge conflicts across issues - to hopefully prevent conflicts across issues before they become too deeply entangled.

This planning work culminated in a presentation at DrupalCon Los Angeles - where the community provided some great feedback, and dove into help us with some architectural components during the extended sprints.

Implementation of the new Issue Workspaces architecture will certainly take some time - but we’re excited to have a plan to work from as we move forward.

Community Initiatives

Two Factor Authentication

May saw the initial roll out of Two-Factor Authentication on Drupal.org. Users with elevated privileges on Drupal.org now have the option of enabling TFA, and this may become required for all elevated roles in future.

Next we want to make two factor available to all authenticated users on Drupal.org. However, before we can allow every user to enable two factor it is important that we create a support policy for resetting accounts with TFA enabled, which is still under discussion.

DrupalCI

DrupalCon Los Angeles was a great opportunity to meet with the community and talk about the current state of DrupalCI, and it’s upcoming release.

As of the end of May, DrupalCI is very close to being ready for integration on Drupal.org. All of the environments requested for the MVP deployment are functional, and the Drupal Association staff is getting ready to demo the integration with Drupal.org on a development site - at the same time work is continuing on the results site componenet and the test-runner’s results publishing capabilities.

DrupalCI will be rolled out in parallel with the existing PIFT/PIFR infrastructure for at least a few months following initial deployment as a sanity check.

Localize.Drupal.org

Click-testing has identified several additional issues going into the end of May, and the Association team continues to work on knocking the issues down as they appear. When the current set of identified issues is resolved, we intend to notify the most active translation groups and ask them to perform a final round of testing on the staging environment.

When any issues from that final round of testing are resolved, we will deploy the D7 version of Localize.drupal.org.

Revenue-related projects (funding our work)

DrupalCons

DrupalCon Los Angeles was a productive and fun event for the community and the Association staff - in every way a success. At the conference we made several announcements about the upcoming DrupalCons, including 2016 locations.

First, we announced the opening of the call for papers for DrupalCon Barcelona, September 21st-25th. The call for papers for Barcelona closes on June 8th.

We then announced our next two conferences, and launched their websites.
DrupalCon Asia will be held in Mumbai in February of 2016.

And the next DrupalCon North America will be held on May 9th-13th, 2016 in New Orleans!

Sustaining Support and Maintenance

The Git servers replacing our existing Git infrastructure are nearly ready for thorough testing and deployment. These servers give us a highly available cluster for git.drupal.org, in addition to increased storage capacity, a newer operating system, and dedicated hardware for Git services on Drupal.org.

Our Fastly CDN deployment for downloads (ftp.drupal.org) was a success, and soon to follow is the same new architecture for updates traffic (updates.drupal.org). This new architecture uses dynamic purging to reduce the number of update requests served by our origin servers. It also decreases the latency between packaging a release and serving the update data from a number of minutes to a few seconds.

---
As always, we’d like to say thanks to all volunteers who are working with us and to the Drupal Association Supporters, who made it possible for us to work on these projects.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra.

May 16 2015
tvn
May 16

Look for links to our Strategic Roadmap highlighting how this work falls into our priorities set by the Drupal Association Board and Drupal.org Working Groups.

Community User Role Expanded

The community user role which we introduced in March will now be automatically granted to users who reach a certain level of participation on Drupal.org. While the exact activities that can grant this role will not be explicitly published (as we do with other spam prevention measures) the activities are representative of those an engaged community member would take while participating on Drupal.org.

Existing users who have already reached the required level of contribution will receive the role upon their next activity on Drupal.org. As of the end of April the automatic role granting had extended the Community user role to more than 5000 users.

Content Strategy and Visual Design System for Drupal.org

and

Making Drupal.org Search Usable

During April the Association staff focused on communicating the results and recommendations of our Content Strategy work with the Working Groups and the Drupal Association Board of Directors.

A deep investigation of the current organization of content on Drupal.org, the workflow provided by Drupal.org for our User Personas, and the governance of content on Drupal.org has brought us to a comprehensive proposal for the future state of Drupal.org.

These proposals involve creating new sections on Drupal.org that better match to common user activities and better content types to support those activities. As we begin organizing Drupal.org into new and updated content types we’ll also be rolling in our initiative to improve search on Drupal.org. As we work on each content type we’ll be assessing the search facets for each type.

The next step to move this proposal forward has been to create issues for the specific proposals that have evolved from the content strategy project to date and the feedback from the Working Groups.

This issue and child issues that follow are based on the findings of the Content Strategy project performed by the Drupal Association staff in partnership with Forum One Communications during December 2014 - April 2015.

Community Initiatives (D8 Blockers)

DrupalCI

Drupal Association staff and community volunteers have continued pushing hard to get DrupalCI production ready and integrated with Drupal.org.

The community helped tremendously by providing some formal guidance into the minimum viable and ideal state of the test environments.
Association staff has the primary environment successfully running all tests, and will be working on the additional environments as well as the Drupal.org integration in the run up to DrupalCon Los Angeles.

Again - tremendous thanks to our community volunteers who sprinted with us in Portland: Jeremy Thorson, Nick Schuch, Bastian Widmer, Ricardo Amaro, Paul Mitchum, Mike Prasuhn, Karoly Negyesi-- and to Shayamala Rajaram, Angie Byron, and Jonathan Hedstrom who helped us from afar!

Localize.Drupal.org

In partnership with the community members who have been working on the port of localize.Drupal.org to Drupal 7, association staff have been working to get this migration across the finish line.

We focused fire on the issues found in click-testing, and hope to deploy localize.Drupal.org on Drupal 7 in May.

Revenue-related projects (funding our work)

Try Drupal

We’ve created Try Drupal with our Premium Hosting Supporters to make it easier for CMS evaluators and Drupal.org newcomers to test and work with a Drupal demo site. The Program will showcase a selection of Hosting Companies where a new user can quickly (in less than 20 minutes) sign up and have a Drupal demo site up and running for them to use for free.

DrupalCons

It’s almost time for DrupalCon Los Angeles! In the run up to DrupalCon Los Angeles we’ve been fixing bugs on Events.Drupal.org and preparing for the launch of the DrupalCon Barcelona full site.

We’ve also just started planning out our work for the next Cons to be announced at DrupalCon Los Angeles - more to come there after Los Angeles!

Sustaining Support and Maintenance

Pre-Production Infra Rebuild

An issue was reported to the Drupal.org infrastructure team that uncovered an installed rootkit on our pre-production (dev and staging) environment on April 19th. We stopped all services on these servers. The access was gained through an open VNC port on our OpenStack environment that allowed hijacking of an open console session. The attacker was attempting to create a distributed denial of service (DDoS) attack on targeted IPs.

There is no evidence that information was taken from our staging database or that user information was compromised.

To ensure site integrity, we rebuilt our staging and development environments. Our infrastructure team took the opportunity during the rebuild to address some best practices and better security configuration options. The majority of these environments are now on Amazon Web Services. Particularly for our development environments, this gives us options for more easily scaling up and down our development needs, and gives us more separation between production and pre-production servers.

---
As always, we’d like to say thanks to all volunteers who are working with us and to the Drupal Association Supporters, who made it possible for us to work on these projects.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra.

Apr 15 2015
tvn
Apr 15

The first initiative on the Drupal.org 2015 roadmap is ‘Better account creation and login’. One of the listed goals for that initiative is “Build a user engagement path which will guide users from fresh empty accounts to active contributors, identifying and preventing spammers from moving further.” This is something Drupal Association team has been focusing on in the last few weeks.

The first change we rolled out a few days ago was a ‘new’ indicator on comments from users whose Drupal.org accounts are fewer than 90 days old. The indicator is displayed on their profile page as well. We hope this will help make conversations in the issue queues and forum comments more welcoming, as people will be able to easily see that someone is new, and probably doesn’t know yet a lot about the way community works.

Today we are taking another step towards making Drupal.org more welcoming environment for new users. But first, a bit of background.

New users and spam

It is not a surprise for anyone that a big number of user accounts registering on the site are spam accounts. To fight that and prevent spam content from appearing on Drupal.org, we have a number of different tools in place. Of course, we don’t want these tools to affect all active, honest users of the site, and make their daily experience more difficult. To separate users we are sure about from those we aren’t sure about yet, we have a special ‘confirmed’ user role.

All new users start without such a role. Their content submissions are checked by Honeypot and Mollom, their profiles are not visible to anonymous visitors of the site, and the types of content they may create are limited. Once a user receives a ‘confirmed’ role, his or her submissions will not be checked by spam fighting tools anymore; their profile page will be visible to everyone, and they will be able to create more different types of content on the site.

This system works pretty well, and our main goal is to ensure that valid new users get the ‘confirmed’ role as quickly as possible, to improve their experience and enable them to fully participate on the site.

The best way to identify someone as not a spammer is have another human look at the content they post and confirm they are not spammers. Previously, we had a very limited number of people who could do that-- about 50. Because of that, it usually took quite some time for new user to get the role. This was especially noticeable during sprints.

Today we’d like to open a process of granting a ‘confirmed’ role to the thousands of active users on the site.

‘Community’ user role

Today, we are introducing a new ‘Community’ role on the site. It will be granted automatically to users who have been around for some time and reached a certain level of participation on Drupal.org. Users who have this role will be able to ‘confirm’ new users on the site. They will see a small button on comments and user profile of any user who has not yet been confirmed. If you are one of the users with ‘Community’ role, look out for this new Confirm button, and when you see one next to a user - take another look at what the person posted. If their content looks valid, just click ‘confirm’. By doing so, you will empower new users to fully participate on Drupal.org and improve their daily experience on the site.

We expect to have at least 10,000 active users with the ‘Community’ role. With so many people to grant the ‘confirmed’ role, new users should be confirmed faster than ever before.

If you aren’t sure if you have the ‘community’ role or not, don’t worry. We will send an email notification to every user whose account receives the new role. The email will have all the information about the role and how to use it.

Thanks for helping us make Drupal.org a better place!

Apr 09 2015
tvn
Apr 09

Look for links to our Strategic Roadmap highlighting how this work falls into our priorities set by the Drupal Association Board and Drupal.org Working Groups.

Community User Role

The Community user role is the next step of a larger project of improving user role progression on Drupal.org. We began this work by streamlining the account creation and login workflow, which makes it easier for newcomers to jump into Drupal.org and contribute without losing context. The Community user role extends this work further by providing new tools to our existing community members and broadly expanding the base of users who can help shepherd newcomers into the community.

We started by simply adding a “New” indicator to all user accounts under 90 days old. It’s a small but critical change that helps long standing community members recognize newcomers to the fold, and encourages them to give these new users a warm welcome and a bit of extra help.

The basis of the Community user role is the old spam fighter role, which previously was manually granted to only a small subset of users. This role has primarily been responsible for confirming that other users on the site are not spammers (by granting them ‘trusted’, now called ‘confirmed’). The expanded Community role has the same job - to confirm that users are human - but will now be a role that can be automatically achieved when users reach a certain level of engagement on Drupal.org. We expect as many as 10,000 users to receive this role in the initial grant when the new feature is enabled. This should dramatically increase the attention paid to confirm new users, and make the process of confirming new users at code sprints and training days much, much easier.

The role itself has already been created, and the ‘confirm’ button appears on user profiles. Early April we’ll make sure that users with Community role can confirm users within comments as well - and then enable the initial role grant along with a communication to all Community users.

Issue Comment Attribution and maintainer Credit UI

In mid march we launched the UI for attributing comments as individuals - as individuals on behalf of an organization - or as individuals on behalf of an organization and/or a customer.

Since the release of the comment attribution feature 3 weeks ago, we’ve seen 5,564 comments in the issue queues attributed to an organization, representing around 14% of total comments in the issues queues.

We’ve also just launched the UI for project maintainers to take the attribution data and store final credit for the users and organizations.

All these steps bring a greater level of transparency and introspection to the project and let give us some real data about how Drupal is driven forward. Work on this attribution system will be ongoing, with an option to explicitly attribute comments as a volunteer being released shortly, and work towards integrating these attributions into commit messages coming up soon. We’ll also be updating both organization and user profiles to better display the work that has been credited in issues.

We have scheduled time with a community member that has extensive Solr configuration experience to see what quick wins we can achieve through better configuration.

More extensive search improvements are going to come out of the content strategy work as we define the most important information to show per content type when they appear as a result in search.

The draft Governance plan outline was finished and presented to Working Group members last month. This follows previously shared draft Drupal.org Content Model. Forum One was busy working on the first draft version of the updated Site map for Drupal.org.

In the second half of the month we were focused on working out detailed content types outline. We had a set of brainstorming meetings, where we discussed how all those potential new content types could be implemented technically. Those brainstorms, as well as helpful feedback from the Working Groups, led us to some of the new ideas and changes to the original plans. Hence we are now working on the next revision of all content strategy deliverables, revision which will incorporate all feedback from the Working Group members we have so far.

At this point all the different conversations about separate parts of the whole content strategy project fall into place and we see a clear picture of future state content strategy and information architecture of Drupal.org. We are excited to transfer this vision into a set of slides we can share with the Working Groups, Board and the Drupal Community.

We’ve just wrapped up hosting the DrupalCI sprint made possible by Drupal 8 accelerate. It was a herculean effort, but we made tremendous progress.

The architecture of the complete stack was built out, the test runner code built to it’s final form, containers for test environments created, and we ran through the complete chain from API ? Test Runner ? Results Site. There is still significant work ahead, but the community members who joined us in Portland did phenomenal work and put in long nights and extra days to produce an impressive testing suite.

Association staff architected the integration point between Drupal.org and the DrupalCI API and designed the UI for interacting with DrupalCI in the issue queues. On April 8th, association staff and the community volunteers we sprinted with met to recap the sprint and discuss the roadmap items that remain.

Special thanks to our community volunteers who sprinted with us in Portland: Jeremy Thorson, Nick Schuch, Bastian Widmer, Ricardo Amaro, Paul Mitchum, Mike Prasuhn,
Karoly Negyesi-- and to Shayamala Rajaram, Angie Byron, and Jonathan Hedstrom who helped us from afar!

Revenue-related projects (funding our work)

Try Drupal

Early in April, we’ll be releasing some small changes to the Drupal.org home page -- changes that we will continue to iterate on over the course of the coming months. Primarily we’re trying to create rational pathways through the front page for each of our user personas, as well as updating the homepage to better promote and support some of our revenue programs. Try Drupal is one such program that serves both goals.

For Newcomers to the Drupal community Try Drupal will ensure that their first experience with Drupal is first class, by helping these users create a Drupal site in 20 minutes or less. In return, our partners providing this service get to put the best of their work forward together with the best of Drupal.

DrupalCon Barcelona

Even as we ramp up to DrupalCon Los Angeles in May, we’re getting ready to release the full site for DrupalCon Barcelona. This will be the second site on the new events.Drupal.org unified site, so we’ve be proving out some of the work we did to make it multi-event friendly, and making some additional adjustments and changes as we need them.

We’ll also be preparing for announcements for next year’s cons (Shh!) so there’s some additional UX and feature work underway to support those upcoming sites as well.

Sustaining Support and Maintenance

Elections 2015

Elections were a great success this year. Improvements to the candidate profiles, ballot pages, and voting UI helped us reach our highest level of community engagement in Board elections. 24 candidates from 14 countries nominated, and with 1,432 ballots cast, we doubled our voter turnout compared to last year.

Congratulations to Addison Berry who joins as the new Director-at-Large from the community!

We’re collecting feedback on the experience from both candidates and voters and will continue to improve the elections process next year.

Fastly

The Drupal.org updates infrastructure (updates.drupal.org) is next to receive an architecture refresh. We are working to move the updates infrastructure to use a similar “instance purge” model, allowing for updates to be delivered more quickly. This also lets us set a very long TTL because new updates will purge the previous versions.

---
As always, we’d like to say thanks to all volunteers who are working with us and to the Drupal Association Supporters, who made it possible for us to work on these projects.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra.

Mar 11 2015
tvn
Mar 11

One of our long standing traditions here in the Drupal Association was to give community regular updates on the latest Drupal.org related activities in a form of week notes posts. We’ve been publishing those for over 2.5 years now and it feels like the time has come for a slight change in the format.

From now on we’ll publish monthly ‘What’s new on Drupal.org’ posts, which will showcase new and upcoming features, functionality and user experience improvements. We’ll schedule these around the public Board meetings, so that both the Board and community get the same information at the same time.

So here is our first update in this new format..

What’s new on Drupal.org: February 2015

Look for links to our Strategic Roadmap highlighting how this work falls into our priorities set by the Drupal Association Board and Drupal.org Working Groups.

Better account creation

Account creation improvements

Account creation is now much more streamlined, quick, and lets people get back to the context they started from once the registration is done. Recently added 2nd step of registration allows us to prompt users to quickly and easily fill out the most important fields of their user profile, upload picture and sign up for Newsletters, before proceeding back to the task they were at before registering.

Newsletters signups right on your Drupal.org profile

While working on account creation improvements, we wanted to make it really easy for new users to see what kind of newsletters are available and sign up for the ones they are interested in. Previously only Mailman-powered newsletters were available for sign up on Drupal.org, while for the Drupal Association Newsletter we used MailChimp.

MailChimp allows us to send beautifully designed html emails, provides great content creation experience and detailed analytics. Thus we decided to standardize on one solution and migrate existing newsletters from Mailman to MailChimp. This is partially done, with Security Announcements and Maintainer News following soon.

MailChimp module is now installed on Drupal.org, and all various newsletters are available for sign up right on your user profile edit form. Users will also be able to unsubscribe from their profile or using the unsubscribe links provided through MailChimp.

Organization and user profile improvements

Issue comment attribution and credits

Both issue comment attribution and issue credit UI are ready. At the beginning of March, we opened them for community testing. The feedback is pretty positive so far. Deployment is tentatively scheduled for March 12th.

Content Strategy and Redesign

The draft Drupal.org content model was presented to Working Group members and we are now collecting and incorporating feedback, while at the same time working on a more detailed outline of content and entity types. The outline will include detailed information about fields and settings, as well as view modes with wireframes per content type.

The next deliverable, which is about ready for Working Groups feedback is Content Governance Plan, which reflects the new content model and suggest some improvements in the way we govern content: create, edit, moderate, archive and delete.

The third deliverable, which is nearly done as well, is the Communication Channels Plan, which aims to answer the long standing question of “what is the one place I need to go to to find all important Drupal community news and announcements?”.

The next deliverable we are about to switch out focus to is the updated Drupal.org Site Map, which will reflect suggestions for better IA and navigation on *.drupal.org.

DrupalCI (community initiative and Drupal 8 blocker)

DrupalCI may move into a formal initiative in March as staff works to implement a production environment with the help of the community members that have been involved in the architecture and development.

The test runner is nearly working. Several major portions of the stack are in production—if not yet totally configured—as well as the PrivateTravis containers running php 5.4, 5.5, and 5.6 with mod_php. Overall, it is proceeding on track to have an MVP ready by the end of the sprint sponsored via Drupal 8 accelerate program, which is scheduled to take place at the end of March in Portland.

Revenue-related projects (funding our work)

DrupalCon Los Angeles

On February 25th, registration for DrupalCon Los Angeles went live on the new Drupal Events (events.drupal.org) subsite. Events will be the primary site for all DrupalCon websites moving forward as well as the archive for past events. This will give us great flexibility with historical reporting and make maintenance and security releases for DrupalCon websites more efficient. It also means that any new feature development for DrupalCon websites now benefits all future con sites.

The first of those new features is a set of improvements to registration. We’ve streamlined the experience of purchasing a ticket both for individual users, but also for users who may be purchasing large blocks of tickets for their organization. There are three time-saving new registration features:

  1. Users can now copy their registration data from a previous ticket. This means that a user purchasing several kinds of tickets such as the DrupalCon ticket and a Business Summit ticket can save time entering fields. It also means that users attending future cons will be able to save time entering their registration data for the next Con.
  2. Someone purchasing a ticket on behalf of another attendee can now enter that attendee’s email address and a link to redeem the ticket will be sent to them. This saves the purchaser time, and allows the attendee to keep their registration data private.
  3. Finally someone purchasing a large block of tickets who does not yet know who will attend can now purchase reservation codes which can be given out to attendees to be redeemed.

We’re also working closely with our early registrants and DrupalCon sponsors to further streamline these new features.

Sustaining Support and Maintenance

Elections 2015

In February, we spent some time polishing the nominations and voting functionality on assoc.d.o, which powers 2015 Drupal Association Board Elections. This year we have much better looking nomination pages, as well as more smooth voting process. Voting is open until March 20. Have you voted yet?

Fastly

The Drupal.org download infrastructure (ftp.drupal.org) is undergoing an architecture refresh. Fastly has signed on as a Drupal.org Technology Supporter and the existing FTP mirror infrastructure is being dissolved in favor of the CDN backed by Drupal.org’s static web servers.

Server Density

Server Density (drupal.serverdensity.io) was selected as our replacement for Nagios alerting and Munin graphing. Server Density provides us with an alternative to OSL’s shared Nagios and Munin instances, and does not require us to host and manage our own internal monitoring service. Server Density also supports Nagios checks and integrates nicely with our existing infrastructure.

---
As always, we’d like to say thanks to all volunteers who are working with us and to the Drupal Association Supporters, who made it possible for us to work on these projects.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra.

Jan 25 2015
tvn
Jan 25

Now that we are few weeks into 2015, we’d like to look back at 2014 and share some interesting numbers about Drupal.org.

Audience

Last year Drupal.org received almost 48.9 million visits from 21.2 million unique visitors. The spike around September/October is due to spam-related traffic, and, of course, DrupalCon Amsterdam.

Users

152,200 users logged in to Drupal.org at least once during the year. Out of those, 31,466 users performed at least one activity on the site, such as commented, created a node or committed code.

More than 21,500 people left a comment or more in the issue queues. More than 4,000 people commented in the Drupal core issue queue.

Commits

Overall 145,907 commits happened on Drupal.org, with more than 4,000 commits to Drupal core specifically.

More than 3,200 people committed code to contributed projects (not counting Drupal core), with an average of 37.43 commits per user.

More than 1,400 people got commit mention in Drupal core patches.

Comments & Issues

Our users left 569,217 comments, 94% of them were comments in the issue queues. 30% of all comments in the issue queues happened in Drupal core queue.

On average there were 22.4 comments per user, with 38.74 comments per user in the Drupal core issue queue.

Our users created 78,505 issues, with an average of 4.55 issues per user.

5,192 contributed projects were created on Drupal.org in 2014. 31% of those are sandbox projects.

Infrastructure

On the infrastructure side our uptime was 99.97% over 12 months, and the average full page load time for the year is 3.64 across Drupal.org. It improved throughout the year; we are down to 3.08 as an average for December. Our time to first byte response was 1,374ms in January; we are down to 441ms for December.


Drupal.org testbots tested over 33,300 patches. An average test queue and test duration times for Drupal 8 core were about 35 minutes each.

Support

On support front 82% of issues in Drupal.org-related issue queues got a response within 48 hours after being created.

An average response time (time between an issue was created and first comment not by issue author) across all issue queues on Drupal.org was 82.87 hours. For Drupal core issue queue this number was 60.68 hours. For Drupal.org related queues 34.19 hours.

* * *

Full stats you can find in the 2014 stats spreadsheet.

Compared to 2013 some of the user activity numbers go down, which is directly related to the phase of the Drupal release cycle. Right after Drupal 7 release user activity peaked and then was slowly going down as Drupal 7 and contrib ecosystem matured. We are looking forward to Drupal 8 release! In the recent Drupal Association community survey about 80% of respondents said they have firm plans to adopt Drupal 8, suggesting that release will cause a huge boost in user activity on Drupal.org.

2014 was a great year, and thank you for spending some part of it on Drupal.org! We are excited to see what 2015 will bring.

Dec 22 2014
tvn
Dec 22

It’s been awhile since our last week notes post, but before a lot of you will go off to celebrate holidays, we wanted to tell you about some of the recent exciting changes on Drupal.org

Responsive Drupal.org

As a New Year’s present to all Drupal.org visitors, today we deployed the initial responsive version of Bluecheese theme. This means that Drupal.org will now look much better on small and large screens. Drupal.org subsites on Drupal 7, such as Api.Drupal.org and Assoc.Drupal.org, will follow as they are tested.

The work on responsifying the theme started at Drupal Dev Days in Szeged earlier this year, with the final testing happening during BADCamp in November.

We’d like to thank our wonderful volunteers for making this happen:
LewisNyman, dasjo, emma.maria, mallezie, Manuel Garcia, sqndr, thamas.

This is the initial deployment, though more page-by-page improvements will be coming. Drupal.org is a big site, with lots of pages that will need specific mobile-first design changes.

Improved user profiles

During the last few months we’ve been slowly migrating user profile fields to Field API fields, which gives us more flexibility to improve profile pages. While migrating the fields, we started to rearrange layout of user profiles. A lot more work will happen on layouts now that field migration is complete, but even now you can see that profiles look much better.

One particular change which happened during fields migration is the ‘My mentors’ field layout. Previous it was just a list of blue links, but now we actually show user pictures of mentors.

New database servers

Drupal.org’s database servers were due for a hardware refresh. New hardware was purchased, provisioned and deployed for Drupal.org that has improved page response times across the site. The new hardware brings solid state drives, and quadruples the amount of processors and memory.

---
As always, we’d like to say thanks to all volunteers who are working with us and to the Drupal Association Supporters, who made it possible for us to work on these projects.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra

and

Dec 14 2014
tvn
Dec 14


As part of our mission to reinvent Drupal.org, we’ve been digging deep to understand who uses the website and how. At DrupalCon Austin, we began the process of discovering the personas of users who visit Drupal.org: to do so, we interviewed numerous Drupal.org users and asked questions about how frequently they use Drupal.org, how they use the website, their frustrations with Drupal.org, the things they enjoy about the site, and how we can make it easier for people to learn, use, and connect on Drupal.org.

Once we had that data, we set about looking for patterns and common themes. We built categories where we grouped people's similar experiences and frustrations together, and at the end of the process we had come up with five distinct personas that can apply to everyone who visits Drupal.org. These personas detail our users’ familiarity with Drupal software and Drupal community, how they use Drupal.org, how they contribute (or don’t), and more.

The five personas that we drew up are based on proficiency in Drupal and the Drupal ecosystem. They are:

  • Newcomer: This person has heard of Drupal, but has never built a Drupal site and doesn’t know where to start.
  • Learner: This person knows a bit about Drupal and the general Drupal ecosystem. He or she may have built a Drupal website, but likely has used only a few contrib modules and hasn’t made any customizations.
  • Skilled: This person understands and is fluent in Drupal-specific terminology, can build a Drupal website themselves using contributed modules, themes or distributions, or with the help of Drupal service providers. She or he has spent a decent amount of time working with Drupal, and is lightly engaged with the community, often not directly, via some sort of liaison.
  • Expert: This person has a deep understanding of Drupal and the Drupal ecosystem, knows how to build advanced websites with Drupal. Expert typically has been working with Drupal for at least a couple of years, is actively engaged with the community online and via local/national events, and actively contributes back in a variety of ways.
  • Master: This person has pervasive knowledge of Drupal and the Drupal ecosystem. He or she knows how to build Drupal websites of great complexity, is deeply engaged in the Drupal community, knows and has access to other Masters. Usually this person has been using Drupal and been around the Drupal community for a long time.

Proficiency-based personas are a new facet through which we can look at our community. It’s important to note that these personas are NOT only about developers. All kinds of roles can be on different levels of this ladder — UX designers, project managers, and business owners can be Experts and Masters, just like developers and themers. Simultaneously, people can have different backgrounds and be experts in other areas, but when it comes to fluency in Drupal and Drupal ecosystem, they would be represented as Newcomers, or Learners, or any of the other personas.

How will we use personas?

User personas will guide feature prioritization and feature development for Drupal.org, as we improve the site to make it easier for our users to progress from Newcomers to Masters. There are a variety of different ways we can go about it, but since our resources are limited, we will focus on just a few critical areas that will have the biggest impact on the overall user experience. So, to start our work, we’ll be focused on removing barriers and helping our users move more easily from Learners to Skilled. We found that our users have great success moving from Newcomer to Learner today, whereas moving from Learner to Skilled is much more difficult, since so much of the project is focused on doing things “the Drupal way” and learning the processes. Our secondary focus will be on moving users from Skilled to Expert.

Growing our pool of Skilled users is crucial, because by doing so we grow the number of people who own and/or build websites using Drupal, thus grow Drupal adoption. On the path from Skilled to Expert is when our users begin to give back by contributing patches, writing documentation, building and sharing modules and themes, helping others in the issue queues, and bringing in their friends. By growing the number of Skilled and Expert users on Drupal.org, we’ll directly grow our community. It’s a win-win.

By growing Drupal adoption and growing our community, we directly support our mission and goals as an organization (you can read more about those in our 2015 Leadership plan and budget), and that’s why improving Drupal.org is one of our organizational imperatives in the coming year. The 2015 Drupal.org roadmap outlines the numerous ways we’re planning to do it.

As we use personas in our work, you may hear us refer to our “Primary” (Learner and Skilled), “Secondary” (Expert), and “Tertiary” (Master and Newcomer) personas — these distinctions correspond to the order of conversions we look to make easier, not to the users’ importance. Every Drupal.org user is important to us!

As we modify Drupal.org, we’ll be using the personas to help us make the experience for the whole community better. After all, that’s what these personas are — a representation of the entire Drupal community. To help bring our personas to life, we talked to five different community members, each representing one user persona. Over the next few days we’ll share the stories of each person’s unique Drupal journey so that we can see how they got to where they are now. We’d like to say a big thank you to each of our volunteers for sharing their personal stories — as always, they’ve reminded us how fantastic our community really is.

At the end of the series, we’ll close it all off with interviews with several prominent community members who will share their views on how personas can be used outside of Drupal.org development.

We enjoyed working on the user research project and are excited to share user personas with the Drupal community. As a reminder, you can view and download the full report. Take them, use them, go out and make great things!

Nov 24 2014
tvn
Nov 24

Several weeks ago, we issued an RFP for Drupal.org Content Strategy project. We got a number of great submissions, and the next couple of weeks the Drupal Association staff and the Drupal.org Content Working Group members spent reviewing proposals and interviewing potential vendors.

Today we are happy to announce that we’ve selected a vendor for the content strategy project: Forum One, an open-source digital agency.

Their proposal met all our project requirements and outlined a solid plan for how we can make this project happen. During the interviews Forum One impressed us with their professionalism, passion for content strategy, their extensive experience working on large content strategy projects, and their deep knowledge of the Drupal community and Drupal.org, the website.

We believe that together our staff and Forum One team will make this project a success. Can’t wait to start working and improve Drupal.org content for all our varied audiences.

Oct 23 2014
tvn
Oct 23
Seomagic84's picture

You made a superb decision. Now only people who wanted to be here can be here.

dgtlmoon's picture

Is it possible to ban 'recruitment' agencies from spamming Groups too? perhaps a no-recruitment-agency policy?

Cathy Theys

Feb 06 2014
tvn
Feb 06

The Drupal.org Software Working Group is looking for volunteers to take on leadership roles and help guide development of the Drupal.org Community Tools, which include Forums, Drupal Planet, User Groups (groups.drupal.org), User Profiles, User Dashboard.

Basically, this team is about all the tools community uses for things other than code development. Historically all these tools were separate from each other and there was no clear vision or single roadmap/plan. We hope that by improving the decision making structure we can expedite development of these tools and ensure they meet community needs.

Take a look at the current proposal for the scope, roles, responsibilities and authority of the team.
For the full background see this post.

The Software Working Group will appoint the new team by the end of February, 2014. Right now we are looking for candidates for the roles on the team. Interested? Know someone who you think would be a great candidate? Let us know by February 20!

Jan 27 2014
tvn
Jan 27


Recently we published our Tech team plan for the first quarter of 2014. To ensure that we are not only telling you about our plans, but also update on the progress as we go, we are bringing back our traditional bi-weekly ‘week notes’ posts. Welcome to the first week notes of 2014!

Drupal 7 upgrade cleanup

We managed to fix quite a few critical and major issues in the past 2 weeks. Project usage and download statistics are now fixed, text search is improving, root cause of the missing Git commits on Drupal.org is also fixed (thanks marvil07 for help!), so we hope to close a few related issues soon.

We have only one critical issue left and one critical D.o UX issue, both of which are our top priority for the next two weeks.

If you wanna help us finish the clean up sooner, here are the few issues you are welcome to work on:

BDD tests

D7 staging site needs some work to improve its stability, before we can setup regular BDD test runs. We’ve been working on this on and off during the past 2 weeks, not done yet, but slowly getting there.

Public Bluecheese

Transition to the public plus private (which includes branded elements) versions of Bluecheese finished. There is now a public repository, which should make it much easier for people to contribute to Drupal.org. Public version of the theme doesn’t look exactly like the full theme and it won’t be supported as installable theme for other sites. However it does just enough for it’s purpose, which is making it easier for volunteers to contribute to Drupal.org. We’d like to thank Lewis Nyman and Melissa Anderson for their help on this project.

Drupal.org changes tool

The tool is under development. The first step is upgrading infrastructure.drupal.org to Drupal 7, which is almost done. A few pieces left, for example, porting bounce_handler to Drupal 7, which is being done by Ricardo Amaro (thanks!).

Setting up OpenStuck cluster

In the past two weeks the server has been installed and initially configured with OpenStack Havana thanks to help from Jason Ford and the team at Blackmesh. We are continuing to configure the server this week, we have added a base instance image, and have requested a new VLAN for the cluster. The next steps to finish configuration are setting locking down firewalls, setting up web access and working through issues that crop up along the way.

Moving db4 to the new server

This move will let us create a high availability MySQL cluster for Drupal.org subsites. Currently the new db4 is being configured and should be in production soon.

Moving devdb to another server

This is now finished! Devdb now has a much better hardware and we’ve got 2x space for our development environments.

Drupal Store

The store on association.drupal.org is in progress, we are now testing the first prototype.

Drupal Job Board

We are still collecting proposals for the Drupal Job Board, take a look at this post for more details on the RFP. Proposals will be accepted through February 7, 2014.

Lastly, there is one more thing we are working towards improving for Drupal.org, which wasn't in our Q1 plan post, and that is getting better at tracking various Drupal.org metrics...

Drupal.org dashboard

At the end of 2013 we've made a list of stats we'd like to track for Drupal.org. Some of them are the ones listed in the Drupal Association's 2014 Leadership Plan, but there are also a lot of additional userful metrics. Later this year, around June, we are planning to automate the process and make the stats available as a pretty dashboard somewhere, e.g. https://drupal.org/metrics. However until we automate this, we decided to collect the stats manually once per month, to ensure we have the data. On January 1st we pulled the data for the first time, and you can find various stats for December 2013 and the whole 2013 year in this Drupal.org 2013 Dashboard. We will publish the stats for January in our week notes in February, and we'll do so every month. We hope this will be of use not only for the staff but for the community at large!

This is it for the week notes today. Follow us on Twitter for updates and expect the next post in 2 weeks!

As always, we’d like to say thanks to all volunteers who are working with us and to the Drupal Association Supporting Partners, who made it possible for us to work on these projects. The Supporting Partner Program crowd sources funds that pay for the development team’s time and Drupal.org hosting costs.

Jan 23 2014
tvn
Jan 23

At the end of February, the Drupal.org Software Working Group will be appointing decision-makers for Drupal.org Community Tools. Interested? Know someone who you think would be a great candidate? Let us know by February 20!

Overview

The Drupal.org Software Working Group (drumm, eliza411, tvn, webchick) was chartered to provide vision and direction for Drupal.org by working with the community on Drupal.org software in a number of ways. The first duty on our list is establishing team leadership.

From the charter:
(DSWG) Creates and removes teams for each major area of the Drupal.org websites, which have authority to make software and feature decisions within their scope. The DSWG defines and appoints the leadership roles within each team, such as a technical lead, product owner, QA lead, etc.
Full charter: https://drupal.org/node/1929526

The first team we set out to appoint was Developer Tools Team. That process is at the finish line right now. We are getting ready to announce team members on January 30.

Therefore, it’s time for us to start working on the next team.

Proposal

Community Tools Team Scope

Proposed scope of functionality for the Community Tools Team includes:

  • Forums
  • Planet
  • User profiles
  • User dashboard
  • User groups (groups.drupal.org)

We feel that, outside of the issue queue, these are the sections of the site where community interaction happens. Therefore it makes sense to group them under one leadership team, which will take a high-level look at the current tools and work with the community to find out what kind of tools are needed for they things our community does outside of code development, e.g. for discussions, collaboration on initiatives, events and local groups organization, job search etc.

Roles

  • Product Owner
  • Project Manager
  • UX Lead
  • Section Owner for Forums
  • Section Owner for Planet
  • Section Owner for User Profiles
  • Section Owner for User Dashboard
  • Lead Architect for User Groups

Proposed Timeline

January 23 - February 2 — Online Discussion
Process Facilitator: tvn

Responsibilities, Authority and Conflict Resolution will largely follow the ones we defined for Developer Tools Team. The discussion template below contains some of the questions for consideration for this team specifically.

Discussion Template

  1. Scope: Are the areas proposed related enough to group them and under one team? Is there something else, which should be listed?
  2. Roles: What roles are needed for this area (technical lead, product owner, QA lead, etc)?

February 3 - DSWG to summarize community discussion, publish the final proposal and publish the call for candidates

By February 28, Drupal.org Software Working Group to appoint leadership team

Jan 15 2014
tvn
Jan 15

There is never a shortage of things to do on Drupal.org and that’s never been more true than right now. We have D7 upgrade clean up to finish, as well as variety of improvements that were on hold during the upgrade. We want you to know what our tech team is focused on and where we’re spending our time, so here are some of the pieces in our first quarter roadmap.

Overall themes

Drupal 7 Cleanup

We are still putting as much developer time as possible into addressing the issues raised by the D7 upgrade. A team of two people can only do so much, however, so while we are working on the showstoppers, we also have two other themes this quarter.

Empowering Volunteers

Right now, we recognize that it is really difficult to contribute to Drupal.org as a volunteer. We’re working on eliminating some of the technical barriers so that you can help fix what ails you most with much more ease.

Hiring

Even with an army of amazing volunteers (which all of you are), we simply need more staff to keep the internal workings of Drupal.org up and running well. Rudy Grigar (basic) just joined our team in a DevOps role, and we’ll be adding another Developer and an Issue Queue Support person in this quarter, along with preparing for future hires.

So those are the major themes, but what does the work actually look like? Things change, and our plans might need to change as well, but at this point in time here is our rough to-do list for the next 3 months:

January

D7 upgrade follow ups

The project is done, but we are still cleaning up follow up issues and bugs. As of this week we have 2 critical and 19 major bugs. They, together with the critical issue related to the issue page UX regressions, are at the top of our priority list.

BDD tests

We did write the whole suite of BDD tests during the D7 upgrade. We now want to set up regular BDD test runs against D7 staging site, and incorporate BDD tests into our deployment process, to ensure the quality of our deployments.

Public Bluecheese

Together with lewisnyman we worked on splitting up Bluecheese theme into public and private repos. We are going to finish this work and adjust our build process to work with these new repos.

Drupal.org changes tool

Drupal.org Software Working Group came up with the idea of a tool to manage all changes on Drupal.org from the idea all the way to deployment. The tool will be located on infrastructure.drupal.org, which is currently being ported to D7.

Setting up OpenStack cluster

We have purchased and begun configuration of a new server aimed at hosting virtual machines for the Drupal.org development and staging environments. This server will help Drupal.org improve its development and staging workflow, as well as consolidate existing virtual machines on faster hardware.

Moving db4 to the new server

We have also purchased new hardware to replace the existing db4 MySQL server. The new hardware has dual Intel E5-2450 processors and 64GB of memory. This hardware will replace the existing db4 server and pair well with db3 to create a high availability MySQL cluster for Drupal.org subsites. The old db4 server will be repurposed to provide PostgreSQL databases for services using Postgres.

Moving devdb to another server

devdb is currently housing the development site databases and is running on aging hardware. The new devdb virtual machine will provide faster hardware along with more disk space to enable hosting of more development environments.

DrupalCon Austin site launch

The site is live! We’re almost done with it by now.

Drupal Store

We did have a simple store on association.drupal.org before. It’s wasn’t very user friendly and had few products in it. Brendan will be building a better version, which will let us sell t-shirts and various Drupal-related souvenirs all year long, not just at DrupalCons.

February

D7 upgrade follow ups

We expect to be done with criticals completely at this point and concentrate on finishing up major and normal issues.

2 of the 3 ‘quick win’ features DSWG prioritised for Q1:

  • Back-linking commits on issues
    When a commit is made, which is referencing specific issue, a comment will be posted on that issue to notify issue participants about the commit.
  • Show user pictures on D.o

Finish setting up OpenStack cluster

Continuing with the OpenStack configuration we will be creating configuration management and templates for staging and development virtual machines. Testing will begin on vm creation and deletion, private network integration and separation for development and staging environments, and authentication and authorization.

CiviCRM updates

There are some security related and general Drupal integration improvements we need to do on association.drupal.org to ensure our membership system is working well.

Drupal Store

Finish building the store on association.drupal.org.

March

The 3rd ‘quick win’ feature DSWG prioritised for Q1:

  • Re-design project pages to better display project metrics and statistics

Internal Site Status dashboard

This is another item, which came from the Drupal.org Software Working Group. Internal dashboard will pull together information from all our sub-sites, to let us know at a glance e.g.: which modules on which sites have updates available, which php errors we have in the logs, etc. This should make our day-to-day site maintenance easier and help us catch various problems early.

Migrate staging and dev vms to OpenStack cluster

OpenStack testing should be complete, and we will begin migrating the existing vms to the new cluster. This will give us better performance, more room to grow for our development and staging vms, as well as easier access to manage vms for the community.

CFengine service level configuration to Puppet

We will begin our push to move the remaining CFengine configuration management bits to Puppet configuration management. This will result in Drupal.org configuration residing in the same system and should improve our ability to push out changes quickly.

User Research

Drupal.org Content Working Group is getting ready for a complete website redesign in 2015. First steps are user research and content strategy for Drupal.org. In March we will engage user researcher, who will work with our community through spring, to develop a set of user personas for Drupal.org

Landing pages

As a content improvement and a revenue source on Drupal.org, we want to introduce a few more landing pages this year. We will start working on the first one of those in March. The audience for the page will be determined by the Content Working Group.

Ongoing

Of course, the things above aren’t the only ones we are going to do during these months. There are also ongoing security updates to all sites, general maintenance, answering questions in the issue queue, Drupal.org Working Group meetings, etc.

This is it for us for this quarter. We are bringing back our regular bi-weekly ‘week notes’ posts, and will keep you updated on our progress! You can also expect similar posts at the beginning of each quarter.

Dec 19 2013
tvn
Dec 19

The Drupal.org Software Working Group is looking for volunteers to take on leadership roles and help guide development of the Drupal.org Developer Tools, which include Projects (project pages, releases, issue queues, change records), Version Control (git repositories, commit logs, repository viewer etc.) and Testbots.

This is by far, the most important part of Drupal.org. It's that section of the site which lets the community do their main thing - develop Drupal. It is, therefore, very important to have a clear decision making structure to ensure that we can deploy changes and improve the section to give Drupal contributors all the tools they need to do their jobs.

Take a look at the current proposal for the scope, roles, responsibilities and authority of the team.
For the full background see this post.

The Software Working Group will start appointing the new team on January 15, 2014. Right now we are looking for candidates for the roles on the team, specifically for Product Owner, Project Manager, UX Lead.
Interested? Know someone who you think would be a great candidate? Let us know!

Nov 01 2013
tvn
Nov 01

If you are reading this announcement right now, then we did it! Drupal.org runs on Drupal 7! This was a big and complicated project, which took longer than we expected. But we are finally done!

What changed?

Our goal was a straight port to Drupal 7 without major changes to functionality or layout, but with greatly improved code under the hood. However some things did change, please see Drupal.org D7 F.A.Q. for details. Overall Drupal 7 gives us more flexibility to implement new features and there will be a boost in performance for some of the pages.

NOTE: issues are still being indexed, listings and searches will show incomplete results till the indexing is done.

What’s next?

There probably will be some bugs. If you encounter something unusual, please check the Drupal.org D7 F.A.Q. first. It may be that the change was intentional. If you are sure that you found a bug, please use the D7 upgrade QA queue to report them.

* * *

The only thing we really want to say now is.. let’s party THANK YOU!

Thank you to all of you for being patient with us during this long project. We know it took longer than anticipated and there were some bumps along the way. Our only goal throughout the project was to make Drupal.org better for all of you.

Thank you to all our fantastic contributors. There are so many of them, we even have a special page. Thank you:

Andrei Mateescu / amateescu
Joel Moore / banghouse
Rudy Grigar / basic
Brandon Bergren / bdragon
Tom Behets / betz
Bojhan Somers / Bojhan
Chi / Chi
Ian Carrico / ChinggizKhan
Nell Hardcastle / chizu
Karoly Negyesi / chx
Bill O'Conner / csevb10
Dave Reid / davereid
David Strauss / David Strauss
Meghan Palagyi / dead_arm
Dave Fletcher / dfletcher
Derek Wright / dww
Melissa Anderson / eliza411
Frank Baele / frankbaele
Greg Lund-Chaix / gchaix
Greg Knaddison / greggles
Dylan Tack / grendzy
Jose Marquez / hackwater
Michael Halstead / halstead
Herman van Rink / helmo
Chad Phillips / hunmonk
Jason Savino / jasonsavino
Jonathan Hedstrom / jhedstrom
Jennifer Hodgdon / jhodgdon
Jeremy Thorson / jthorson
KS Sundarajan / ksbalajisundar
Lewis Nyman / lewisnyman
Mark Pavlitski / markpavlitski
Marco Villegas / marvil07
Michael Prasuhn / mikey_p
Mitchell Tannenbaum / mitchell
Nick Veenhof / nick_vh
Narayan Newton / nnewton
Theodore Biadala / nod_
Pradeep Kumar / pradeeprkara
Peter Wolanin / pwolanin
Robert Ristroph / rgristroph
Chris Ruppel / rupl
Sam Boyer / sdboyer
Joel Farris / Senpai
Sam Richard / snugug
Venkata Suresh / sachin2honi
Howard Tyson / tizzo
Tyler Ward / twardnw
Angela Byron / webchick
Steve Edwards / wonder95
Roy Scholten / yoroy

Thank you to the Drupal Association Supporting Partners, who gave us the funding required to make the upgrade happen.

We couldn’t have done this without you!

/ drumm & tvn

Oct 31 2013
tvn
Oct 31

If you are reading this announcement right now, then we did it! Drupal.org runs on Drupal 7! This was a big and complicated project, which took longer than we expected. But we are finally done!

What changed?

Our goal was a straight port to Drupal 7 without major changes to functionality or layout, but with greatly improved code under the hood. However some things did change, please see Drupal.org D7 F.A.Q. for details. Overall Drupal 7 gives us more flexibility to implement new features and there will be a boost in performance for some of the pages.

What’s next?

There probably will be some bugs. If you encounter something unusual, please check the Drupal.org D7 F.A.Q. first. It may be that the change was intentional. If you are sure that you found a bug, please use the D7 upgrade QA queue to report them.

* * *

The only thing we really want to say now is.. let’s party THANK YOU!

Thank you to all of you for being patient with us during this long project. We know it took longer than anticipated and there were some bumps along the way. Our only goal throughout the project was to make Drupal.org better for all of you.

Thank you to all our fantastic contributors. There are so many of them, we even have a special page. Thank you:

Andrei Mateescu / amateescu
Joel Moore / banghouse
Rudy Grigar / basic
Brandon Bergren / bdragon
Tom Behets / betz
Bojhan Somers / Bojhan
Chi / Chi
Ian Carrico / ChinggizKhan
Nell Hardcastle / chizu
Karoly Negyesi / chx
Bill O'Conner / csevb10
Dave Reid / davereid
David Strauss / David Strauss
Meghan Palagyi / dead_arm
Dave Fletcher / dfletcher
Derek Wright / dww
Melissa Anderson / eliza411
Frank Baele / frankbaele
Greg Lund-Chaix / gchaix
Greg Knaddison / greggles
Dylan Tack / grendzy
Jose Marquez / hackwater
Michael Halstead / halstead
Herman van Rink / helmo
Chad Phillips / hunmonk
Jason Savino / jasonsavino
Jonathan Hedstrom / jhedstrom
Jennifer Hodgdon / jhodgdon
Jeremy Thorson / jthorson
KS Sundarajan / ksbalajisundar
Lewis Nyman / lewisnyman
Mark Pavlitski / markpavlitski
Marco Villegas / marvil07
Michael Prasuhn / mikey_p
Nick Veenhof / nick_vh
Narayan Newton / nnewton
Theodore Biadala / nod_
Pradeep Kumar / pradeeprkara
Peter Wolanin / pwolanin
Robert Ristroph / rgristroph
Chris Ruppel / rupl
Sam Boyer / sdboyer
Joel Farris / Senpai
Sam Richard / snugug
Venkata Suresh / sachin2honi
Howard Tyson / tizzo
Tyler Ward / twardnw
Angela Byron / webchick
Steve Edwards / wonder95
Roy Scholten / yoroy

Thank you to the Drupal Association Supporting Partners, who gave us the funding required to make the upgrade happen:

We couldn’t have done this without you!

Oct 28 2013
tvn
Oct 28

The Drupal.org D7 upgrade launch is confirmed. Today is Monday, 28th of October, we have 0 launch blocking issues and performance tests are looking fine. Therefore, we are going to launch on Thursday, October 31st, 2013.

What will the launch process be like?

Drupal.org will be down for approximately 24 hours during deployment. It will be replaced by a static page with a download link for the latest Drupal release available. Sub-sites will stay online, but with user logins disabled. Both updates.drupal.org and ftp.drupal.org will stay online. drush make / dl will work fine, update status module as well.

We will start deployment around 15:00 UTC on October 31st. We expect the site to be back up by 15:00 UTC on November 1st.

We realize this will be a significant inconvenience for users who rely on Drupal.org, and will try to minimize downtime as much as possible.

What if there are problems? Do you have a backup plan?

Yes, we do. If we encounter significant problems during migration, we will roll back to the Drupal 6 version of Drupal.org and restore with a backup made right before migration started.

How can I find out what’s going on during deployment?

What changes will I experience when the site comes back online?

You can find information about the changes in functionality or UI in the Drupal.org D7 F.A.Q. Most pages on the site won’t change as far as layout or functionality. Our goal for this project was a straight port from Drupal 6 to Drupal 7. The only place where you will see significant UI changes is the issue page. This blog post explains what is changing on the issue page and why in detail. In general Drupal 7 gives us more flexibility to implement new features and there will be a boost in performance for some of the pages.

Why aren’t we waiting and upgrading to Drupal 8 once it releases?

The Drupal 7 upgrade began in March 2012. The upgrade took longer than we anticipated due to a variety of reasons that include the scale and complexity of Drupal.org and resource contstraints. We decided to push ahead and complete the Drupal 7 upgrade so Drupal.org can be on the latest release of Drupal, and so we can use the learnings in future upgrades.

How can I access the files from the October 30th security announcements while Drupal.org is offline?

You can use the following direct URLs to the archives containing the updates:

Quiz:

Filefield sources:

Monster menus:

Oct 16 2013
tvn
Oct 16

After a month-long Community QA, we are getting ready to deploy Drupal.org D7 upgrade. During the last couple of weeks we were limiting the number of ‘to-do before launch’ issues to those that are absolutely essential. Currently our launch blocker list consists of the 12 open issues.

We took a look at the upcoming Drupal events to find a quiet week, which won’t interfere with major camps and sprints, and..

Launch date

If by Monday, 28 of October, launch blocker issue is down to 0, we plan to deploy the Drupal.org D7 upgrade on Thursday, 31 of October.

If by Monday, 28 of October, the launch blocker issue count is higher than 0, we will have to postpone deployment for a few weeks.

What will the launch look like?

Drupal.org will be down for approximately 24 hours during deployment. It will be replaced by a static page with a download link for the latest Drupal release available. Sub-sites will stay online, but with user logins disabled. We realize this will be a significant inconvenience for users who rely on Drupal.org, and will try to bring it up as soon as possible.

We will start deployment around 15:00 UTC on October 31st. We expect the site to be back up by 15:00 UTC on November 1st.

Update: both updates.drupal.org and ftp.drupal.org will stay online. drush make / dl will work fine, update status module as well.

What if there are problems? Do you have a back up plan?

Yes, we do. If we encounter significant problems during migration, we will roll back to the Drupal 6 version of Drupal.org and restore backup made right before migration started.

How can I find out what’s going on during deployment?

Twitter accounts to follow are @drupal_org and @drupal_infra. IRC channels: #drupal and #drupal-contribute.

What changes will I see when the site comes back online?

Most pages on the site won’t change. Our goal for this project was straight port from Drupal 6 to Drupal 7. The only place where you will see significant UI changes is the issue page. Some time ago we wrote up this blog post, which explains what is changing and why in detail. We will also publish an F.A.Q. right before launch, which will list all changes you might encounter on the website.

How can I help?

To ensure we are able to launch on time, you can help us by bringing launch blockers count to 0.

Here are the issues:

Oct 11 2013
tvn
Oct 11

The most important event of September was of course DrupalCon Prague. 10 out of 12 members of the Drupal.org Working Groups were there. We finally got to meet each other and work together in person! This means that we spent huge part of the week locked in a meeting room, but.. we had a great time and got a lot done.

Content Working Group

At DrupalCon Prague, the DCWG focused on a number of short and long-term objectives. We finalized interview questions for the Drupal.org site builder landing page and began soliciting survey participants. We also worked on a draft scope of work for user persona research to be conducted in 2014, an editorial content scheduling plan, mission statement, and points of coordination with the other working groups. In addition, work was done to coordinate community efforts around Drupal 8 launch promotion with the official marketing plan developed by the Drupal Association.

If you’re a Drupal site builder, and you’re interested in helping us make Drupal.org a better resource, please sign up for an interview.

We also encourage anyone interested in Drupal 8 (technical or non-technical) to tell us what kind of information you’d like to see on the Drupal 8 resource page(s) by taking this survey: http://www.surveymonkey.com/s/DrupalContent

Software Working Group

In Prague the DSWG presented the summary of the community ideation process and our thinking on the short-term Drupal.org roadmap for 2014. You can read more about this in a separate post: https://association.drupal.org/node/18588

We had a lot of productive discussions about our 2013 projects as well. Progress has been made on our improved Drupal.org development workflow proposal. There is still some work to be done, but we are getting ready to publishing it for a larger community feedback.

Not all sub-site maintainers were in Prague, but we managed to meet SebCorbin and Gabor Hojtsy from localize.drupal.org and discuss with them future plans for the website and next steps on D7 upgrade.

We also spent some much needed time discussing our internal processes and tools, which should help us be more efficient as a Group.

Infrastructure Working Group

During DrupalCon Prague the DIWG made progress towards our main goal for this year: growing the team. We documented all the various tasks Infrastructure team needs help with and published them: https://association.drupal.org/node/18573. During the DIWG community conversation the attendance was much higher than we expected. 14 people attended in total (with Jeremy Thorson joining via Skype). Friday sprint was even more exciting, we had a full table of people working on Drupal.org infrastructure all day long. A number of people were cleaning up Infrastructure queue and got it from 600 open issues down to 300! We also go initial loghost setup done, created BitBucket account for the team and started moving our internal BZR repos there, moved more static camp sites to our servers, etc. Big thanks to everyone who joined us during these days, we hope to work with you in future!

Jul 15 2013
tvn
Jul 15

This week our notes are all about the Drupal.org D7 upgrade.

We are currently at 70 open issues out of 212 total. Which means we fixed 68% and have 32% to go. This matches our hours spent as well. We projected 940 hours, and are currently at 68.5% spent and 31.5% left. And we do still have about half of the budget left. Though not all invoices came in yet, so this number might be a little lower. As we enter the QA phase, we expect the number of open issues to go up, which is natural.

Melissa Anderson (eliza411) joined us this months to help with the QA phase of the project. Our BDD tests (https://drupal.org/project/doobie) will be an important part of this phase and future Drupal.org maintenance.

This is how we plan to approach QA:

  1. At first our small team with the help of a few BDD volunteers will work through existing BDD tests, updating them to match the application behavior expected from our development site and getting as much of the test suite to pass as possible.

    We’ll approach tests one section of the site at a time. Once developers feel that a particular area is done, we’ll work on the related tests, filing bug reports or updating tests.

  2. Once most of BDD tests are passing, we’ll switch to a short period of manual testing. At this point we’ll look if there are any problems not uncovered previously by BDD testing and improve our test coverage where necessary. Oh, and we’ll fix bugs, of course.

  3. After manual testing, we will open the site for a wider community QA. We’ll provide process details and detailed instructions on where/how to file bug reports. At this point, we will have all the data we need to begin to schedule the deployment.

We are currently at the step 1 of the QA phase. And here are our test stats.

We have 719 documented scenarios for Drupal.org, 677 of which have been automated. 69% of the automated scenarios are passing. We have 80% visibility, which means that roughly 20% of the steps are being skipped because they follow a step failure. As the failures are corrected, either by updating the test or updating the code and configuration, new issues invariably arise.

The numbers will provide the baseline by which we can gauge QA progress. The closer we get to 100% visibility, the closer we are to having the best view possible of the upgrade status as far as BDD testing is concerned, and the closer we are to the second QA phase.

As always, we’d like to say thanks to all volunteers who are working with us and to the Drupal Association Supporting Partners, who made it possible for us to continue the upgrade project. The Supporting Partner Program crowd sources funds that pay for the development team’s time and Drupal.org hosting costs.

Jul 09 2013
tvn
Jul 09

Almost a month ago Drupal.org Working Groups were announced. I bet some of you are curious what we’ve been up to since then! Here is the first update from the Software Working Group.

Scheduling anything for our WG is a challenge, because members are located across 3 continents. But the 4 of us (eliza411, drumm, kim.pepper and I) managed to get together for our initial meeting couple of weeks ago. Angie was greatly missed! We discussed various process and infrastructure questions. Some of the outcomes from our discussions:

The central place for all information about our Working Group is Governance section on Drupal.org: https://drupal.org/governance/drupalorg-working-groups/software

Our monthly meetings will happen on last Tuesday (or Wednesday in some parts of the world) of the month. We’ll publish a blog post summary with a link to meeting minutes after each meeting.

We’ll tag any issues we’re working on with the ‘Drupal.org Software WG’ tag.

For the rest of 2013 we are planning to concentrate on the following 3 projects:

  • Better process/workflow for Drupal.org development
    Where do we collect ideas/feature requests? How do we plan, implement, test and deploy new features?
  • Better maintenance for *.d.o
    Who are all the volunteer teams maintaining different parts of *.d.o? Where are there gaps? What processes and tools do they use?
  • Helping get sub-sites on D7
    What are all the sub-sites? Which need to be upgraded to D7? What is the status of upgrades? How can we help?

We’ll expand more on these projects, what exactly we mean, and how will we tackle them in our future updates.

Due to time constraints we also had to start planning for 2014 right away (in order to align with the Drupal Association’s budgeting process for the next year and ensure we get a little budget of our own). So this week we’re having ‘virtual retreat’ to get rough plan for 2014 together, beginning with projects and priorities that have already been identified, but postponed due to D7 upgrade. We’ll get a sense of what the DA can realistically fund for the next year and continue our planning during a in-person meeting at DrupalCon Prague. We will share our draft 2014 plans as soon as we can!

Jun 27 2013
tvn
Jun 27

It's been a month since DrupalCon Portland and oh what a busy month has it been! Quick notes below.

Drupal 7 upgrade

The upgrade project slowed down due to recent events. There was limited progress for 2 weeks following the security incident as everyone was working on security hardening. We started slowly to get up to speed again last week and the open issues count should be going down again soon.

We are getting closer and closer to the QA phase. Currently we are working on the plan for QA and discussing contract details with a person who will help us with this part of the project.

As always, we’d like to say thanks to all volunteers who are working with us and to the Drupal Association Supporting Partners, who made it possible for us to continue the upgrade project. The Supporting Partner Program crowd sources funds that pay for the development team’s time and Drupal.org hosting costs.

Drupal.org Governance

A couple of weeks ago we announced Drupal.org Working Groups. Groups spent these weeks figuring out tools and processes they will use, having initial meetings for members to meet each other, setting up regular meeting schedule, etc. Once they start having their regular meetings, you should expect meeting minutes published, as well as updates from the Group chairs.

Association.drupal.org

As many of you noticed association.drupal.org was down for some time while we were working on hardening the site following the security incident. The site has been back up for some time now, however we kept our membership system down to upgrade it and improve its security. Today the system is back, you can go to https://association.drupal.org/membership/ and join or renew your Drupal Association membership!

May 09 2013
tvn
May 09

Just like any other community website, Drupal.org has lots of places which could be changed and improved. However upgrading such a website to the next major version of Drupal is already a huge undertaking. So when we originally started the Drupal.org D7 upgrade project our goal was a straight port to Drupal 7. No major regressions, no major new features.

Nearly all sections and pages on D7 Drupal.org will look the same as they do now on Drupal 6. This includes documentation pages, project pages, case studies and marketplace nodes, issue queues, change records etc.

There is however one page which will change - the issue page. Knowing that this is an essential part of the website, where most contributions to Drupal are coordinated and where some people spend many hours each day, we want to explain why we chose to change certain things, show the new UI we are working on, let you try it out on our test site and tell us what is working for you and what isn’t.

Why is the issue page changing?

Redesigning the issue page was not part of our plan. However the Project* suite, and especially the Project Issue Tracking module, suffer from a lot of technical debt. This is some of the oldest contributed code in the entire Drupal project (node/3281 is the url for the Project module).

While porting to Drupal 7, we had to finally pay-off some of this debt:

  1. We rewrote the issue tracker from the ground up using the D7 core Field API and modern best practices.
  2. We tried to remove as much code as possible from Project Issue Tracking and either use existing field modules (core fields, Entity Reference, Field Group, etc) or move our custom code or functionality into general-purpose, reusable field modules (Machine Name, Dereference List, Extended File Field, Node Changes, etc).

At the end it’s safe to say we rewrote close to 90% of the code behind issue pages.

Since it was such a massive rewrite already, we used it as a chance to take a look at how people use the queue and to make changes to the UI. Many of these are ideas that we’ve discussed with Leisa Reichelt through the Prairie Initiative, and that many people on the Drupal.org team have been wanting to address for a long time.

In D6 there are two separate ways to update an issue:

  • to update the issue summary, you have to use the edit tab
  • to update anything else, you have to add a new comment and the comment form includes a bunch of "fields" from the node form.

Not only is this confusing (why are there two separate “modes” for updating an issue?), it means the issue summary is more rarely updated. Not everyone even understands they can do it, and since it’s a different/extra step, it tends to be ignored. Worst of all, it’s actually broken for deep technical reasons (aforementioned technical debt).

How you update an issue in D6 is simply weird. Although it seems natural to many people in the Drupal community that have been using Drupal.org for a while, having to add a comment to try to change something is unnatural and goes against the experience people have with adding comments on basically any other website. Support requests periodically appear in the Project Issue Tracking issue queue from people trying to use this module on their own sites wondering how to update issues once they post them. Newer contributors often find our issue queue strange. This is another Drupalism that many of us have internalized.

Another big problem in the D6 issue page UI is that unless you’ve been participating all along and following an issue closely, it’s often very hard to figure out what patch(es) are current, relevant, worth reviewing or testing, etc. You basically have to read the entire comment history to understand what files are still relevant, and often this means dozens of comments.

So, recreating the exact D6 UI where you have to use the comment form to change certain things about the issue and where most files are attached directly to comments would have taken a huge amount of work and special-case hacks to get working. We thought it was better to just fix the problems and build a new UI using code that will be easier to extend and maintain for the future.

Drupal 7 issue page

There are 2 major changes for the D7 issue page (and a bonus):

1. Single UI for updating an issue

Instead of having two separate modes for updating things about the issue, we're going to move everything into a single mode - updating the issue (by simply editing the node):

  • Key point: if you want to change anything about an issue from now on, you will need to update the issue by editing it. This includes changing the status, changing the summary, attaching additional files, anything.
  • When an issue is updated, the nodechanges module will automatically generate a comment on that issue. It will include a diff of everything that was changed in the new revision. This way comments will still contain the history of all changes to the node.
  • Nodechanges injects a "Reason for your change" field into the node edit form, and if that has a value it's used as the comment body of the auto-generated comment.
  • There are still raw comments, that are *just* discussion about the issue. They are posted via usual comment form without any changes to the issue itself and are just naturally interleaved with the issue-update comments.
  • In addition to the standard node ‘Edit’ link, we added a prominent ‘Update this issue’ button and information when issue was last updated below it.
  • Simultaneous editing of nodes is normally not allowed. So, we wrote and deployed the Conflict module to try to help with this. We’re also considering if it’d be possible to do real-time notification of issue updates.
  • Lastly, commits that reference the issue could trigger an automated comment. This is not in the D7 upgrade MVP, but is in our future plans.

2. The way we handle attachments


Files will always be attached to the issue nodes themselves, never to comments.

  • When you update an issue to upload additional files, the auto-generated comment will reference and display those files, but they wouldn't be directly attached to those comments.
  • There will be a table on the issue page showing the files attached to that issue, including a column linking to the comment that was generated when the file was attached, the user that uploaded the file, etc.
  • Users can choose which files to show in the table (via “Display” checkbox on files), since showing all attached files could be overwhelming on a large issue with lots of patches.
  • Hidden files, without that checkbox set will be located in a separate table, inside a collapsed “N hidden files” fieldset.
  • All changes to the visibility of a file are clearly shown in the comment history.

All of this functionality is either provided by the Extended File Field module (which we wrote to extend the core field file) and/or the Node Changes module.

The basic workflow for attachments is:

  • Upload a patch, defaults to visible.
  • If you re-roll, you update the issue again, attach the new file, and ideally remove the 'Display' checkbox from the now obsolete patch.
  • A comment is auto-generated, showing that you attached the new file, and that the old one is now hidden.
  • When anyone views the issue, they first see the table of relevant files.

Note: Testbot functionality hasn't been ported yet.

3. Bonus: Issue relations


This was not in our MVP however community volunteer (thanks amateescu!) decided to write the code for this new feature while we are busy with our upgrade tasks. So D7 issue pages will have issue relations.

You will be able to specify one parent for any issue. On the issue page we’ll display a parent for this issue and a list of child issues. Additionally there will be a possibility to specify multiple related issues. In the future we are planning to add metadata to this relations, such as ‘druplicate duplicate of’, ‘blocked on’ etc., to make relations even more meaningful.

Your thoughts?

We’ve been working hard building this functionality and think it will be an improvement in many ways. But we are not done yet. Before finishing it completely, we’d like to give you an opportunity to play with this on the test site and tell us what is working for you and what isn't. The D7 upgrade is already late, and we of course don't want to delay it any further, but this piece of the site is too important to rush. So we want to get your feedback while there's still time to make changes if needed to make this work.

Use drupal/drupal to access it and your usual username/password to login.
Note that it is being rebuild every day at 08:00-09:00 UTC, during that time it is inaccessible.

Let us know what you think either here in comments or in Drupal.org site upgrade QA queue. While this is not the full Drupal.org D7 QA yet, we encourage you to open new issues there for any problems with the issue pages you might find.

-Drupal.org team

Apr 15 2013
tvn
Apr 15

Drupal.org D7 upgrade

The D7 upgrade project is doing pretty good! We re-started the project having 122 open issues tagged ‘Drupal.org D7’. We are currently at 84 open issues. Fixed issues count went from 320 up to 372 issues.

Last week Marco Villegas (marvil07) joined our team as a contractor, and we are actively working on a contract with Jeremy Thorson (jthorson). Both of them previously did a lot for the D7 upgrade project as volunteers, we are happy to start working with them in a more sustainable way. We also are talking with couple more people and hope they will join us soon.

Metal Toad has finished their external review of the D7 upgrade issue queue. Their estimates while a bit different, largely confirm our own estimates and projected number of hours. We are more confident moving forward. And we’ll keep their advice in mind – limiting the scope of the project and pushing issues to later launches when possible.

Drupal.org landing pages

A few weeks ago, we blogged about creating persona-specific landing pages to better serve our Drupal.org visitors, and our first landing page will be for site builders.

We are making great progress developing the site builder landing page thanks to several volunteers including Laura Scott, Lisa Rex, Matt Cheney, Miesko Czyzyk, Natalie Roberts, and Martin Coady. With their help, we created a few wireframes that outline the persona’s lifecycle and provide the links to content needed to complete one phase and successfully move onto the next one. It also provides tips and tricks, videos, and links to other helpful content. Here is a sample of one wireframe. Currently we are getting ready to conduct user testing of the wireframes. If you are a beginner or intermediate site builder and want to participate, let us know via comments or send an email to [email protected].

As always, we’d like to say thanks to all volunteers who are working with us and to the Drupal Association Supporting Partners, who made it possible for us to continue the upgrade project. The Supporting Partner Program crowd sources funds that pay for the development team’s time and Drupal.org hosting costs.

Apr 01 2013
tvn
Apr 01

The Drupal.org D7 upgrade project is active again and we are back with our regular week notes! During the past couple of months we mostly kept quiet because drumm and I were busy with various non-Drupal.org related tasks, such as the DrupalCon websites, association.drupal.org, etc. For the coming weeks the Drupal.org D7 upgrade is our top priority.

Currently we are working in three: Neil Drumm, Derek Wright and I. Jeremy Thorson and Marco Villegas helped us with some of the Project* suite issues and Lewis Nyman is helping with the Bluecheese theme.

On the week of March 18 we were cleaning up queues and bringing project back on track. Last week the actual work in the issue queues started. Most importantly the table of attachments on the issue pages is now done! Some more issues for the Project* suite were fixed as well. And while working on those Derek got his 1000th commit pushed to the Project module!

For the Bluecheese theme we improved changed completely the Sass partials structure to make the code more logically organized and easier to find and work with.

The Solr port is mostly finished, we are working on the last bits of the solr related parts of the drupalorg.module, such as /download page, which is looking good already.

We’d like to say thanks to volunteers who are working with us and to the Drupal Association Supporting Partners, who made it possible for us to continue the upgrade project. The Supporting Partner Program crowd sources funds that pay for the development team’s time and Drupal.org hosting costs.

Mar 18 2013
tvn
Mar 18

In late April 2012 we kicked off a sprint to upgrade Drupal.org to Drupal 7. Over the next months a huge amount of work was done, resulting in about 300 closed issues (not counting bdd team issues). The Project module suite and many other parts of our site were erased of nearly ten years of technical debt brought on by numerous patchwork fixes over the years. In addition, many parts of the new Drupal.org were rebuilt to take advantage of the features and functionality of Drupal 7. A lot of work went into something completely new for Drupal.org - Behaviour-Driven Development (BDD) testing infrastructure and suite of tests for the most important parts of Drupal.org, including Git & Project functionality, Solr search, and various sections like Marketplace and Case studies. All told, by the end of 2012 the upgrade was close to completion. But, at this point we exhausted our initial budget. The decision was made to suspend all activities until we could reassess the situation, define a solid path forward, and get budget for the finishing part of upgrade project approved.

So what has happened?

In the weeks following that decision we took a step back and reflected on what caused us to miss our deadlines and exhaust our budget. We had the usual discoveries of decisions that could have been made better and identified areas where we could have used more help. In the end we narrowed it down to the following issues:

  • Not adjusting project plan
    We began the project with the understanding that this would be a partnership with other major users of the Project* suite, with the DA shouldering about 25% of the costs and development responsibility. When that partnership model fell through, we failed to properly re-define and re-scope the project. Rather, we attempted to complete all of the scoped work, without adequately adjusting the expectations for time or budget.
  • Too many moving pieces
    All the different teams started working on their parts of the project simultaneously, which caused some teams being blocked on others for long periods of time and volunteers getting demotivated. Looking back, better decision would have been to engage the teams at different times, finishing parts of the upgrade one by one. Additionally, we engaged with many developers for very specific and short-term deliverables, with varying rates of pay and timelines.
  • Inaccurate estimates
    Estimating issues is an art and a science. We failed in the art of initial estimating, and at the science of refining those estimates as more work and data became available. An estimate, by definition, is rarely accurate, but we can do better.
  • Inefficient communications
    There were troubles in communication between many people working on different parts of the project, which resulted in people often not being on the same page and the board and community receiving unrealistic information as a result.
  • Unclear decision-making authority
    Decisions made by one development team sometimes caused lengthy discussions with another group of developers, which caused delays in the project. As a community built website we have many different stakeholders and owners. The lines of who can make decisions on what sections of the website were not as clear as originally thought when the project began. With a very tight budget, and an already out-of-scope project, every discussion chipped away at our tenuous ability to deliver the upgrade.
  • Ongoing deployments on Drupal 6 site
    During most of the upgrade project we were also developing and deploying new features on the live Drupal 6 site, including updates to Marketplace, new Case studies and Books sections. These deployments required DA and community time that was not then available for the upgrade.

And what are we going to do?

Having all of the above in mind we are re-starting the upgrade project this week. What will be different this time?

  • Properly Defining and Scoping the Project
    This time, we will recognize our finite resources at the outset, and plan to only address the most critical issues to deliver a minimally viable product. We recognize that this means we won’t produce a 100% feature complete upgrade, but it will be sufficient to meet our needs. Although the work is not yet complete, we are working with outside partners to review the issue queue to produce a smaller, more manageable list of critical issues to address.
  • A smaller team
    To complete the upgrade, we will focus on a small core team, bringing volunteers in primarily for the QA phase only.
  • Outside review
    We have engaged Metal Toad, Drupal web development shop, to review the issue queue and provide a second set of opinions about the estimates. This will help us reconsider issue estimates from the outset and hopefully provide a more accurate starting point. Those estimates will be available by March 20 and incorporated into the project after that.
  • Better communication
    To some degree, the smaller project team and internal project management will help address communications issues between team members. We will also have the entire team on twice-weekly “scrums,” rather than meeting in smaller teams. I will be attending all meetings and sharing progress with the community in regular update posts. We will encourage project team members to share real and honest feedback as the project progresses and we’ll act on that feedback.
  • Giving Neil Drumm decision-making authority
    We would like to begin phase 2 of the project prior to the completion of governance. And, even if we waited for governance to be complete, testing the newly formed governance structure with the emotionally charged issues related to the D7 upgrade would likely kill governance before it begins. For the purpose of finishing this project, we are asking community to support Neil Drumm in making all technical decisions required to complete the project (including Project* related decisions). Neil has commit access to the Project module, significantly increasing our ability to move forward quickly.
  • Minimizing updates during upgrade project
    We won’t develop and deploy any new features or changes to existing features on live Drupal 6 site, except for a small number of absolutely needed ones, such as new revenue-generating pages.

Next steps

Last Wednesday at the Board meeting our plan for the part 2 of the upgrade project was approved, so we are now slowly getting up-to-speed again. We should get results of the issue queue review by MetalToad on Wednesday and finalize our MVP - list of issues to work on - by the end of the week. At the same time Neil Drumm and I will start working on some of the issues. The first person to join us as a contractor will be Derek Wright, who will, of course, work on the Project* module suite. We expect couple more people to join the team in the following weeks.

Our next update will be published in a form of our good old Drupal.org team week notes. So make sure to watch my blog here or join the Drupal.org group.

Thank you

Before we step into the second part of the Drupal 7 upgrade project, we’d like to say a big thanks to those people who already spent a lot of time on the project and helped us during the first phase of the upgrade. We’ve listed them all on a separate page. Thank you!

Feb 28 2013
tvn
Feb 28

Start: 

2013-03-04 17:30 - 18:00 Europe/London

Organizers: 

Have you ever visited Drupal.org Case studies, or Marketplace, or Drupal Planet? All of them are moderated by a small team of volunteers. And they really need your help! No coding skills are required to contribute to Drupal.org in this way.

Each Monday content team meets in the #drupalorg IRC channel. Join the meeting and help us improve content on Drupal.org!

Dec 21 2012
tvn
Dec 21

In November 2012 we created the new Drupal.org content project - an area specifically for the content issues. The project covers marketplace listing, case study promotion, and front page promotion requests; issues related to Drupal Planet and other non-documentation content on Drupal.org, such as forums, landing pages etc. Most of them were previously in the Webmasters queue and were moved to the new Content project.

The idea of a separate project for Drupal.org content management and moderation has been around for some time already, and recent discussions brought it to the light again.
There were several goals for this move:

  • we needed a dedicated and organized place for content-related issues, where each moderated section has own component;
  • separate queue would be much smaller than Webmasters queue, making it less overwhelming for people to follow it and stay up-to-date;
  • we wanted to make it easier and less intimidating for new people to start contributing.

To meet the goals, particularly the last one, we also created a set of guides - “Contribute to Content Moderation”. They list all the parts of the queue, team leads and steps to get involved and contribute.

Drupal.org content team: who we are

There is already a number of people active in the Drupal.org content issue queue. Using this opportunity I’d like to thank all of them for all the work they do to help maintain various parts of Drupal.org:

Danita Bowman, DSquaredB
Terry Hughes, Tezza
Lisa Rex, lisarex
David Hochhausen, dddave
David, silverwing
Ivo Van Geertruyen, mr.baileys
Greg Knaddison, greggles
Steve Purkiss, stevepurkiss
Alex Laughnan, laughnan
Alberto Paderno, kiamlaluno
Codi Lechasseur, codi

Thank you!

Dedicated IRC channel

To make both Content and Webmasters issue queues more effective and help onboard new people we’ve established a dedicated IRC channel: #drupalorg. Everyone is welcome to join (especially webmasters and content team members)!

Get involved

We still have more things to sort out for this new project and documentation is still in progress, now is the perfect time to jump in and help us.

Particularly help is needed with our latest addition to the content project - a queue for the new Books section of the Marketplace. The guidelines are in place and we need volunteers to review and add new books to the listing.

If you would like to do that or help with another part of the queue - join #drupalorg and we’ll help you get started!

P.S. On a related note, if you are one of the people who contribute to Drupal.org - you can now show it on your profile page.

Nov 16 2012
tvn
Nov 16

Past month(s) we all have been very busy working on Drupal 7 upgrade of our website. Below are latest updates on the project status from Joel ‘Senpai’ Farris.

Drupal 7 upgrade

Sprints

We had several sprints: at the end of October in Seattle, WA for the Pacific Northwest Drupal Summit, and in Portland, OR at OpenSourcery’s offices (http://groups.drupal.org/node/262558), then again in early November each day at the BADcamp. There was a little bit of work accomplished at the PNWDS sprint, some good stuff relating to ApacheSolr and the project_solr module happened at the Portland sprint, and a lot of great work surrounding the last six remaining contrib modules happened at the BADcamp sprint. The Diff module has reached a stable release, as well as Machine Name. Entity Reference and Entity API are 98% release-able as well, Field Collection has no criticals left, and Materialized Views is no longer needed for drupal.org so we took it off the list.

Current status

As of now we are hoping to finish D7 upgrade sometime in the early new year. According to most recent re-estimates of the modules and related tie-ins we have several hundreds hours of work still remaining on the Project* suite, which is blocking Git and other teams.

We also discovered that for performance reasons we need to conclude the Search API discussion. It is lively and challenging, and will cost us an unexpected ~80 hours of additional development, but currently seems to be the only way to have searchable issue queues in D7.

Demo site

At the BADcamp we invited everyone to take a look at our progress on the Drupal 7 demo site. Right now that site is temporarily down due to some problematic SQL queries that are threatening to take down drupal.org itself. We are going to restore it once we’ve identified and fixed the performance of all those queries.

Other initiatives

While D6 Drupal.org is currently on “feature and code freeze”, we managed to do one last deployment and finish long lasting community initiative. Thanks to LoMo and jhodgdon we now have new Books section within the Marketplace and new book listing content type. Previously we had a set of documentation pages with information about Drupal books. This new dynamic listing will be filterable, making it easier to find the book you need. Right now new section is empty, we expect it to be filled with content in the next few days.

Thanks

We would like to say thanks to Drupal Association Volunteers and Supporting Partners, who have made Drupal.org improvements possible. The Supporting Partner Program crowd sources funds that pay for the development team’s time and Drupal.Org hosting costs. And volunteers are a key part of our team. They donate huge amounts of time and talent to help us make Drupal.org better.

Oct 02 2012
tvn
Oct 02

We all have been waiting for this.. Drupal.org upgrade project is entering QA stage! Over 190 people already signed up to be A/B testers. Announcement has been updated with further instructions, make sure to check it.

In other upgrade news - the Bluecheese team got more volunteers now with new people joining last week and team's issue queue is active again!

The Project team had a lot of progress in the past 2 weeks and fixed issues like:

Drupal 6

Last 2 weeks we’ve been busy with some final clean-ups to the Marketplace. Read more about Marketplace updates and changes in the news post. Hopefully we have 1 more deployment this or next week and after that Drupal.org D6 development will be frozen.

Webmasters

We are working on improvements to Webmaster’s queue and processes, to make them more organized and efficient. This will be multi-step discussion and first part already started. If you are Drupal.org webmaster, please take time to read and give your feedback. Or better - edit associated issue’s summary and add your name to one of the teams.

Thanks

We would like to say thanks to Drupal Association Volunteers and Supporting Partners, who have made Drupal.org improvements possible. The Supporting Partner Program crowd sources funds that pay for the development team’s time and Drupal.Org hosting costs. And volunteers are a key part of our team. They donate huge amounts of time and talent to help us make Drupal.org better.

Sep 25 2012
tvn
Sep 25

July this year first ever Drupal Governance Sprint took place. Drupal.org was among the topics discussed. There were some ideas on how to ensure that site maintenance goes smooth, policies are in place, user’s issues are being answered and experience in the queue is consistent for the users. While implementing those ideas should have positive effect on Drupal.org maintenance, that process will take quite some time. And we don’t necessarily need to wait. We can start discussing existing problems and self-organizing right now. This will also make governance project go much easier and faster once it moves to Drupal.org.

I want to start discussion on Drupal.org governance with one huge and important part of the Drupal.org which is webmasters queue.

Current situation

  • For the past months the webmasters queue persistently has had over 1000 issues open at any given time.
  • While our webmasters do a lot, answer lot’s of issues, overall this queue has rather inconsistent nature (not an active structured team effort).
  • There are currently 158 people with “site maintainer” and “administrator” permissions, but most of them are really busy with other projects, they maintain contrib modules, contribute to core etc.
  • We don’t have a clear process of how one can become part of the team, apart from rather vague “start doing something and maybe someone will notice and promote you”.
  • People who do have permissions to do things often are not sure if they are supposed to do something in specific case and what exactly they need to do.
  • Webmasters are not really a team now, but a number of people with permissions. There are no clear “leader” or roles, responsible for specific tasks.
  • There is no clear way of how to reach this people: no dedicated irc channel, no g.d.o group, issue queue is too huge, mailing list is not really used much.

The site is growing, new and new sections being added, which require maintenance. The current situation is not sustainable anymore.

We have examples of Security and Infrastructure teams with clear structure, roles, ways you can reach them. While not saying they are ideal and not copying them we can at least do small steps in order to bring more organization to Webmasters team.

First and obvious step would be to define “owners” of different parts of webmasters issue queue. Some parts have 1-2 active people, constantly working on them. Others - don’t. Documenting this will help us see who is doing what and where we have gaps.

In order to do this I’ve opened an issue. The summary lists various parts of webmasters queue and people responsible for them. If you are a webmaster and want to commit to specific part of the issue queue - please add your name.

I suggest a 2 week timeframe for all interested individuals to comment on that issue and choose which part of the queue they want to commit to. In 2 weeks we’ll see the team we have currently and where we have gaps.

Documenting current contributors and their specific roles will be the first step. Once this is done, we can move further: building teams and improving processes around those people to ensure webmasters queue is active and efficient.

-----
Thanks to greggles and silverwing for feedback and help with the post.

Pages

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web