Upgrade Your Drupal Skills

We trained 1,000+ Drupal Developers over the last decade.

See Advanced Courses NAH, I know Enough
Jan 29 2021
Jan 29

January 29, 2021

What is hook_update_N()?

Let’s say you are developing a Drupal module (custom or contrib) which tracks how many visitors landed on specific node pages, version 1 of your code might track visitors by nid (node id) in the database using a table like this:

nid visitors 1 4 13 22

Let’s set aside the debate over whether the above is a good idea or not, but once your code has been deployed live to production sites, that’s what the data will look like.

This module might work very well for a long time, and then you might have the need to track not only nodes but also, say, taxonomy term pages. You might reengineer to look like this:

type id visitors node 1 4 node 13 22 term 4 16

To achieve this change when the first version of your database is already out in the wild, you need to tell target environments to update the database schema. This is done using hook_update_N(), and you would replace the N() by incremental numbers, something like this:

/**
 * Update database schema to allow for terms, not only nodes.
 */
function hook_update_9001() {
  ...
}

If this case 9 is the major version (Drupal 9) and 001 because this is the first update to your code.

Each module tracks which version it’s using, so that if code introduces new hook_update_N() functions, it will know to run them only once. You can tell which schema version any installed module is using by running, for example:

drush ev "print(drupal_get_installed_schema_version('webform'))"

This might tell you, for example, that Webform’s current schema is 8621. This means that the latest update hook that was run is Webform’s hook_update_8621(). If the codebase introduces hook_update_8622(), say, or hook_update_8640() (you can skip numbers if need to), then the database will be marked as out of date and running drush updb will run the the new hook and update the installed schema version.

If you ever need to re-run an update hook (which happens rather rarely), you can update the schema, like this:

drush ev "drupal_set_installed_schema_version('webform', 8620)"

So what’s wrong with this?

This works well almost all the time, and you can automate your deployment process to update the database, making sure your schemas are always in sync. However as developers and site users it is important to be aware of certain drawbacks of hook_update_N(), which I’ll get to in detail:

  • hook_update_N() tightly couples the database to the code version;
  • it makes gradual-deployment on multi-container setups such as Kubernetes fragile (or impossible);
  • rollbacks are not possible;
  • it can add considerable compexity to deployment of configuration.

The shaky foundation of database-driven websites

The idea of version control is paramount to how we conceive of computer code. If you’re following the precepts of continuous deployment, then every version of your code needs to “work” (that is, tests need to pass, or, at the very least, it needs to be installable).

For example, let’s assume a bug makes it to your production envrionment for version 5 of your code, and you know this bug was not present on version 4 of your code, you should theoretically be able to check out version 4 and confirm it was working, then figure out what the difference it between version 4 and 5.

In fact this is exactly how things work on static sites such as Jekyll: all your data and your functionality (Javascript) are in your codebase. Each version of your code will be internally coherent, and not rely on an external unversioned database to do something useful.

On database-driven projects based on Drupal or Wordpress, if you check out version 4 of your codebase, it will probably not do anything useful without a database dump which was created using version 4 of your code.

Therefore, although we all use version control for our code, in a way we are fooling ourselves, because critical parts of our project are not version-controlled: the database dump, the ./sites/default/files folder, and the private files folder.

Although it makes sense for certain elements to be a database or on ./sites/default/files, for example, an encrypted user account password or a user’s avatar; for other elements such as your “About page” text, it would really make a lot more sense for this to be under version control.

In fact, the blog post you are reading right now is a file under version control on Jekyll, which you can see using this link, and not some collection of opaque, unversioned, entries in database tables with names like node__body, node__field_tags, node_field_revision, which can be changed at a moment’s notice by any module’s hook_update_N() functions.

Oh, did I mention that I love Drupal?

Tight code-database coupling

Let’s imagine a world where the database schema never changed. A world where hook_update_N() does not even exist.

In such a world, you could take any version of your code, and any version of your database dump (say, the latest version), combine the two on a test environment, and debug errors at will.

In the real world, every time any module updates the database schema, it makes the database more tightly coupled to the current version of the codebase.

Let’s take our “number of visitors per entity” code we had earlier: if I use an old codebase which expects my table to contain fields “nid” and “visitors”, but my only available database dump has fields “type” “id”, “visitors”, the history of my carefully version-controlled codebase will be useless, and old versions will fail with an error such as:

ERROR 1054 (42S22): Unknown column 'id' in 'field list'.

Gradual deployments

Mostly we think of Drupal sites as being on a server with one copy of the codebase, and one copy of the database. So the concept of keeping the database and code “in sync” makes sense.

But as more and more teams use containers and Kubernetes-type container-orchestration systems, high-traffic sites might have, say, one performance-optimized database, and then 5, 10 or 20 load-balanced copies of your PHP code.

Acquia uses such a setup behind the scenes for its cloud hosting, so it’s good to develop with this in mind. On Acquia’s setup, all PHP container use a single, shared database, as well as shared private and public files directories.

But the PHP containers do not share the /tmp directory. This means that every time you perform a web request on a server, the load balancer might direct you to a container with its own /tmp, whose contents differ from other containers’ /tmp.

It’s important to realize this if your code stuff such as building large files over several web requests, and can lead to hard-to-diagnose bugs such as:

But, in addition to providing you with headaches such as the above issues, multiple containers can also allow you to do gradual deployments of new code, reducing the cost of potential failure.

For example, let’s say you have 20 Drupal containers with 20 copies of your codebase, and each Drupal container is connected to a shared database, and shared files and private files directories. If you are deploying a risky update to your code, you might want to start by deploying it to 25% of the containers (5). Then if there are no adverse effects, scale up to 10 the next day, then the entire 20 the day after.

Code that uses hook_update_N() can break this workflow: because all containers share the database, if container 1 has the new version of your code and updates the database accordingly (so that the new database fields are “type” “id”, “visitors”); then container 10 (which uses the old version of your code) will fail when it looks up the database field “nid”.

Rollbacks

Let’s forget about fancy container orchestration and just look at a typical Drupal website. A simple real-world site might have a “contact us” webform and some pages, plus some custom functionality.

Let’s say you are deploying a change to your codebase which triggers a hook_update_N(). No matter the amount of unit tests and testing on stage, there is always the possibility that a deployment to production might trigger unforseen issues. Let’s assume this is the case here.

A typical deployment-to-production scenario would be:

  • You backup your production database.
  • You install your new code.
  • You run drush updb which updates the database schema based on your hook_update_N().
  • A few hours pass. Several people fill in your contact form, which means now your database backup from step 1 is out of date.
  • You realize your newly-deployed code breaks something which was not caught by your stage testing or your automated tests.

In a situation like this, if you did not have hook_update_N()s in your code, you could simply roll back your codebase on production to the previous version.

However, this is no longer an option because your database will not work with previous versions of your codebase: there is no hook_downgrade_N(). You are now forced to live with the latest version of your code, and all the benefits of version-controlling your code are for naught.

Config management

Let us recall the elements which make up a Drupal website:

  • Versioned code.
  • Unversioned database and file directories.

If you are using configuration management and a dev-stage-production workflow, there is a third category:

  • Configuration, including the list enabled modules, defined node types and fields, which exist both in the database and in unversioned code.

It is worth recalling a typical workflow:

  • add field_new_field to the article node type on your local machine.
  • the field is now in your local development database but not in your codebase
  • drush config-export
  • the field is now in your local development database and also in your codebase
  • do all your testing and push your code to production

At this point your field is in your production codebase but not your production database.

You probably have a deployment script which includes a “drush updb” step. The question is: do you run “drush config-import” before or after “drush updb”?

It turns out this is not that easy a question to answer. (Drush also provides a drush deploy command which combines configuration import and database updates.)

Regardless of your deployment process, however, we need to take into account a more troubling possibility:

In addition to relatively benign database schema updates, hook_update_N() can modify configuration as well.

In such a case, if you are not careful to run hook_update_N() first on your development environment, then export the resulting configuration, then run your deployment, you may run into the following problem:

#3110362 If an update hook modifies configuration, then old configuration is imported, the changes made by the update hook are forever lost.

Let’s look at a real-world example using the Webform module. Let’s install a new Drupal 8 site with Webform 5.23, then export our configuration, then upgrade to Webform 6.x and import our old configuration. We’ll look at the kind of headache this can lead to (note to beginners: do not do this on a production site, it will completely erase your database).

composer require drupal/webform:5.23
drush site-install -y
drush en webform_ui -y
drush config-export -y

This puts your current site configuration into code. Among said configuration, let’s focus on a single piece of configuration from Webform:

drush config:get webform.settings settings.default_page_base_path
# 'webform.settings:settings.default_page_base_path': form

The base path for webforms is “form”. This tells Webform to build URLs with a structure such as https://example.com/form/whatever.

Let’s now update webform, and our database.

composer require drupal/webform:6
drush updb -y
drush config-import -y

In Webform’s webform_update_8602(), the config item webform.settings:settings.default_page_base_path is changed from “form” to “/form”.

But we are re-importing old config, which overwrites this change and reverts webform.settings:settings.default_page_base_path to “form”, not “/form”

To see the type of hard-to-diagnose error to which this might lead, you can now log into your Drupal site, visit /admin/structure/webform, create a webform named “test”, and click on the “View” tab.

Because the base path lack the expected leading prefix, you now get the “not found” URL /admin/structure/webform/manage/form/test, instead of the expected /form/test – a critical bug if you are on a production site.

In addition, this has a number of cascading effects including the creation of badly-formatted URL aliases which you can see at /admin/config/search/path.

If you find yourself in this situation on production, you need to revert your Webform schema version on your development environment, export your config, reimport it on production, and resave your forms, and potentially fix all your paths starting with “form” on /admin/config/search/path so that they start with “/form”.

To be fair, this is not the fault of the Webform maintainers. In my opinion it shows a fundamental frailty in hook_update_N() combined with lack of documentation on deployment best practices. However, if we strive for Drupal to be a robust framework, there should not be a single point of failure (in this case not strictly adhering to fickle, badly-documented deployment procedures) which can lead to major instability on production.

How do we fix hook_update_N()?

Here are a few approaches to avoid the potential damage done by hook_update_N():

Approach 1: don’t use hook_update_N()

When possible, you might consider not using hook_update_N() at all. Consider our “number of visitors per node” module from earlier.

Instead of a hook_udate_N(), your code could do something like this:

  • Do not change the field name from “nid” to “id”. Even though “id” makes more sense, the field is called “nid”, just leave it at that.
  • Do not expect there to be a “type” field. If your code needs it, for example if creating an entry for the first visitor to a non-node entity, your code can create it.
  • Assume an empty “type” means you are dealing with a node.

The above approach adds complexity to your code, which you can add to a “storage” abstraction class. Although not ideal, this does away with the need to use hook_update_N().

Approach 2: Don’t use hook_update_N() to update configuration

Updating configuration, as seen previously, is even more dangerous than updating non-configuration database tables. So if at all possible, avoid it.

In the Webform example given above, it might have been reasonable to consider keeping with the old non-leading-slash format for path prefixes, rather than update configuration.

When you absolutely must update configuration, you could consider the possibility that certain users might have reimported old configuration, and provide error-checking and hook_requirements() (displaying error messages on the /admin/reports/status page) accordingly.

Approach 3: Robust exception handling

Do not assume that your database schema, or your configuration structure, is up-to-date. If you decide to provide a hook_update_N() to update the schema from, for example, “nid” and “visitors” to “type”, “id”, “visitors”, when querying the database, you might want to consider the possibility that for whatever reason the database is not up-to-date. Here is some pseudo-code:

public function num_visitors_for_entity($id, $type = 'node') : int {
  try {
    return $this->query_database($type, $id);
  }
  catch (\Exception $e) {
    $this->logAndDisplayException($e);
    return 0;
  }
}

That way, if your database and code are not in sync, it’s not going to break your entire site, but rather log an exception and fail gracefully.

Approach 4: keep config changing logic idempotent and separate from update hooks

Let’s look again at Webform’s webform_update_8602(), the config item webform.settings:settings.default_page_base_path is changed from “form” to “/form”.

I would recommend having a separate function to update config, and call that function from the update hook. That way, if a development team makes the mistake of not updating their configuration before importing it into production, it will become easier to run, say “my_module_update_configuration()”.

Then, your hook_requirements() might perform some sanity checks to make sure your configuration is as expected (in this example, that the “webform.settings:settings.default_page_base_path” config item has a leading slash). If this smoke test fails, developers can be directed to run my_module_update_configuration() which will update all configuration to the required state.

In addition, my_module_update_configuration() can be made idempotent, meaning: no matter how often you run it, you will always end up with the desired state, and never get an error.

Resources

Please enable JavaScript to view the comments powered by Disqus.
Jan 20 2021
Jan 20

January 20, 2021

This post is aimed at web development teams and is not tied to a specific technology. We will aim to not get more technical than is needed, but rather to explore what Continuous integration (CI) is, and how it can help save teams money within a month of it being set up.

What is continuous integration?

Although several definitions of CI have been proposed, we will use the following definition in the context of this post:

Cotinuous integration (CI) is the practice of running any number of tests, automatically, on a project, periodically and/or whenever the code changes. For CI practitioners, the number one priority is for tests to always be passing.

A simple example, please

Here is the very simplest example I can think of:

Let’s say you’re maintaining an old-school HTML website (no fancy stuff like databases or PHP), your team may decide to set up CI to make sure a file called “index.html” exists in your codebase: if it exists, your test passes; if it is absent, your test fails.

Checks may be run every time your code is changed.

Your team might store code on GitHub, and link a cloud CI provider such as CircleCI to your codebase, having it trigger every time your code changes.

You will then define a script which is your definition of “what it means for your your codebase to pass”: checking for the existence of “index.html” is a one-line script.

A more complex example

Although the example above has value, it is very simple, and you may soon find yourself wanting to add higher-value tests to your script. This ability to add complexity over time is a powerful feature of CI: getting started is simple, and you can add as many tests as you want over time depending on your available resources.

Let’s say your team is maintaining a Drupal or Wordpress codebase with lots of complex code, your team may set up a CI server that:

  • checks for broken links on the live environment every so often;
  • checks every few minutes that the live environment is responding and has some expected keywords on its front page;
  • every so often, checks that the live environment adheres to certain Accessibility standards;
  • every so often, checks that the live environment is not reporting any errors;
  • on every code change, perform some static analysis on custom PHP code: for example, that a function which expects an array as an argument is never called with a string.
  • on every code change, make sure PHP code adheres to coding standards (for example, functions should have comments; and indenting should be correct).
  • on every code change, create a dummy Drupal or Wordpress site with a dummy database and make sure your site fires up, and run some end-to-end tests against it.
  • etc., etc.

A cloud-based tool such as CircleCI can work well to check the codebase when it is changed; and a hosted tool such as Jenkins might be a good fit for running periodic checks (such as a sanity check making sure the production environment works).

The above example corresponds to real-world checks I perform on lost of projects I maintain; and both CircleCI and Jenkins are tools I have been using for years.

So how much does all this cost?

“How much does this cost?” is actually the wrong question. “How much can I save?” is a better way of putting it. Consider the following graph, the horizontal axis is time, and the vertical axis is cumulative project cost.

  • The red line is business as usual: because we are not maintaining CI scripts or setting up tests, the up-front cost is low. But eventually you’ll lose control of your codebase and spend all your time putting out fires (I call this the “technical debt wall”).
  • The blue line is the CI approach, higher up-front cost to set things up, but eventually you’ll get less errors.
  • Where the two lines intersect, I call the “sweet spot”. That’s when you start saving money. Your “sweet spot” is not months or years away: I firmly believe it should happen within a month. If it takes longer than a month, you’re overengineering your CI system.

So what are these up-front costs?

The up-front costs are:

  • Creating a simple script which defines what it means for your code “to work”. If you find this intimidating, just have your script check for a file that must be present, as in the simple example presented earlier.
  • Make sure your code is tracked in GitHub or BitBucket.
  • Make sure your entire team accepts the principle that making tests pass is the number one priority. This is crucial. If you start accepting failing tests, then CI becomes a useless burden. This also means every member of your team must agree with every test that is performed. If a test is not important enough to warrant dropping everything when it fails, then you should not have that test in your codebase.
  • Integrate a simple, free CI cloud provider like CircleCI and make sure it works.

All of the above, together, can take between an hour and a day.

How about the ongoing costs?

Ongoing costs are closely relate to the complexity of your CI setup. If you are just testing for an “index.html” file, your ongoing costs are close to zero, but may include:

  • dealing with errors and updates in the CI script itself. Don’t forget the CI script is computer code, and like any computer code, it needs to be maintained.
  • updating the CI script to deal with API changes in the cloud CI provider.
  • fixing false negatives. For example, someone may change the filename from index.html to index.htm, which might require you to fix your test script to also test for index.htm in addition to index.html.
  • onboarding new team members to understand the importance of making sure tests always are passing.

If your tests are super simple (such as checking that an “index.html” file exists), the above costs are low, probably less than one hour a month.

If your tests are complex (as in our second example, above), you might set aside 5 to 10 hours a month for ongoing costs.

Obviously, if your ongoing costs are higher than your savings, then you are “over-testing”.

So what are the benefits?

The fundamental trick of CI is to keep your benefits higher than your costs. Let’s go back to our simple “index.html” example:

  • We have already established that there are minimal up-front and ongoing costs.
  • There are also ongoing savings: once you know that your index.html file is guaranteed to exist, your manual testing time decreases.
  • The cost in lost revenue, lost confidence, and debugging time in case someone accidentally deletes index.html from your website would be considerable high.

Based on the above, you can conclude whether it’s worth implementing CI.

Continuous improvement of your CI setup

Checking for “index.html” is probably of very low value, but once you’ve done that, you’ve also set up the foundation to improve your script. Every time you feel your CI script has a positive cost-benefit ratio, it is time to improve your CI script. In practice, I have found that in projects under active development, the CI setup gets constantly improved.

Specifically, any time a problem makes its way to production, it should be a gut reaction to introduce a fix, along with a test to make sure the problem never happens again.

The key is making incremental improvements, making sure your cost-benefit ratio is always positive.

Docker and containerization

Docker, and containerization generally, embed software and configuration in computer code along with your project code.

The widespread adoption of Docker and containerization in recent years has been crucial for CI. Without containerization, let’s say you want to run PHP static analysis, start a database with a Drupal site, run end-to-end tests, you need to install a bunch of software on your CI server (or your laptop), make sure the versions and configuration are in sync with your local development setups. This is simply too expensive.

Docker makes all this easy: simply put, Docker abstracts all the software and configuration, making software act the same on any computer that has Docker installed.

If you are not using Docker and you’d like to see how simple this makes things, install and launch Docker Desktop on your computer, give it 6Gb RAM instead of the default 2Gb in its preferences, then you’ll be able to run all tests on my Drupal Starterkit project, without any additional fiddling with configuration of software:

cd ~/Desktop && git clone https://github.com/dcycle/starterkit-drupal8site.git
cd starterkit-drupal8site
./scripts/ci.sh

It should take about 10 minutes to run all tests and it will not add any software to your computer; everything is done on throwaway “containers”. (In general, tests become a lot more frustrating to developers as they take longer to run; which is why I have a policy of not accepting tests which take more than 20 minutes to run.)

The amount of software packages and configuration required to run all the tests in this example is enormous: database servers and configuration, passwords, permissions, PHPUnit, the right version of PHP and Apache or Nginx…; however it’s all defined in Docker files and in code, not on host computers.

Which is why you can run the tests in three lines of code!

This makes it possible to run these complex tests on your computer without installing any software other than Docker.

This also makes it possible to run these exact tests, sans extra configuration, on CircleCI or other CI providers which support virtual machines with Docker preinstalled. In fact, that’s exactly what we’re doing with the Drupal Starterkit. CircleCI even provides a cute badge to indicate whether tests are passing.

Click on the badge below to see test results on CircleCI, which should be identical to the results on your computer if you ran the the above script (you’ll need to log in with your GitHub or BitBucket account).

CircleCI

Security

Whether you are using a cloud service such as CircleCI, or hosting your own CI server with Jenkins or other software, be aware that it adds a potential attack vector for hackers, especially because by design, CI software needs access to your codebase.

In early 2021, a vulnerability was discovered in JetBrains TeamCity (Widely Used Software Company May Be Entry Point for Huge U.S. Hacking, New York Times, January 6th, 2021) in relation to the major SolarWinds hack.

Make sure you have a solid security policy, including the Principle of Least Privilege (POLP) and other industry-standard security approaches; also make sure your codebase, even if it’s private, does not contain any sensitive data, including API keys.

Conclusion

With continuous integration (CI), you can let computers do the grunt work of looking for bugs in your codebase, liberating your developers to do more productive work, reducing the number of bugs that make it into production, and increasing the level of confidence of all stakeholders in your software, and deploying frequently.

And, above all, saving money.

CI can be as simple or as complex as you need: start small, then let your CI process grow as your team becomes more comfortable with it.

Please enable JavaScript to view the comments powered by Disqus.
Oct 03 2017
Oct 03

October 03, 2017

This article is about serving your Drupal Docker container, and/or any other container, via https with a valid Let’s encrypt SSL certificate.

Edit: if you’re having trouble with Docker-Compose, read this follow-up post.

Step one: make sure you have a public VM

To follow along, create a new virtual machine (VM) with Docker, for example using the “Docker” distribution in the “One-click apps” section of Digital Ocean.

This will not work on localhost, because in order to use Let’s Encrypt, you need to demonstrate ownership over your domain(s) to the outside world.

In this tutorial we will serve two different sites, one simple HTML site and one Drupal site, each using standard ports, on the same Docker host, using a reverse proxy, a container which sits in front of your other containers and directs traffic.

Step two: Set up two domains or subdomains you own and point them to your server

Start by making sure you have two domains which point to your server, in this example we’ll use:

  • test-one.example.com will be a simple HTML site.
  • test-two.example.com will be a Drupal site.

Step three: create your sites

We do not want to map our containers’ ports directly to our host ports using -p 80:80 -p 443:443 because we will have more than one app using the same port (the secure 443). Port mapping will be the responsibility of the reverse proxy (more on that later). Replace example.com with your own domain:

DOMAIN=example.com
docker run -d \
  -e "VIRTUAL_HOST=test-one.$DOMAIN" \
  -e "LETSENCRYPT_HOST=test-one.$DOMAIN" \
  -e "[email protected]$DOMAIN" \
  --expose 80 --name test-one \
  httpd
docker run -d \
  -e "VIRTUAL_HOST=test-two.$DOMAIN" \
  -e "LETSENCRYPT_HOST=test-two.$DOMAIN" \
  -e "[email protected]$DOMAIN" \
  --expose 80 --name test-two \
  drupal

Now you have two running sites, but they’re not yet accessible to the outside world.

Step three: a reverse proxy and Let’s encrypt

The term “proxy” means something which represents something else. In our case we want to have a webserver container which represents our Drupal and html containers. The Drupal and html containers are effectively hidden in front of a proxy. Why “reverse”? The term “proxy” is already used and means that the web user is hidden from the server. If it is the web servers that are hidden (in this case Drupal or the html containers), we use the term “reverse proxy”.

Let’s encrypt is a free certificate authority which certifies that you are the owner of your domain.

We will use nginx-proxy as our reverse proxy. Because that does not take care of certificates, we will use LetsEncrypt companion container for nginx-proxy to set up and maintain Let’s Encrypt certificates.

Let’s start by creating an empty directory which will contain our certificates:

mkdir "$HOME"/certs

Now, following the instructions of the LetsEncrypt companion project, we can set up our reverse proxy:

docker run -d -p 80:80 -p 443:443 \
  --name nginx-proxy \
  -v "$HOME"/certs:/etc/nginx/certs:ro \
  -v /etc/nginx/vhost.d \
  -v /usr/share/nginx/html \
  -v /var/run/docker.sock:/tmp/docker.sock:ro \
  --label com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy \
  --restart=always \
  jwilder/nginx-proxy

And, finally, start the LetEncrypt companion:

docker run -d \
  --name nginx-letsencrypt \
  -v "$HOME"/certs:/etc/nginx/certs:rw \
  -v /var/run/docker.sock:/var/run/docker.sock:ro \
  --volumes-from nginx-proxy \
  --restart=always \
  jrcs/letsencrypt-nginx-proxy-companion

Wait a few minutes for "$HOME"/certs to be populated with your certificate files, and you should now be able to access your sites:

A note about renewals

Let’s Encrypt certificates last 3 months, so we generally want to renew every two months. LetsEncrypt companion container for nginx-proxy states that it automatically renews certificates which are set to expire in less than a month, and it checks this hourly, although there are some renewal-related issues in the issue queue.

It seems to also be possible to force renewals by running:

docker exec nginx-letsencrypt /app/force_renew

So it might be worth considering to be on the lookout for failed renewals and force them if necessary.

Edit: domain-specific configurations

I used this technique to create a Docker registry, and make it accessible securely:

docker run \
  --entrypoint htpasswd \
  registry:2 -Bbn username password > auth/htpasswd

docker run -d --expose 5000 \
  -e "VIRTUAL_HOST=mydomain.example.com" \
  -e "LETSENCRYPT_HOST=mydomain.example.com" \
  -e "[email protected]" \
  -e "REGISTRY_AUTH=htpasswd" \
  -e "REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm" \
  -e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd \ 
  --restart=always -v "$PWD"/auth:/auth \
  --name registry registry:2

But when trying to push an image, I was getting “413 Request Entity Too Large”. This is an error with the nginx-proxy, not the Docker registry. To fix this, you can set domain-specific configurations, in this example we are allowing a maximum of 600M to be passed but only to the Docker registry at mydomain.example.com:

docker exec nginx-proxy /bin/bash -c 'cp /etc/nginx/vhost.d/default /etc/nginx/vhost.d/mydomain.example.com'
docker exec nginx-proxy /bin/bash -c 'echo "client_max_body_size 600M;" >> /etc/nginx/vhost.d/mydomain.example.com'
docker restart nginx-proxy

Edit: Reverse proxy on Drupal 8 or 9

Thanks to @wells on this issue and nitin.k on this issue for pointing me in the right direction on how Drupal can know its base url should be HTTPS. In order to use modules such as social_auth_google and metatag which require Drupal to know its public URL even if it is behind a reverse proxy, you need to figure out the reverse proxy IP.

To do so temporarily install devel_php on your site, and then go to /devel/php and enter dpm($_SERVER['REMOTE_ADDR']);. This will give you a result such as 172.18.0.5. It is not the same IP as what you get when you ping your URL, or when you inspect the headers the reverse proxy sends to Drupal.

Then add this to your settings, and clear your cache:

$settings['reverse_proxy'] = TRUE;
$settings['reverse_proxy_addresses'] = ['172.18.0.5'];

Enjoy!

You can now bask in the knowledge that your cooking blog will not be man-in-the-middled.

Please enable JavaScript to view the comments powered by Disqus.
Aug 21 2016
Aug 21
TL;DR The Google Summer of Code period ends, and am glad that I am able to meet all the goals and develop something productive for the Drupal community. In this blog post, I will be sharing the details of the project, the functionality of the module and its current status.

I am glad that I was one of the lucky students who were selected to be a part of the Google Summer of Code 2016 program for the project “Integrate Google Cloud Vision API to Drupal 8”. The project was under the mentorship of Naveen Valecha, Christian López Espínola and Eugene Ilyin. Under their mentoring and guidance, I am able meet all the goals and develop something productive for the Drupal community.



Google Cloud Vision API bring to picture the automated content analysis of the images. The API can not only detect objects ranging from animals to famous monuments, but also detects faces on emotions. In addition, the API can also help censor images, extract text from images, detect logos and landmarks, and even the attributes of the image itself, for instance the dominant color in the image. Thus, it can serve as a powerful content analysis tool for images.

Now let us see how can we put the module to use, i.e. what are its use cases. To start with, the Google Vision API module allows Taxonomy tagging of image files using Label Detection. Label Detection classifies the images into a number of general purpose categories. For example, classifying a war scenario to war, troop, soldiers, transport, etc. based on the surroundings in the images. This feature of the module is especially important to filter the images based on some tags.

Second feature listing our use case is the Safe Search Detection. It quickly identifies and detects the presence of any explicit or violent contents in an image which are not fit for display. When this feature is enabled in the module, the Safe Search technique validates any image for explicit/violent contents. If found, these images are asked for moderation, and are not allowed to be uploaded on the site, thus keeping the site clean.

Please click here for video demonstration of the two above-mentioned use cases.

Continuing with the other use cases, the third one is Filling the Alternate Text field of an image file. Label, Logo, Landmark and Optical Character Detection feature of the Google Cloud Vision API have been used to implement this use case. Based on the choice entered by the end user, he/she can have the Alternate Text for any image auto filled by one of the four above-mentioned options. The choice “Label Detection” would fill the field with the first value returned in the API response. “Logo Detection” identifies the logos of famous brands, and can be used to fill the field accordingly. Likewise, “Landmark Detection” identifies the monuments and structures, ranging from natural to man-made; and “Optical Character Detection” detects and identifies the texts within an image, and fills the Alternate Text field accordingly.

Next comes the User Emotion Detection feature. This feature is especially important in cases of new account creation. On enabling this feature, it would detect the emotion of the user in the profile picture and notify the new user if he/she seems to be unhappy in the image, prompting them to upload a happy one.

Lastly, the module also allows Displaying the similar image files. Based on the dominant color component (Red, Green or Blue), the module quickly groups all the images which share the same color component, and display them under the “Similar Content” tab in the form of a list. Each item links itself to the image file itself, and is named as per the filename saved by the user. Users should note here that by “similar contents”, we do not mean that the images would resemble each other always. Instead we mean here the same dominant color components.

All the details of my work, the interesting facts and features have been shared on the Drupal Planet.

Please watch this video to know more on how to use the above-mentioned use cases in proper way.

[embedded content]


This is the complete picture of the Google Vision API module developed during the Google Summer of Code phase (May 23, 2016- August 23, 2016).


I would put to use these concepts in best possible way, and try to contribute to the Drupal community with my best efforts.
Aug 16 2016
Aug 16
TL;DR Last week I had worked moving the helper functions for filling Alt Text of image file to a new service; and moving the reused/supporting functions of the tests to an abstract parent class, GoogleVisionTestBase. This week I have worked on improving the documentation of the module and making the label detection results configurable.

With all major issues and features committed to the module, this week I worked on few minor issues, including the documentation and cleanup in the project..

It is an immense pleasure for me that I am getting the feedbacks from the community on the Google Vision API module. An issue Improve documentation for helper functions was created to develop more on documentation and provide the minute details on the code. I have worked on it, and added more documentation to the helper functions so that they can be understood better.

In addition, a need was felt to let the number of results obtained from the Vision API for each of the feature as configurable, and allow the end user to take the control on that. The corresponding issue is Make max results for Label Detection configurable. In my humble opinion, most of the feature implementations and requests to the Google Cloud Vision API have nothing to do with allowing the end user to configure the number of results. For instance, the Safe Search Detection feature detects and avoids the explicit contents to be uploaded, and does not need the number of results to be configurable. However, the taxonomy tagging using Label Detection should be user dependent, and hence, I worked on the issue to make the value configurable only for Label Detection purpose. This value can be configured from the Google Vision settings page, where we set the API key. I have also developed simple web tests to verify that the value is configurable. Presently, the issue is under review.

I have also worked on standard coding fixes and pa-reviews and assisted my mentor, Naveen Valecha to develop interfaces for the services. I assisted him on access rights of the functions, and fixing the documentation issues which clashed with the present one.

Lastly, I worked on improving the README and the module page to include all the new information and instructions implemented during the Google Summer of Code phase.

With all these works done, and all the minor issues resolved, I believe that the module is ready for usage with all the features and end user cases implemented.
Next Week, I’ll work on creating a video demonstration on how to use Google Vision API to fill the Alt Text attribute of an image file, detect the emotion in the user profile pictures and to group the similar images which share the same dominant color.
Aug 09 2016
Aug 09
TL;DR Last week I had worked on modifying the tests for “Fill Alt Text”, “Emotion Detection” and “Image Properties” features of the Google Vision API module. The only tasks left are moving the supporting functions to a separate service, in addition to, creating an abstract parent class for tests and moving the functions there.



There are few supporting functions, namely, google_vision_set_alt_text() and google_vision_edit_alt_text() to fill the Alt Text in accordance to the feature requested from the Vision API, and also to manipulate the value, if needed. I moved these functions to a separate service, namely, FillAltText, and have altered the code to use the functions from there instead of directly accessing them.

In addition, there are a number of supporting functions used in the simple web tests of the module, to create users, contents and fields, which were placed in the test file itself, which in one way, is a kind of redundancy. Hence, I moved all these supporting functions to abstract parent class named GoogleVisionTestBase, and altered the test classes to extend the parent class instead and in place of WebTestBase. This removed the redundant code, as well as, gave a proper structure and orientation to the web tests.
These minor changes would be committed to the module directly, once the major issues are reviewed by my mentors and committed to the module.
Aug 03 2016
Aug 03
TL;DR Last week, I had worked on and developed tests to ensure that the Alt Text field of an image file gets filled in accordance to the various detection features of the Vision API, namely Label Detection, Landmark Detection, Logo Detection and Optical Character Detection. This week I have worked to modify and add tests to various features of the Google Vision module, namely filling of Alt Text field, emotion detection of user pictures and grouping the image files on the basis of their dominant color component.

My mentors reviewed the code and the tests which I had put for review to get them committed to the Google Vision API module. However, the code needs some amendment pointed out by my mentors, which was to be corrected before commit. Hence, I spent this week working on the issues and resolving the flaws, rather than starting with a new feature. Let me start discussing my work in detail.

I had submitted the code and the tests which ensure that the Alt Text field gets properly filled using various detection features according to the end user choice. However, as was pointed out by my mentor, it had one drawback- the user would not be able to manipulate or change the value of the field if he wishes to. Amidst the different options available to the end user to fill the alt text field of the image file, there was a small bug- once an option is selected, it was possible to switch between the options, however, disabling it was not working. After, been pointed out, I worked on modifying the feature and introducing the end user ability to manipulate the value of the field as and when required. Also, I worked on the second bug, and resolved the issues of disabling the feature.

Regarding the Emotion Detection(Face Detection) feature of the Vision API, I was guided to use injections instead of using the static methods directly, and to modify variables. For example, the use of get(‘entity_type.manager’) over the static call \Drupal::entityTypeManager(). Apart from these minor changes, a major issue was the feature was being called whenever an image file is associated with. However, I need to direct it to focus only when the user uploads an image, and not on its removal (as both the actions involves an image file, hence the bug).

In the issue, Implementation of Image Properties feature in the Vision API, I had queried multiple times to the database in the cycle to fetch results and build the routed page using the controllers. However, my mentor instructed me that its really a bad way of implementing the database queries to fetch the results. Hence, I modified the code and changed them to single queries to fetch the result and use them to build the page. In addition, I was asked to build the list using ‘item_list’ instead of using the conventional ‘#prefix’ and ‘#suffix’ to generate the list. Another important change in my approach towards my code was the use of db_query(), the use of which is deprecated. Hence, I switched to use addExpressions() instead of db_query().
Presently, the code is under review by the mentors. I will work further on them, once they get reviewed and I get further instructions on it.
Jul 27 2016
Jul 27
TL;DR Last week, I had worked on and developed tests to ensure that the similar images are grouped in accordance to the Image Properties feature of the Vision API. The code is under review by the mentors, and I would continue on it once the review is done. Meanwhile, they also reviewed the “Fill Alt Text” feature issue, and approved it is good to go. This week, I have worked on developing tests for this issue.

An important feature that I have implemented in the Google Vision API module is the filling of Alt Text field of an image file entity by any of the four choices- Label Detection, Landmark Detection, Logo Detection and Optical Character Detection. My mentor suggested me to check the availability of the response and then fill the field, as we can not fully rely on the third party responses. With this minor suggestion being implemented, now its time to develop tests to ensure the functionality of this feature.

I started developing simple web tests for this feature, to ensure that the Alt Text field is properly filled in accordance to the choice of the user. It requires the selection of the four choices one by one and verify that the field is filled correctly. Thus we require four tests to test the entire functionality. I have added an extra test to ensure that if none of the options are selected then the field remains empty.

I created the image files using the images available in the simpletests. The images can be accessed through drupalGetTestFiles(). The filling, however, requires call to the Google Cloud Vision API, thus inducing dependency on the API key. To remove the dependency, I mocked the function in the test module, returning the custom data to implement the feature.

The first test ensures that the Label Detection feature returns correct response and the Alt Text field is filled correctly. The simpletest provides a list of assertions to verify it, however, I found assertFieldByName() to be most suitable for the purpose. It asserts the value of a field based on the field name. The second test ensures that the Landmark Detection feature works correctly. Similarly, the third and fourth test ensures the correct functionality of the Logo and the Optical Character Detection feature.

The fifth test which I have included perform tests when none of the options are selected. It ensures that under this case, the Alt Text field remains empty, and does not contain any unwanted values.
I have posted the patch covering the suggestions and tests on the issue queue Fill the Alt Text of the Image File using Google Vision API to be reviewed by my mentors. Once they review it, I would work on it further, if required.
Jul 26 2016
Jul 26
TL;DR In the past two weeks I had worked on using the Image Properties feature offered by the Google Cloud Vision API to group the image files together on the basis of the dominant color components filling them. In addition, I had also worked on detecting the image files and filling the Alternate Text field based on the results of Label/Landmark/Logo/Optical Character Detection, based on the demand of the end user. This week, I have worked on and developed tests to ensure that the similar images are grouped in accordance to the Image Properties feature of the Vision API.

At present, the Google Vision API module supports the Label Detection feature to be used as taxonomy terms, the Safe Search Detection feature to avoid displaying any explicit contents or violence and the User Emotion detection to detect the emotions of the users in their profile pictures and notify them about it.

I had worked on grouping the images on the basis of the dominant color component(Red, Green or Blue) which they are comprised of. I got the code reviewed by my mentors, and they approved it with minor suggestions on injecting the constructors wherever possible. Following their suggestions, I injected the Connection object instead of accessing the database via \Drupal::database().

After making changes as per the suggestions, I started developing simple web tests for this feature, to ensure that the similar images gets displayed under the SImilarContents tab. It requires the creation of new taxonomy vocabulary and adding an entity reference field to the image file entity. After the creation of the new Vocabulary and addition of the new field to the image file, I created the image files using the images available in the simpletests. The images can be accessed through drupalGetTestFiles(). The first test ensures that if the Vocabulary named ‘Dominant Color’ is selected, the similar images gets displayed under the file/{file_id}/similarcontent link.

The grouping, however, requires call to the Google Cloud Vision API, thus inducing dependency on the API key. To remove the dependency, I mocked the function in the test module, returning the custom data to implement the grouping.

To cover the negative aspect, i.e. the case when the Dominant Color option is not selected, I have developed another test which creates a demo vocabulary to simply store the labels, instead of the dominant color component. In this case, the file/{file_id}/similarcontent link displays the message “No items found”.
I have posted the patch covering the suggestions and tests on the issue queue to be reviewed by my mentors. Once they review it, I would work on it further, if required.
Jul 14 2016
Jul 14
TL;DR Previous week I had worked on detecting the emotion in the profile pictures of the users, and notifying them to change the image if they do not look happy. The work is under review by the mentors. Once it gets reviewed, I would resume it if it needs any changes. This week I have worked on filling the ‘Alt Text’ field of an image file based on any one of the method selected by the end user- Label Detection, Landmark Detection, Logo Detection and Optical Character Detection.

Last week, I had worked on implementing the Face Detection feature in the Google Vision API module. The code is currently under the review by the mentors. Once, they review it, I would develop further on it if it requires any changes.

The Google Cloud Vision API provides the features to detect popular landmarks in an image(Landmark Detection), logos of popular brands(Logo Detection), texts within an image(Optical Character Detection), in addition to Label Detection. These features, though of less significance, are helpful in identifying an image. Hence, I have started working on implementing a new helpful case for the users- Filling of the Alternate Text field of an image file using these features.

The Alt Text field of the image file entity is modified to incorporate the options to fill the field using the features. The user may select any one of the four options to fill the Alt Text field of the image.

Coming to the technical aspect, I have made use of hook_form_BASE_FORM_ID_alter() to alter the Alternate Text field of the image file entity. I have modified the edit form of the Alt Text field to add four radio options, namely- Label Detection, Landmark Detection, Logo Detection and Optical Character Detection. The user may select any of the options and save the configuration. The Alternate Text field would be filled up accordingly.
Presently, the code is under the review by the mentors. Once it gets reviewed, I would make suggested changes, if required.
Jul 07 2016
Jul 07
TL;DR Previous week I had worked on grouping the contents based on the dominant color component in their images, if present. The work is under review of the mentors. And once, it gets reviewed, I would work further on that issue. Meanwhile, I have started developing and implementing the Emotion Detection feature of the Google Cloud Vision API. It would detect the emotion of the person in the profile picture uploaded, and if the person looks angry or unhappy, he would be notified thereof. This feature is especially important when building sites for professional purposes, as the facial looks matters a lot in such cases.

Last week, I had worked on implementing the Dominant Color Detection feature in the Google Vision API module. The code is currently under the review by the mentors. Once, they review it, I would develop further on it if it requires any changes.

Hence, meanwhile, I have started working on implementing a new feature Face Detection in an image. This feature gives us the location of the face in an image, and in addition, the emotions and expressions on the face.

I have used this feature to detect the emotion of the person in the profile picture uploaded by him/her. If the person does not seem happy in the image, he/she is notified thereof of their expressions. This is especially useful when the end users are developing a site for professional purposes, as in professional matters, expressions matters a lot.

Coming to the technical aspect, I have made use of hook_entity_bundle_field_info_alter() to alter the image fields, and check the emotions in the uploaded images. This function has been used, as we only want to implement this feature on the image fields. If the image is not a happy one, then appropriate message is displayed using drupal_set_message(). This feature also makes use of Constraints and Validators just like the Safe Search detection feature. Presently, the code is under the review by the mentors.

In addition to the implementation of Face Detection, I also worked on expanding the tests of the Safe Search Detection feature of the Google Vision API module to test other entities as well, in addition to the nodes. I have expanded the tests to test the safe search constraint on the comment entity as well. This requires the creation of a dummy comment type, adding an image field to the comment type, and attaching the comment to the content type. The image field contains the safe search as the constraint on it. This test is basically similar to the tests present in the module for the node entity. The code is under review by the mentors and would soon be committed to the module. For reference on how to create dummy comment types and attaching it to the content types, the CommentTestBase class is very helpful.
Jun 29 2016
Jun 29
TL;DR The safe search constraint feature is now committed to the module along with proper web tests. So, this week I started off with a new feature offered by the Google Cloud Vision API- “Image Properties Detection”. It detects various properties and attributes of an image, namely, the RGB components, pixel fraction and score. I have worked on to detect the dominant component in the image present in any content, and display all the contents sharing similar dominant color. It is pretty much like what we see on the e-commerce sites.

Previous week I had worked on writing web tests for the safe search constraint validation on the image fields. This feature is now committed in the module Google Vision API.

This week I have worked on implementing another feature provided by the Google Cloud Vision API, namely, Image Properties Detection. This feature detects the color components of red, green and blue colors in the images along with their pixel fractions and scores. I have used this feature to determine the dominant color component (i.e. red, blue or green) in the image, and to display all those contents which have the same dominant color in their images.

I have developed the code which creates a new route- /node/{nid}/relatedcontent to display the related contents in the form of a list. This concept makes use of Controllers and Routing System of Drupal 8. The Controller class is extended to render the output of our page in the format we require. The contents are displayed in the form of list with the links to their respective nodes, and are named by their titles.

In addition to the grouping of similar contents, the colors are also stored in the form of taxonomy terms under a taxonomy vocabulary programmatically generated under the name Dominant Colors.

This issue is still under progress, and requires little modification. I need to add the link to the new route in each of the nodes, so as to  get a better interface to access those contents. Henceforth, I will put this patch for review.
A very simple example of creating routes and controllers in your module can be found here.
Jun 29 2016
Jun 29
TL;DR It has been over a month since I started working on my Drupal project “Integrate Google Cloud Vision API to Drupal 8”, and gradually I have crossed the second stage towards the completion of the project, first being selection in the Google Summer of Code 2016 programme. Here, I would like to share my experiences and accomplishments during this one month journey, and also I would like to summarize my further plans with the project and the features which I would be implementing in the coming two months.

Let me first describe the significance of this post and what actually does “midterm submission” means? The GSOC coding phase has been divided into two halves, viz. Midterm submission and Final submission. In the first half, the students try to accomplish around 50% of the project, and submit their work to the mentors for evaluation. Those who passed the midterm evaluations are allowed to proceed further and complete the remaining portion of their project.

Now coming back to my experiences, after successfully passing through the Community Bonding period of the GSOC 2016 programme, now it was the time for start coding our project proposal to reality. As I had shared earlier that during the Community Bonding period, I came to know that the project has already been initiated by Eugene Ilyin,(who is now a part of my GSOC team). So, we discussed upon the project and set a roadmap of the project and the goals we need to achieve in the GSOC period. I had started coding the very first day of the coding phase, moving the new as well as existing functions to services. My mentors Naveen Valecha, Christian López Espínola and Eugene Ilyin really helped me a lot and guided me whenever and wherever I needed their guidance and support. They helped me to get through new concepts and guided me to implement them in the most effective way to make the module the best that we could.

During this period, I also came to learn about a lot of new techniques and concepts which I had not implemented earlier. Right from the very first day of the coding period, I have been coming across new things everyday, and it is really interesting and fun to learn all those techniques. I also learnt about the use of validators and constraints and how can they be implemented both on general basis or specifically on fields. I also learnt about how to create test modules and alter various classes and their functions in our tests so as to remove the dependency on web access or on valid informations for our testing purposes. I learnt new things every day and enjoyed implementing them to code our module plan into reality. At present, the module supports the Label Detection feature of the Vision API, along with the tests to verify whether the API key has been set by the end user or not. Currently, the feature of Safe Search Detection is available as a patch which can be found here, which would soon be committed to the module.
[embedded content] I have shared all the details of my work on the Drupal Planet. Please watch this video for detailed information on how to use the Google Vision API module on your Drupal site.
Jun 23 2016
Jun 23
TL;DR In my last post Avoid Explicit Contents in the images using Google Vision module, I had discussed about the services which “Safe Search” feature of the Vision API provides us, and how we have put this into use in the Google Vision module as a constraint to all the image fields which would be validated when the option is enabled. This week I have worked on developing simple web tests for testing this feature whether it gives us the results as expected.

Last week I had worked on developing the code to use the Safe Search detection feature as a constraint to the image fields which would validate the images for the presence of explicit contents, provided that the user enables the configuration for the concerned image field.

Besides the code, testing the functionality using simple web tests are equally essential to ensure that the feature executes perfectly when necessary steps are implemented. Hence, this week I have worked on developing simple web tests, which ensures that we have a fully functional feature.

I have tested both the conditions with safe search enabled and disabled to verify the results which should be obtained. When the safe search is enabled, any image containing any sort of  explicit content, is detected, and asked for moderation. If the image is not moderated, then the image is not saved. When the same image was passed through the second test, with safe search disabled, it was stored successfully, thus providing us the expected results.

To conduct the test successfully, I had to create a demo content type in my tests using drupalCreateContentType(), which would have an image field with the ‘Enable Safe Search’ option. This was something new to me to how to add an extra field to the default content type settings. The Drupal documentation on FieldConfig and FieldStorageConfig were of great help to understand the underlying concepts and functions which the field offers, and thus helping me to create custom fields programmatically. However, in order to perform the test, I had to call the API directly, which required a valid API key and an image which actually contains explicit content. Hence, my mentors asked me to override the functions of the class (mocking the services) in such a way that it removes the dependency from both the key and the image. Thus, I created a test module inside the Google Vision module, and override the function.

Summarizing the above, I can say that in addition to learning how to test the constraints and validators, I also came to learn about some really cool techniques, including the creation of custom fields in tests and mocking the services.
Jun 14 2016
Jun 14
TL;DR Safe Search detection of the Google Cloud Vision API allows the end users to avoid any explicit content or violence in images to be uploaded on the site. I worked on integrating this feature to the module as a constraint to those image fields which have the “Safe Search” feature enabled.

Let me first give a brief summary about the current status of the module Google Vision. In the earlier weeks, I had implemented the services and wrote tests for the module, which are now committed to the module.

Now, coming to the progress of this week.

I had started with integrating the Safe Search detection feature in the module. Safe Search detection allows its users to detect any explicit contents within the image, and hence can be very useful for site administrators who do not want to display any explicit images on their sites.

This feature was initially integrated using a function call in the module. However, my mentors suggested that this feature should rather be a Constraint on the image fields, which would be validated if the feature is enabled for the field. It depends on the user whether to keep safe search on their site or not, and they can implement it any time they want by just enabling/disabling the checkbox on the image field. Hence, now it is a user choice, rather than the developer’s choice whether to implement this feature or not.

Presently, the code is under review by my mentors whether it needs changes or is ready for commit.

Constraints and Validators are wonderful features of Drupal 8. Constraints, as the name goes, are certain restrictions which we pose on the various fields. Validators are used to implement the logic to create the situation under which the constraints are to be applied. Some helpful examples of applying custom constraints and validators can also be found Sitepoint.
This week had a lot of new things stored for me. I had no idea about the constraints and validators when I was asked to implement them at the first place. I spent hours on them, learning about them and seeking guidance from my mentors on the issues I faced. I developed gradually on it, and by the end of the week, I was able to implement them for the safe search detection feature.
Jun 07 2016
Jun 07
TL;DR I have already created services for the functions which bridges the module to the API, implementing the features offered by the Google Cloud Vision API, thus completing my first step towards Integrating Google Cloud Vision API to Drupal 8. This week I worked on generating error reports if the API key is not set by the user, and developing tests to test the API key configuration and whether the key is stored successfully or not.

The first step towards the integration of Google Cloud Vision API to Drupal 8 was completed with the functions moved to services. I had posted the patch for the review by my mentors. They provided their suggestions on the patch, which I worked on, and every step resulting in a better and more beautiful code.

I would also like to share that this week, our team expanded from a team of three, to a team of four members. Yes! Eugene Ilyin, the original maintainer of the module Google Vision API has joined us to mentor me through the project.

Now coming to the progress of the project, the schedule says I need to configure the Google Cloud Vision API at taxonomy field level, so that the end users may use the taxonomy terms to get the desired response from the API. However, the module already used the configuration for Label Detection, and in a discussion with my mentors, it was figured out that the current configuration does not need any changes, as at present the behaviour is pretty clear and obvious to let the developers use it easily; rather we should work on implementing the runtime verification and storage of API key supplied by the end users. I was required to write and implement the code which would give error report if the API key was not saved prior to the use of the module, and also to write tests for verifying the configuration and ensuring the storage of the key.

I created patches and posted it in the issue to get it reviewed by my mentors. I brought the suggested changes in the code and finally have submitted the patch implementing the required functionalities. I also worked on this issue, solving and implementing minor suggestions before it gets ready to be accepted and committed! And finally, my first patch in this project has been accepted and the changes has been reflected in the module.
At the end of these two weeks, I learnt about services and dependency injection which prove to be very useful concepts implemented in Drupal 8. I also had experiences of writing tests to check the runtime functionality of the module.
May 31 2016
May 31
TL;DR I have started working on the project Integrate Google Cloud Vision API to Drupal 8. As of the task of this week, the common functions have been moved to services. The crucial concepts involved Dependency Injection and use of Guzzle over curl.

My mentors suggested me to create issues for my tasks in the issue queues of the module Google Vision API, where they would track my progress and also discuss and leave suggestions for the improvement of the module. Thus, starting with my task for the first week, I created the issue Moving the common functions to services in the issue queue of the module and started coding the functions into services and injecting them as and when needed. I started the week by learning the concepts of services and containers, and gradually learnt about Dependency Injections in Drupal 8. The post on Services and dependency injection in Drupal 8 and the videos of Drupalize.me were of great help to understand the concept of services and implement dependency injection.

After completing this part, I put the patch for review, and there followed the next part- Use of Guzzle over curl in the services and injecting the httpClient service. I spent significant time learning the concept of Guzzle which was quite new for me. My mentors Naveen Valecha and Christian López Espínola helped me a lot to understand Guzzle, and subsequently this task was completed with Guzzle replacing curl and injection of httpClient. In addition, the present code made use of concatenated strings for sending the data during API call. I changed the code to make use of arrays and Json utility class and its static functions to send the data instead of strings. When the code seemed perfect, my mentors suggested me to perform clean up along with proper documentation.
At the end of the week, I successfully uploaded the patch with all the suggestions implemented, clean up done and documentation added, thereby completing the task for my first week.
May 23 2016
May 23
TL;DR I’ll be working on Integrating Google Cloud Vision API to Drupal 8 this summer as part of Google Summer of Code programme. During my community bonding period I got the chance to work with lots of grey shavers in drupal community.

I have been selected as one of the eleven students for the Google Summer of Code, who would be coding their summer away for Drupal. This summer I will be working on project Integrate Google Cloud Vision API to Drupal 8. The API detects objects ranging from animals to famous monuments and landmarks. Not only that, it supports the recognition of a wide range of famous logos/brands. Face detection, text detection within an image and safe searches are among the other features which this API provides. It is really cool for a Content Management System to have such a powerful feature, so I would be working this summer to implement this feature in Drupal.

Before jumping directly into the coding phase, the students selected for the Google Summer of Code, were given a month to bond with the community, their mentors and other friends with whom they share the same platform. This phase is known as “Community Bonding Period”, which lasted from April 23, 2016 to May 22, 2016.

Everyday brings up new opportunities to do something new, meet like-minded people across the globe, and contribute to awesomeness!! This is the simplest way in which I can put the "Community Bonding" period.

This is the ideal time when the students get closer to the community, interact with their mentors, learn more about the organization codebase and have a more detailed discussion on the project and its timelines before actually start the coding.

To mentor me through the project, I have been blessed with two experienced mentors- Naveen Valecha and Christian López Espínola. I have been discussing with them on the project, its execution and the tough areas for last few weeks. I was also guided to learn some basic methodologies, including the services and configuration management which would be of great help in my project. Meanwhile, I also devoted my time to the community by contributing in the core issues, providing suggestions, developing patches and reviewing the project applications, including Field Group Table Component and AddressField Copy of the newbies in the Drupal world, highlighting how can they develop it in a better way, and thus lightening the burden of the community as far as I could.

In this one month, I met a lot of like-minded across the globe who have gathered together to take the community to greater heights, and I felt myself lucky enough to get an opportunity to work with them. I tried developing good bonding with my mentors, being in touch with them and bringing to their notice all my actions and tracks. 
This I suppose is the most essential purpose of this one month pre-coding phase.

Here, I would also like to share a strange experience of mine during the community bonding period. During this phase, I came to know that the project on which I am working on, was already initiated by Eugene Ilyin. Here lies the bond strength of the Drupal Community. On having the discussion on this with Eugene, he not only agreed to let me work on the project, but has also agreed to mentor me on my project. It once again proved the awesomeness of the community, and how all of us come together, collaborate and help each other to grow as well as take the community to greater heights. This was all about my month of Community Bonding.

The month ends, and now we jump into coding the summer away.
Apr 30 2016
Apr 30
TL;DR My first contribution to Drupal was a small contributed module named Image CircleSlider. It had gone through different stages of examination by different reviewers, and during this process, I came to know about the work flow of a sandbox project to a full project.

Since the release of my first module Image Circle Slider, I have been asked this question at several instances that what did I do to get my module to be accepted.

So here I am discussing my experience of the first module I contributed. I was an intern at Innoraft Solutions Pvt. Ltd., where I came across this jQuery plugin, and was asked to implement it in Drupal. Though it was a very simple plugin, yet was an innovative one, and brought the images in a very modern way, instead of traditional sliding system. After developing the code of the module, I created it as sandbox project, and queued it to get it reviewed by the community. P.S. A useful tip in order to go to the top of the priority list- Take part in the review process yourself, and help others to get their review done. It earns a special tag to your project: "PAReview Bonus".  Drupal is a friendly community, and we here believe in joining hands and working, instead of the stiff competition which exists in today's world, and hence this review process. Once the reviewers start reviewing your project, they would be providing their valuable suggestions, and each step would take your project to its completion and acceptance. And when sufficient reviews have been done, and there would be no application blockers, it would be marked as "Reviewed and Tested by the Community", which the pre-final step and ultimately the git administrator would give you the access to release your project as Full Project. This is a one-time process, and once you have been given git access, you can directly release your projects, and give your service to the community. Another important point: I have seen many a times that the contributors gets restless during the process, and want to get it done within a single day. However, things does not go this way. Please be patient during the review process. These things does not happen overnight, and you need to be patient and coordinate with the community.

Hope this helps to all the new comers. Happy contributing!!

Mar 15 2016
Mar 15
TL;DR I'll be working on the project Integrate Google Cloud Vision API to Drupal 8 this summer as part of the Google Summer of Code programme. This API offers automated content analysis of the images, including landmark detection, logo detection, face detection, text detection in images among others.

The Google Summer of Code results are out!! And I have been one of those few students who have successfully been able to get their names on the list of selected students, who would be working for the next three months on the Open Source Projects.

Now, let me describe here the project and its applications.

Well, Google Cloud Vision API bring to picture the automated content analysis of the images. The API can not only detect objects ranging from animals to famous monuments, but also detects faces on emotions. In addition, the API can also help censor images, extract text from images, detect logos and landmarks, and even the attributes of the image itself, for instance the dominant color in the image.

What is so special about this project idea? This is the first question which comes in every mind regarding any GSOC project. So here is the answer. This feature has not been implemented in the Content Management Systems(CMS), and its integration in Drupal can give the users, a powerful tool which carries out automated content analysis of the images. Integration of this API would not only add to Drupal characteristics, but also would open up a new horizon for Drupal. Won't it be great if a CMS can detect logo of famous brands? Or some explicit contents not fit to be displayed? Or the detection of natural as well as artificial structures in an image? So, to summarize the aspects of Vision API, it offers the following features:
  1. Label Detection- It detects broad set of categories in an image, ranging from living animals to non-living places.
  2. Explicit Content Detection- Detects and notifies about adult or violent content within an image.
  3. Logo Detection- Detects the logos of popular brands.
  4. Landmark Detection- Detects natural as well as artificial structures within an image.
  5. Image Attributes- Detects general attributes of an image, such as dominant color and background shades.
  6. Optical Character Recognition(OCR)- Detects and extracts texts within an image. Vision API supports a broad range of languages along with the support for automatic language identification.
  7. Face Detection- Detects multiple faces within an image, along with associated key facial attributes like emotional state. However, it still does not embed the support of facial recognition.
  8. Integrated REST API- Access via REST API to request one or more annotation types per image.

For more details on Google Cloud Vision API, please have a look at Vision API 
May 10 2015
May 10

After using the new gmail 'tabs' for a while, I began just ignoring anything not in the primary tab. It turns out, this included notifications from Godaddy regarding domain renewals. I neglected to renew devbee.com, and as sure as the sun rises, it got scooped up immediately by a ... person.

I've owned domains for the last twenty years, and I never thought it would happen to me :) At first I thought no big deal, I'll just use my backup devbee.net domain (which I registered a month after devbee.com and so was able to recover it before it expired). But I've been using [email protected] for my client work for about ten years now, and there's nothing that says amateur hour like bounced email.

I am in touch with the new owner of my domain, and I'll definitely be reporting back on the results of our discussion.  

Apr 07 2014
Apr 07

The first revision control system I ever used was called RCS. It was the pre-cursor to CVS and stored all revision data locally. It was nifty but very limited and not suited for group development. CVS was the first shared revisioning system I used. It was rock solid, IMHO. But it had a few big problems, like the inability to rename or move files. Everything had to be deleted and re-added. 

Since those days, I've used several other revisioning systems: Perforce, Bitkeeper, Clearcase, Subversion and GIT.

I'm tired of learning yet another system. I just want to know which horse is going to win the race for the forseeable future and go all in.

That's where Google Trends comes in very handy. It quickly reveals that I need to bet on GIT. 

I just hope I can make it through the next 5 years or more before having to learn the next greatest solution to our shared problem of tracking code revisions.

Sep 30 2012
Sep 30

I recently worked on a project which required a updated version of the jQuery library. While there is the jQuery Update module, it only allows you to upgrade Drupal 6 to jQuery 1.3. If you really know what you're doing and want to upgrade beyond that version, you can either hack core or create your own simple module to do it. While hacking core is certainly the easier approach (simply overwriting misc/jquery.js with a newer version), it is very bad practice. You do not want to get yourself in the habit of altering Drupal core unless you want to kill your upgrade path and deal with a new slew of bugs and unpredictable behavior.

Let's start by creating an admin area for configuring the version of jQuery we want to use.

/**
* Module configuration admin form.
*
*/
function mymodule_admin_form() {
 
$form['mymodule_custom_jquery'] = array(
   
'#type' => 'checkbox',
   
'#title' => t('Override Jquery'),
   
'#description' => t('Replace the version of jQuery that ships with Drupal
      (jQuery 1.2.6) with the jQuery library specified below. You will need to
      flush your cache when turning this feature on or off.'
),
   
'#default_value' => variable_get('mymodule_custom_jquery', 0),
  );
 
$form['mymodule_custom_jquery_file'] = array(
   
'#type' => 'textfield',
   
'#title' => t('Jquery file location'),
   
'#description' => t('Specify the location of the updated jQuery library
      you would like to use. The location is relative to the docroot and
      so should probably begin with "sites/".'
),
   
'#size' => 128,
   
'#default_value' => variable_get('mymodule_custom_jquery_file', NULL),
  );

  </span>$form = system_settings_form($form);

  return </span>$form;
}
?>

Next, we're going to write a validation hook that simply makes sure the file exists.

/**
* Admin form validation callback.
*
*/
function mymodule_admin_form_validate($form, &$form_state) {
 
$file = trim($form_state['values']['mymodule_custom_jquery_file']);
 
$form_state['values']['mymodule_custom_jquery_file'] = $file;

  </span>// Only validate this value if js override is turned on
 
if (variable_get('mymodule_custom_jquery', 0)) {
    if (!
file_exists($file)) {
     
form_set_error('mymodule_custom_jquery_file', t('The file you specified does not exist: !file.', array('!file' => $file)));
    }
  }
}
?>

And lastly, we use hook_preprocess_page() to safely replace the jQuery that ships with Drupal core with our own version.

/**
* Implementation of hook_preprocess_page().
*/
function mymodule_preprocess_page(&$variables) {
  if (
variable_get('mymodule_custom_jquery', 0)) {
   
$file = variable_get('mymodule_custom_jquery_file', 0);
   
$scripts = drupal_add_js();

    </span>// remove core jquery and add our own
   
unset($scripts['core']['misc/jquery.js']);
   
$add_scripts['core'][$file] = array(
     
'cache' => TRUE,
     
'defer' => FALSE,
     
'preprocess' => TRUE,
    );
   
$scripts['core'] = array_merge($add_scripts['core'], $scripts['core']);
   
$variables['scripts'] = drupal_get_js('header', $scripts);
  }
}
?>

Make note of the line:

    $scripts['core'] = array_merge($add_scripts['core'], $scripts['core']);
?>

We are careful to add our new jQuery include at the beginning of the array (where the original jquery.js include was) in order to meet any jquery dependencies in the scripts that follow.

Sep 01 2012
Sep 01

I'm not sure how it happened, but today I noticed that Drupal's menus were behaving very oddly. After upgrading to Drupal 6 and installing several additional modules, I noticed duplicate menu entries as well as other disturbing oddities. Items I was placing into the menu were not showing up. Menu items that I moved around were apparently saved but they did not appear properly in a dropdown context. 

Looking further into it via the absolutely awesome SQLYog tool, I verified that there were dozens of duplicate entries. Some items were duplicated up to six times. It was a real mess.

The database tables involved here are menu_links and menu_router. These are two of the more mysterious tables in Drupal. I haven't had the need to spend much time with them in the past, and I know now that this is a good thing. Fortunately, you do not have to know anything about them to fix this problem. While I spent a couple hours carefully deleting items from this table, ultimately I gave up. I was able to remove all the duplicates, but the menus were still misbehaving. At this point, I just wanted to do a factory reset on the menus, but it's not so simple as flushing cache. However, that is not far from the solution.

This solution will do a 'factory reset' on your menus. You will lose any customizations you have made. However, all core and contrib module entries will be restored very nicely.

Please backup your entire database before doing any destructive database manipulation. 

Step one is to empty the corrupted tables:

In your favorite SQL client, run the following commands:

truncate menu_links;
truncate menu_router;

At this point, your site will be completely unusable. But not for long. To complete the final step, you will need to be comfortable with the Drupal admin's best friend, drush.

Simply run the following commands from your terminal (Viva tcsh!):

drush php-eval 'menu_router_build();'
drush cc menu

Now my menus are as fresh as the day they were installed.

Though I could not clearly identify the cause of this problem, I would suggest backing up your database before installing the Taxonomy Menu module

Aug 27 2012
Aug 27

I've been putting it off for a few years, but I finally decided to upgrade devbee.com to Drupal 6. 

I didn't really need to, but it bothered me that I wasn't running supported code and I figured I might learn something. And I did. Mostly obvious things that I should be familiar with already. 

Drush

I've only ever played around with this. I don't like learning new things unless they are going to be truly useful to me. Drush is definitely something I shouldn't have ignored for so long. It comes in particularly handy when doing a site upgrade as you can download and install modules, clear cache, set variables, run DB updates and a lot more all from the command line. This tool is crazy good if you're comfortable in a terminal.

Themeing didn't change much

I was able to convert my dated theme to 6 with very little effort. I created a .info file, changed the names of the region variables in my page.tpl.php file and tweaked a small amount of CSS. Took me about 20 minutes. Pretty painless. I was able to preserve my table-based layout so all you markup nazis will have an example of bad design.

Minimalism

The whole process was made much easier by the fact that I don't like using a lot of modules. I tend to go with a handful of the more popular modules such as Views, CKeditor, CCK, Pathauto, etc... Because I use only popular modules, I'm almost certainly going to have no trouble upgrading them. At least that's my experience.

Key things to remember

  1. Start with a brand new install of D6 with no contrib modules. It's tempting to try to force an upgrade on your existing site, but it's just not going to work. You must create a clean install. 
  2. Import your old database into the clean D6 database. I tried to skip this and just point D6 to my old database, but then  you end up with missing tables and a whole lot of brokenness.
  3. Run DB updates and get your core Drupal site working before worrying about contrib modules.
  4. Lastly, and this is where drush will save you time, download and enable the contrib modules you need one at a time. 

Unfortunately, the upgrade process is no longer something that can be done by the average site owner. There are going to be snags, and you're going to need to know how to interact with your database to get things up and running. I think this is the reason so many users are still on D5 and D6. I'd really like to see an upgrade path that was more accessible to non-developers, but at the same time, I'm also grateful that Drupal is upgradable and there is no need to manually migrate data for major upgrades. 

At this rate, I should be upgrading to D7 sometime around the end of the decade.

Jan 10 2012
Jan 10

The following is a guest post by Mitchel Xavier

One of the challenges of developing with Drupal is to understand Drupal’s structure. Until now, when working with the DOM structure, the DOM inspector has been the best tool for viewing the structure. A new tool has been created to make the visualization of the DOM structure much easier to interpret. It is a Firefox add-on and is called Tilt 3D. It creates 3 dimensional interactive representation of the DOM elements as a layered visual image.

A requirement to use Tilt 3D is that your browser supports WebGL. WebGL is a Javascript software library which allows for the creation of 3D graphics very quickly and without the requirement for additional plugins. Currently Firefox is the only browser to support this tool. Firefox has supported WebGL since version 4. The other requirement for Tilt 3D is that it is supported with a capable graphics card.

Tilt 3D is extremely useful for many reasons. When you click on each node, you can see the structure of each element. You can view the html and css of each node. It is great for debugging html structure issues. It also provides the option to refresh in real time during all changes made in Firebug.

Tilt 3D was created by Victor Prof and Rob Campbell, who created Firebug. One of the advantages of Tilt 3D is it’s smooth interface. It is very intuitive to use and creates a completely new way of interacting and obtaining information about your Drupal website. If you are starting work on an existing Drupal project, Tilt 3D would be a great way to understand the structure of that particular project.

[embedded content]

Mitchel Xavier is a Drupal website designer and Drupal 7 specialist.

Apr 01 2009
Apr 01

Developers are all familiar with the default behavior of the drupal menu systems "local tasks" (aka tabs). These appear throughout most Drupal sites, primarily in the administration area, but also on other pages like the user profile.

Generally, developers are pretty good about creating logical local tasks, meaning only those menu items which logically live under another menu item (like view, edit, revisions, workflow, etc... live under the node/% menu item).

But sometimes, these tabs either don't really make sense as tabs or you simply want to have the flexibility of working with the items as "normal menu items", or those menu items which appear under admin/build/menu.

I recently wanted to move some of the tabs on the user profile page (user/UID) into the main menu so that I could include them as blocks.

For some reason, developers think the user profile page is a great place to put tabs for user related pages such as friendslist, tracker, bookmarks, notifications and so on. But these types of items are less a part of the user's account information than they are resources for specific users. Personally, I would not think to look at my account information on a site to find stuff like favorites or buddies. I'd expect those items to be presented somewhere much more obvious like a navigation block.

Initially, this may seem like a trivial task. My first thought was to simply use hook_menu_alter() and change the 'type' value of the menu item from MENU_LOCAL_TASK to MENU_NORMAL_ITEM. However, for reasons I don't understand well enough to explain in detail, this does not work.

In order to achieve the desired result, you must change the path of the menu item and incorporate the '%user_uid_optional' argument, replacing the default '%user' argument.

All very confusing, I know. Let's look at an example.

The notifications module (which provides notification on changes to subscribed to content) uses the user profile page rather heavily. I don't want its links there, I want them in the sidebar where users can always see them.

/**
* Implementation of hook_menu_alter().
*/
function MODULENAME_menu_alter(&amp;$callbacks) {
 
// NOTIFICATIONS MODULE
 
$callbacks['notifications/%user_uid_optional'] = $callbacks['user/%user/notifications'];
 
$callbacks['notifications/%user_uid_optional']['type'] = MENU_NORMAL_ITEM;
  unset(
$callbacks['user/%user/notifications']);
  <
SNIP>
}
?>

So I have moved the notifications menu into my own menu, changed the type, used %user_uid_optional instead of %user, and unset the original menu item.

This works fine except for the fact that you'll lose all of the other menu items under user/%user/notifications! You need to account for all menu items in the hierarchy to properly reproduce the tabs in the main menu system, so we add the following:

    $callbacks['notifications/%user_uid_optional/thread'] = $callbacks['user/%user/notifications/thread'];
    unset(
$callbacks['user/%user/notifications/thread']);

    </span>$callbacks['notifications/%user_uid_optional/nodetype'] = $callbacks['user/%user/notifications/nodetype'];
    unset(
$callbacks['user/%user/notifications/nodetype']);

    </span>$callbacks['notifications/%user_uid_optional/author'] = $callbacks['user/%user/notifications/author'];
    unset(
$callbacks['user/%user/notifications/author']);
?>

And of course, we don't want this code executing at all if our module is not enabled, so you'd want to wrap the whole thing in:

  if (module_exists('notifications')) {
 
  <
SNIP>

  }
?>

Keep in mind that not all modules implement menu items using hook_menu(). It's becoming more and more common for developers to rely on the views module to generate menu items, and this is a wise choice. Menus generated using views (ala bookmark module) can be modified to get the desired result without any custom code.

Sep 01 2008
Sep 01

I eagerly awaited the videos for the Boston Drupalcon. They never seemed to appear. 

However, the videos for Szeged are already available here ! The quality is very good. Audio is good and the presentation screens are fuzzy but you can get the general idea. 

Very impressive. There are many hours of must watch video here if you were unable to attend the conference live.

Aug 09 2008
Aug 09

After over three years as an independent web developer, I've had an opportunity drop in my lap that I am going to take advantage of.

WorkHabit is in the process of opening up shop in San Diego, and I'm joining them. They've got a top-drawer team, and they don't do background checks, so it's a perfect match for me.

I've enjoyed working independently for the last several years, however, I've also learned a lot about running my own business. The primary lesson I've learned is that running a solo web development shop involves a lot more than web development chops. I've spent more time than I would have liked doing things like billing, support, follow-ups, "sales", and all the various tax and legal oriented tasks that come with working for oneself. 

I consider myself a worker bee. I love web development. All the other stuff, not so much.

WorkHabit is truly committed to contributing to the Drupal community. This is a must-have for me. I truly believe that no Drupal based shop can thrive without being directly and heavily involved in it's people and development. I've got a thing against obviously successful Drupal businesses that take but never give. It's just wrong and not too smart. WorkHabit gets it, and I'm looking forward to being able to increase my work with and contributions to the Drupal community. 

I'll be working alongside Drupal Dojo host Joel Ferris (senpai) and Michael "mad man" Murdoch (froggacuda), helping create WorkHabit's San Diego presence. 

We are going to be more heavily supporting the local San Diego DUG in the future as well.

Mar 16 2008
Mar 16

The number of available Drupal modules is continuing to grow dramatically. Like a lot of other Drupal users, I spend a good deal of time downloading new modules and trying them out to see what they do. Unfortunately, not all contrib modules work as advertised. I may spend several hours working with a new module before realizing there's some small issue with it that prevents it from solving my problem.

Similarly, there are often modules out there that solve problems I didn't even know I had, but I'm simply not aware they exist.

What I want is a resource that leverages the experience of thousands of Drupal administrators. I want to know that I shouldn't even bother with a module because it's too buggy. I want to know what modules other users find useful in specific areas (such as multimedia, file handling, cache issues, etc.).

Other large OSS communities often solve this need for a shared knowledge base by providing user reviews. Mozilla, Thunderbird, Joomla and other modular systems provide user reviews and ratings. It is time that the Drupal community have one too.

When I first started using drupal about three years ago, it was not all that difficult to simply be aware of most modules and how effective they were because there just weren't that many. Now there are multiple new modules released on a daily basis, and I just can't keep up anymore.

Why do we, the Drupal community, not have a shared review/rating system? It's certainly not due to lack of demand. A search for "module ratings" on Drupal.org reveals a great deal of interest in this functionality.

As far as I can tell, the primary reason for not having a rating system for modules is fear. Module developers in particular are concerned with the fairness of ratings. They are concerned with "gaming" of ratings. They are concerned that inexperienced or "dumb" end users may unfairly give a bad review of a module simply because they don't understand how to use it. These are all reasonable concerns. But they are concerns shared by other OSS projects as well. Sure you will see "bad" reviews, giving a module the lowest possible rating along with some inane review such as "tis modules sukcs BEWARES" :) But who cares, it's just noise that will be drowned out by valid reviews. It works for other OSS projects, and it can work for Drupal.


John Forsythe
has released what I believe is the first site dedicated to rating and reviewing Drupal modules drupalmodules.com. No doubt this site will be a source of controversy as developers voice their concerns. But we need this resource now.

I encourage my entire audience (hi, mom!) to register at drupalmodules.com and to submit reviews for both your favorite and most hated Drupal contributions. This is a great way for non-techies to contribute to the community. The site is young, and there is naturally a shortage of ratings on the site now, but that will change as the site brings on more users.

Maybe this database will eventually make its way to Drupal.org. For now we can show our support for this type of system by helping build out the database at drupalmodules.com.

Feb 16 2008
Feb 16

The importance of project management tools is almost never fully appreciated. I am shocked at how common it is for a group of developers to go working without version control, ticket tracking, development documentation and so on. The very first thing I do when working with a new client is to make sure that they get these tools in place if they haven't already.

Those who are used to working without a complete set of project management tools never fail to appreciate the benefits of them once they are introduced. I consider it next to impossible for a team to work together without managing code and tasks in an efficient and highly organized way.[img_assist|nid=155|title=|desc=|link=none|align=right|width=250|height=156]

Hopefully you do not need to be sold on this idea and are using CVS or SVN to manage your project already. You likely have some sort of ticket system. It is a little less likely that you have both of these components integrated with each other.

When it comes to choosing a solution for project management software, a die-hard Drupal user has a dilemna. On one hand, Drupal seems as though it should be the perfect solution. It's fully customizable, has lots of nifty project management related modules and, most importantly, it's Drupal! Why would you not use it? "Eating your own dogfood" is the way to go, right? Meh...

Drupal is generally considered a content management system. Personally, I like to refer to it as a website management system. It is great at managing website related stuff like users, posts, permissions, categorization, and so on. Using contrib modules, you can customize and enhance this core functionality to almost no end. But at the end of the day, Drupal is designed to handle web content and the users that are accessing it. That's what a content management system is (and if content is king, that would make Drupal... well... God).

Managing a project, on the other hand, is a much different business from managing a website. Yes, you have many shared properties such as content and users. But the essence of project management involves things that have nothing to do with website management such as a revision controlled code base edited by multiple users, a need for efficient ticket management, and ideally full integration of everything. Essentials also include stuff like a nice repository browser, user management interface for repository access, fancy reporting for tickets, organization of tasks by milestone, date, person, severity, etc...

It's a very tall order. Yes, you can do all this in Drupal, but not very well. You can piece together something that sorta kinda resembles a project management solution, but in the end, you need to invest a relatively large amount of time to create something that is less than ideal and will require ongoing tweaking and modification. Unless your business is creating an effective project management solution in Drupal (something I dream of!), you should not be using Drupal for project management.

I'm a one man shop, and I do not have time to spare. I cannot justify spending any time at all kludging together a project management solution for a client when there are already far superior solutions available at low cost. I would much rather pay someone a few bucks a month and be done with it. Let them deal with SVN administration and enhancements; let me focus on my primary task which is building cool sites with Drupal.

While there are numerous project management related service providers out there (Fogbugz, Basecamp , Beanstalk to name a few), I want to talk about my personal favorite, Unfuddle. Unfuddle has taken obvious inspiration from the folks over at 37signals, innovators of the simple, clean, effective, it-just-works web application. Unfuddle is an instant project management solution that takes minutes to set up and costs a few dollars a month. The time you'll save in not having to set up SVN and manage SVN users alone makes it worth every penny.

[img_assist|nid=156|title=|desc=|link=none|align=left|width=250|height=221]What you get with a solution such as unfuddle is a ready-to-use repository with integrated documentation, ticketing and reporting. It takes seconds to set up a new user account with permission levels fit for everyone from a developer (gimme root!) or a suit (look but don't touch).

From a single interface, you can browse code, tickets and documentation. Every component integrates with the others. You can even resolve a ticket with an SVN commit message, saving you the trouble of having to go and edit the ticket after your commit! Users can individually subscribe to whatever level of email notificaton they would like to recieve and how often. The developer can shut off all notifications while the manager can get a nice daily summary each morning of milestone completion progress, new tickets, added documentation and so on. The project manager can glance over one of the ticket reports and group tickets into milestones for reasonable short vs long term goals.

SVN comments link back to the tickets they are related to. Tickets contain links to the changesets that resolved them. Viewing these changesets, you can see a beautiful code diff and quickly see what fixed the problem. Senior team members can quickly and easily review code changes submitted by junior staff.

With tools like this available these days, it's just not worth it spending any effort whatever on a lesser solution.

Dec 19 2007
Dec 19

Views Bulk OperationsViews bulk operations is ready for some beta testers.

It is essentially the admin nodes page on steroids. It enables you to perform node operations as well as actions from a custom view.

If you want to quickly check it out, install it, then go to the views overview page. Select 'add' for the 'admin_content' view, configure it and save it. When you first go to the view, you will be prompted to enable actions and/or operations.

For the full effect, install the actions module as well.

If you have any problems using it, please submit a bug!

Nov 29 2007
Nov 29

Drupal books are a great way to organize content. Unfortunately, there is no way to control access to individual books by default. Like default forums, it's all or nothing.

The Book Access module adds the ability to set view, edit and delete access control for individual books and all pages therein.  

This module has no official release as it needs to be tested. The development snapshot is ready for testing though.

If you're interested in such functionality, install this module and please submit feedback and bug reports

Thanks to LifeWire for sponsoring development of this module. 

Nov 16 2006
Nov 16

Until the mid 90s, spam was a non-issue. It was exciting to get email. The web was also virtually spam-free. Netizens respected one another and everything was very pleasant. Spam Those days are long gone. Fortunately, there are some pretty amazing tools out there for fighting email spam. I use a combination of SpamAssassin on the server side and Thunderbird (with its wonderful built in junkmail filters) on the desktop. I am sent thousands of spam messages a day that I never see thanks to these tools.

But approximately five years ago, a new type of spam emerged which exploited not email but the web. Among this new wave of abuse, my personal favorite, comment spam.

I love getting comments on my blog. I also like reading comments on other blogs. However, it's not practical to simply allow anyone who wants to leave a comment, as within a very short period of time, blog comments will be overrun with spam generated by scripts that exploit sites with permissive comment privileges. To prevent this, most sites require that you log in to post a comment. But this may be too much to ask of someone who just wants to post a quick comment as they pass through. I often come across blog postings which I would like to contribute to, but I simply don't bother because the site requires me to create an account (which I'd likely only use once) before posting a comment. Not worth it. Another common practice is the use of "captchas" which require a user enter some bit of information to prove they are human and not a script. This works fairly well, however, it is still a hurdle that must be jumped before a user can post a comment. And as I've personally learned, captchas, particularly those that are image based, are prone to problems which may leave users unable to post a comment at all.

As email spam grew, there were various efforts to implement similar types of protection, requiring by the sender to somehow verify he was not a spammer (typically by resending the email with some special text in the subject line). None of these solutions are around anymore because they were just plain annoying. SpamAssassin and other similar tools are now used on most mail servers. Savvy email users will typically have some sort of junkmail filter built into their email client or perhaps as part of an anti-virus package. And spam is much less a nuisance as a result.

What we need for comment spam is a similar solution. One that works without getting in the way of the commenter or causing a lot of work for the blog owner. Turn it on, and it works. I've recently come across just such a solution for blogs which also happens to have a very nice Drupal module so you can quickly and easily put this solution to work on your own Drupal site.

Enter Akismet

It's called Akismet, and it works similarly to junkmail filters. After a comment (or virtually any piece of content) has been submitted, the Akismet module passes it to a server where it is analyzed. Content labeled as potential spam is then saved for review by the site admin and not posted to the blog.

Pricing

Akismet follows my absolute favorite pricing model. It's free for workaday Joes like me and costs money only if you're a large company that will be pumping lots of bits through the service. They realize that most small bloggers are not making any money on their sites, and they price their service accordingly. Very cool.

Installation

In order to use Akismet, you need to obtain a Wordpress API key. I'm not entirely sure why, but it is free and having a collection of API keys is fun. So get one if you have not already.

The Akismet Drupal module is appropriately named Akismet. It's not currently hosted on Drupal.org, but hopefully the author will eventually host it there as that is where most people find their Drupal modules. Instead, you will need to download the Akismet module from the author's own site. The installation process is standard. Unzip the contents into your site's modules directory, go to your admin/modules page and enable it. There is no need for additional Akismet code as all the spam checking is done on Akismet's servers.

Configuration

After installing Akismet, I was immediately impressed at how professional the module is. There were absolutely no problems after installation. Configuration options are powerful and very well explained. The spam queue is very nice and lets you quickly mark content as "ham" (ie not spam) and delete actual spam. As you build up a level of trust with the spam detection, you can configure the module to automatically delete spam after a period of time.

Spam filtering can be enabled on a per node type basis, allowing you to turn off filtering for node types submitted by trusted users (such as bloggers) and on for others (eg forums users). Comment filtering is configured separately.

Another sweet feature is the ability to customize responses to detected spammers. In addition to being able to delay response time by a configureable number of seconds, you can also configure an alternate HTTP response to the client, such as 503 (service unavailable) or 403 (access denied). Nice touch.

One small problem

I've only been working with Akismet for several days now. And I'd previously been using captcha, which I imagine got me out of the spammers sights for a while (spammers seem to spend most of their efforts on sites where their scripts can post content successfully). So far, Akismet has detected 12 spams, 2 of which were not actually spam. These were very short comments, and I imagine Akismet takes the length of the content into consideration. I assume that as the Akismet server processes more and more pieces of content, it will become more accurate in picking out spam versus legitimate content. Each time a piece of flagged content is marked as "ham", it is sent to Akismet where it can help refine their rule sets and make the service more accurate.

Perhaps Akismet could provide an additional option that allows users to increase or decrease tolerance for spam. I would prefer to err on the side of caution and let comments through.

Nov 13 2006
Nov 13

PHP is an interpreted language. This means that each time a PHP generated page is requested, the server must read in the various files needed and "compile" them into something the machine can understand (opcode). A typical Drupal page requires more than a dozen of these bits of code be compiled.

Opcode cache mechanisms preserve this generated code in cache so that it need only be generated a single time to server hundreds or millions of subsequent requests.

Enabling opcode cache will reduce the time it takes to generate a page by up to 90%.

Vroom! PHP is known for its blazing speed. Why would you want to speed up your PHP applications even more? Well, first and foremost is the coolness factor. Next, you'll increase the capacity of your current server(s) many times over, thereby postponing the inevitable need to add new hardware as your site's popularity explodes. Lastly, high bandwidth, low latency visitors to your site who are currently seeing page load times in the 1-2 second range will be shocked to find your vamped up site serving up pages almost instantaneously. After enabling opcode cache on my own server, I saw page loads drop from about 1.5 seconds to as low as 300ms. Now that's good fun the whole family can enjoy.

Opcode Cache Solutions

There are a number of opcode caching solutions. For a rundown on some of them, read this article. After a bit of research and a lot of asking around, I concluded that Eaccelerator was the best choice for me. It's compatible with PHP5, is arguably the most popular of its kind, and is successfully used on sites getting far more traffic than you or I are ever likely to see.

Implementing Eaccelerator

This is the fun and exciting part. Implementing opcode cache is far easier than you might imagine. The only thing you'll need is admin (root) access to your server. If you're in a shared hosting environment, ask your service provider about implementing this feature if it is not in place already. These instructions apply to *nix environments only.

Poor Man's Benchmarking

If you would like to have some before and after numbers to show off to your friends, now is the time to get the 'before' numbers. Ideally, you will have access to a second host on the same local network as your server so that the running of the test does not affect the results. For those of us without such access, we'll just have to run the test on the actual webserver, so don't submit these results in your next whitepaper:

Apache comes with a handy benchmarking tool called "ab". This is what I use for quick and dirty testing. From the command line, simply type in the following:

ab -n 1000 -c 10 http://[YOURSITE.COM]/

Here is a portion of the results I got on my own test:

    Concurrency Level:      10
Time taken for tests:   78.976666 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      13269256 bytes
HTML transferred:       12911899 bytes
Requests per second:    12.66 [#/sec] (mean)
Time per request:       789.767 [ms] (mean)
Time per request:       78.977 [ms] (mean, across all concurrent requests)
Transfer rate:          164.07 [Kbytes/sec] received
Connection Times (ms)
min  mean[+/-sd] median   max
Connect:        0    7  51.3      0     617
Processing:    77  725 1704.4    300   21390
Waiting:        0  673 1697.5    266   21383
Total:         77  732 1706.2    307   21390
Percentage of the requests served within a certain time (ms)
50%    307
66%    468
75%    625
80%    639
90%    805
95%   3808
98%   6876
99%   8529
100%  21390 (longest request)

The single most useful number is 'Requests per second', which in my case was 12.66.

Download, Build and Install

First, download the source code.

Get it to your server and do the following (I'm assuming you have gcc on your system, if not, get it):

tar jxvf  eaccelerator-0.9.5.tar.bz2
cd eaccelerator-0.9.5
phpize
./configure
make
make install

Configure Apache and Restart

If you have an /etc/php.d directory, create the file /etc/php.d/eaccelerator.ini for your new settings. Alternatively, you can put them in your php.ini file. Your configuration should look something like this:

zend_extension="/usr/lib/php/modules/eaccelerator.so"
eaccelerator.shm_size="32"
eaccelerator.cache_dir="/var/cache/eaccelerator"
eaccelerator.enable="1"
eaccelerator.optimizer="1"
eaccelerator.check_mtime="1"
eaccelerator.debug="0"
eaccelerator.filter=""
eaccelerator.shm_max="0"
eaccelerator.shm_ttl="0"
eaccelerator.shm_prune_period="0"
eaccelerator.shm_only="0"
eaccelerator.compress="1"
eaccelerator.compress_level="9"
eaccelerator.log_file = "/var/log/httpd/eaccelerator_log"
; eaccelerator.allowed_admin_path = "/var/www/html/control.php"

Adjust values according to your particular distribution. For more details on configuring eaccelerator, see the settings documentation.

See Eaccelerator in Action

The value eaccelerator.allowed_admin_path, if enabled, should point to a web accessible directory with a copy of 'control.php' (which comes with the eaccelerator source code). Edit this script, changing the username and password. You can then access this control panel and see exactly what eaccelerator is caching

See the results

After enabling Eaccelerator on devbee.com, I ran my benchmark again, and here are the results:

    Concurrency Level:      10
Time taken for tests:   10.472143 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      13129000 bytes
HTML transferred:       12773000 bytes
Requests per second:    95.49 [#/sec] (mean)
Time per request:       104.721 [ms] (mean)
Time per request:       10.472 [ms] (mean, across all concurrent requests)
Transfer rate:          1224.30 [Kbytes/sec] received
Connection Times (ms)
min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       4
Processing:    20  103  52.1     96     345
Waiting:       17   92  50.1     83     342
Total:         20  103  52.1     96     345
Percentage of the requests served within a certain time (ms)
50%     96
66%    122
75%    137
80%    147
90%    176
95%    201
98%    225
99%    248
100%    345 (longest request)

We are now serving up 95.49 requests per second. That's 754% increase in server capacity. Had I been able to run the tests from another machine on the same network, I believe the numbers would be even more dramatic.

Jul 23 2006
Jul 23

The number one quality that separates Drupal from other popular CMS is its API (most often referred to as "the Drupal API).  Drupal is designed explicitly to allow for adding, altering or removing core functionality. Thanks to this API, there are hundreds of third party modules available for Drupal. Some of these modules provide very specialized features. Others provide integration with the most popular services on the web (including Google Maps, Flickr, del.icio.us, Digg and more). All take advantage of the Drupal API and none include modification of core (again, the basic code base required to run Drupal).

Hacking

Enhancing software that doesn't provide an API usually involves modifying its core code directly. If software doesn't open up its functionality to developers, then developers are left to go in and manipulate the original source code to achieve their goals. In many cases, this is just how you have to do things. Drupal is not one of those cases.

To be clear, when I refer to 'hacked Drupal core', I'm referring specifically to modifications of the files that come with the standard distribution of Drupal, most importantly, the files that are in the /includes/ and /modules/ directories. All of the same concepts apply as well to third party modules, but that's not what I'll be focusing on here.

Who Cares?

Does it really make a difference whether you do things correctly as long as they work? Absolutely. While it may seem much more effective at first to edit Drupal core to add the features you want, this is a big mistake. Let's talk about some of the problems you will run into.

Updates

The first problem you'll likely run into is applying Drupal updates. The Drupal team is excellent about patching security vulnerabilities. This means that if you are steadfast in keeping your Drupal instance updated, your chances of getting 'hacked' are greatly reduced. However, if your team has modified Drupal core, applying updates becomes a painful process requiring careful scrutiny of each update and possibly an even more painful merge of those changes with your hacked core files. In my experience, rather than go through this unpleasant process, owners of sites with a hacked core tend to postpone applying patches and updates. The more the owner procrastinates, the more likely his site is to suffer an attack using a known exploit. Once your site has been exploited, there's no telling how long your site may be down or how long it will take you to recover.

Functionality

The next problem you may run into is broken functionality. By altering Drupal core files, you may be inadvertently modifying functionality depended upon by other parts of the system. You are messing around inside the "black box" that Drupal as a whole depends on. While you may think it's clever to go in and modify the phptemplate engine directly, what you could be very well doing is creating bugs somewhere else in your site. By the time you come across the problem, it is unlikely that you'll immediately realize that it is caused by the changes you made to phptemplate. And, friend, you are now in for a lot of hurt as you rip apart code trying to fix it.

Maintainability and Longevity

Drupal's API is known by hundreds of developers all around the globe. The hacks introduced by your $20/hr programmer found on craigslist are known only to one developer. Should you ever need to update or extend your site, you better have that $25/hr developer on staff or you better be using the Drupal API. If you play by the rules, you can hire any experienced Drupal developer.

Bottom Line

There are legitimate reasons to modify Drupal core. If you've found an actual bug in Drupal, the best thing you can do as a developer is to fix it and submit a patch. Likewise, if you've come up with an enhancement that you feel should live in core, submit it. Aside from these two reasons, neither you nor anyone in your employ should be touching anything in that drupal .tgz file.

If you want to develop extended functionality for Drupal, use the API. If you're hiring a Drupal consultant, find one who is familiar with the Drupal API. Find a developer who's active in the Drupal community. Hiring a knowledge Drupal developer may cost more initially, but if you plan on maintaining your site for any length of time, this investment is sure to pay off down the road.

Jul 10 2006
Jul 10

A search for "blank page" of the Drupal.org domain yields about 37,000 results.

This is probably one of the more common and frustrating problems for Drupal admins.

I've had the problem dozens of times myself. For no apparent reason, pages will simply come up blank. Drupal returns nothing at all, not even a set of empty HTML tags. As a developer, I'm used to seeing broken sites and fixing them. But the typical site administrator may not have the knowledge or experience necessary to debug something like a blank page.

The first thing you want to do is gather all available debugging info. The two primary places you'll find this information is in Drupal's logs (Administer >> Logs) and in your Apache log files (/var/log/httpd on Linux systems). If you're lucky, these logs will provide obvious clues as to what's wrong. Whether or not your able to gather any clues as to the source of your problem, below is a list of the most common causes of empty server responses:

  1. Memory: In your php.ini file there is a value called memory_limit, which is set to 8 megabytes by default. If you've installed a lot of modules, you can easily exceed 8 megs on a given page request. The solution is to increase this limit (I set mine to 32 megs because I have lots of memory on my host and few domains being served on it). If you're in a shared host environment, you won't be able to modify this value. In this case, you'll need to disable modules or get a new host.
  2. Buggy Modules: If you're problem began after installing and activating a new module, move the module's directory outside of /ROOT/modules/. Experimenting with modules is probably the easiest way to "break" your site. My suggestion is to test all modules on a non-production Drupal installation. Move them to your production site only after making sure they work properly (This message approved by Captain Obvious).
  3. Incompatible Modules: Using 4.6 modules on a 4.7 site is also a great way to break stuff. Make sure you've installed the module versions that match your Drupal version.
  4. Broken Themes: A buggy theme can easily render a site unusable. My suggestion is to preview themes at the Drupal Theme Garden or on a non-production Drupal installation. If it's too late for that and you're experiencing blank pages, you should try setting the theme to a "safe", stock theme. If you cannot access your administrative pages (http://.../admin/themes), you will need to move your custom theme from your /themes/ directory to another temporary location. Alternatively, you can run this SQL command on your Drupal database: UPDATE variable SET value = 's:10:"bluemarine";' WHERE name = 'theme_default';

I don't believe I've ever seen the 'blank page' problem in a production release of Drupal core. The problem only crops up when installing and playing around with new modules. So the bottom line is be very careful when experimenting with modules. I almost always use a sandbox installation of Drupal to test modules before installing them on a production site. (updated: Thu Jul 13 14:11:27 2006)
Jun 27 2006
Jun 27

The contributions process for non-core Drupal themes and modules is in need of a revamp.

Currently the process is a very loose one that does not require adherence to any tagging conventions or release process. This makes it very difficult to know the status of the modules that make up a given site. I believe this probably results in many site admins out there just leaving their site as-is because the process of updating is somewhat confusing and tedious.

We want to fix this.

While there are existing 4.6, 4.7, etc... tags applied to contrib modules, they are not consistently used by module authors. Furthermore, tags are being applied inappropriately (4.7 tags applied to 4.6 code that is not actually upgraded).

Ultimately, I'd love to see version and update management built right into Drupal (in the same way it's built into most OSs and many software packages). Drupal might have a single page the admin can visit to check the status of his core Drupal installation as well as all modules and themes he is using. The process of upgrading to the latest versions could be a very simple matter of selecting desired updates and hitting a submit button.

That, however, is a long way off. We need to start by creating a process with which Drupal developers and contributors can "tag" their software releases and make those releases available quickly and easily.

To take part in defining this new release process, please visit "Versioning and release process for contributed projects" and leave a comment.

Jun 26 2006
Jun 26

I just returned from BarCamp San Francisco , and I'd have to say it was a success.

This was my first BarCamp, so I had no idea what to expect. It really is as simple as it sounds. A bunch of folks show up and schedule events on the spot based on who is present and what they're interested in.

It took place at Microsoft's offices on Mission and Embarcadero. While normally I'd be inclined at this point to offer one of several heartfelt opinions about Microsoft, I'll just say that it was very generous for them to allow us their location for the weekend, and it was very well suited for this great event. Thanks, Microsoft.

"Take Aways"

One of the unfortunate survivors of the dot-com bust was a vast set of obfuscated vocabulary collectively known as buzzwords, a value-add semantical meta-markup linguistic solution used to define hi-tech marketecture and other domains. Can you feel the synergy?

Yes, there was no shortage of buzzwords floating through the air, and it's hard to avoid them when they are made up largely of otherwise very usable terms such as "solution" (which now means software application), bandwidth (which now means intelligence or ability ["I don't have the bandwidth to..." actually means "I'm not smart enough to..]") and "orthogonal" (which is used in so many disparate contexts, I don't know what to make of it).

Point being that the following section may be populated with buzzwords, so watch your step.

Talent Co-op

This was probably the coolest concept I came across (read: most relevant to me). The idea of bringing together in a single (virtual) place geographically dissimilar supplies and demands is not new. Elance.com, Guru.com, blah, blah. What hasn't been done, to my knowledge, is this sort of thing without the blatant and irritating commercial aspect to it. Something just plain bothers me about relying on some corporate interloper to supply me with suitable clientèle, particularly when they have no reason to take any concern for the quality of either the supplier or the suplyee.

The internet is too obvious an answer to the problem of many people working (or wanting to work) from Location Unknown doing things they love and are good at, and other people wanting to find good worker units and not really caring where, when or how they work as long as they can do the job.

Citizen Agency is the first effort (that I'm aware of) to bring a bunch of talent together to pool talents, resources and work. The effort is only weeks old, and I believe BarCampSF was the official announcement of the effort.

Micro Formats

Here's an example of something I ignored as more buzzword blather that turned out to actually be something potentially very useful. Microformats is like XML in that it's easy to explain and difficult to completely understand. As I see it, microformats would turn any standard web page into a potential source of data objects like contacts, locations, content objects like reviews, and anything else you might normally look for in a database. Main problem, widespread adoption. Microformats probably won't blossom until these formats are managed behind the scenes by blog software, document editors, email and so on. But I can definitely see that happening, at which point you'll see all sorts of useful tools (first appearing as extended functionality to clients like browsers and email apps). Read up a bit on this so when it finally breaks out, you can say something like, yeah, I saw that coming back in 06.

Communities

No need to go into any detail here. Everyone wants to be like MySpace, Flickr, Facebook, etc... I find the subject tiresome myself. The *concept* of community has no meaning or value to me. Implementation of community oriented websites, features and so on is what interests me.

Drupal

While there was a very nice showing from the Drupal camp. I got to meet Drupal luminaries jjeff, dries, drumm, roland and others. Sporting my own Drupal tag, I struck up Drupal related conversations with most of the folks I met. (I'm hereby pronouncing myself as Drupal's chief petty evangelist.) I don't think I met anyone who wasn't aware of Drupal. Most had at least some direct experience with Drupal. Most folks had a lot of questions about it. Some had the common "I installed it, now what" experience with Drupal. I think we might benefit from expanding, somehow, our efforts to help folks (hi sepeck!) install, use and love Drupal.

The BarCamp experience was better than I had anticipated. I highly recommend you attend one when it hits your town. I'm going to attempt to help organize the first BarCamp San Diego.

References

May 03 2006
May 03

One of the great features of Drupal is its ability to run any number of sites from one base installation, a feature generally referred to as multisites . Creating a new site is just a matter of creating a settings.php file and (optionally) a database to go with your new site. That's it. More importantly, there's no need to set up complicated Apache Virtual hosts, which are a wonderful feature of Apache, but can be very tricky and tedious, especially if you're setting up a large number of subsites.

No worries, there is a solution.

Create a new LogFormat

Copy the LogFormat of your choice, prepend the HTTP host field, and give it a name:

LogFormat "%{Host}i %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" vcombined 

 

Get the script

Next, download the attached script (split-logfile) and store it somewhere like /usr/bin (don't for get to chmod 755 that baby!)

Now, tell apache to use pipe logfiles to your script rather than writing them directly to disk:

CustomLog "| /usr/bin/split-logfile" vcombined 

Restart Apache

/etc/rc.d/init.d/httpd restart

That's it.

Naturally, you may have to modify split-logfile if you don't store your logfiles in the default location.

 

 

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web