Jan 06 2019
Jan 06

2018 was a big year for Webform CiviCRM module. I wanted to take a moment to highlight some of the new features that were added in 2018 (with some examples/screenshots) and take a look at what's to come in 2019!

Webform CiviCRM Integration - what is this?

Webform CiviCRM is a Drupal module that in a nutshell exposes CiviCRM APIs (with which you can create CiviCRM contacts, contributions, memberships, participant registrations, activities - just about any CiviCRM Entity programmatically) to the powerful Drupal Webform module - a very popular (over 450,000 Drupal sites are using it) and highly configurable drag and drop form builder. Webform CiviCRM itself is a popular module - over 3,000 CiviCRM projects are using it. That's more users/sites than the Mosaico Extension has! Webform CiviCRM was invented by Coleman Watts (of the CiviCRM Core Team) and is supported by the CiviCRM community:  https://www.drupal.org/project/webform_civicrm

2018 - highlights

is:pr is:closed updated:>2018-01-01 -> 88 closed! Some highlights include:
  • enhancements to recurring contributions via Webform CiviCRM (thanks to Biodynamics Association (USA) for co-funding this) - you can now configure your webform such that you can pay any amount (Event, Membership) in instalments as well as start a regular recurring open-ended Donation. An example of this would be swim club fees -> a full season is 10 months and costs $3,000 for the entire year. You can now configure your webform such that parents can sign up their child(ren) for Memberships/Events -> and select to pay all at once or in e.g. 10 instalments of $300/month.
  • Stripe support (thanks to contributions by Matthew Wire from MJW Consulting) - Matt has been doing a lot of work on the Stripe Extension and we've been supportive of changes he has PR-ed to Webform Civicrm module.  This means that Webform CiviCRM is now compatible with all major in-line Payment Processors: Stripe, iATS Payments and PayPal Pro.
  • being able to configure financial types (thanks to PEMAC (Canada) for funding this) - it is now possible to e.g. charge the correct Sales Tax [which is defined per Financial Type] based on a member's Province/location.

  • Membership Application
  • added line item support (thanks to Wilderness Committee (Canada) for funding this) - it is now possible to add up to 5 additional lineItems for one Contribution. So you can now do things like: make a donation, purchase a calendar, pay for postage - all on the same webform - and in combination with the financial type improvements - you can control the financial types for every line item, ensuring that (in our example) - the donation becomes eligible for Charitable Tax Receipting but the calendar purchase and the postage do not. 
  • numerous improvements re: cases, activities, memberships (thanks to people at Compucorp and Fuzion, and many others)
  • many other improvements - I apologize for missing anything/anyone!
  • Coleman and I ran a 2h sold-out Workshop on Webform CiviCRM at CiviCamp Calgary 2018. We covered lots of features and in-hindsight wished we had recorded it. Next time! Amongst many other items we covered how the Registration form for CiviCamp Calgary 2018 was built (allowing multiple participants to be signed up for multiple events and also including a Partner discount code field).

  • CiviCamp Calgary Registration

Also filed under 2018 highlights: Jacob Rockowitz officially released his Drupal 8 version of Drupal webform module - it includes wicked new features that make webforms more portable than ever and new fields like signature fields and my favourite: automated country flags for phone numbers (see screenshot further down).

Looking ahead at 2019

D8WFC

How can user organizations Contribute?

D8WFC Buff
May 30 2018
May 30

Drupal module - CiviCRM Contact Distance Search

MillerTech released this Drupal module back in 2015 but have recently updated with new features (map and use your location) and to make it more configurable.

This module offers a fully configurable/extendable Drupal view that provides the functionality to search from a postcode and a distance.

Use case scenario – Find schools from my postcode within a 5 mile radius.

With the example above you would have schools as contacts in your CiviCRM database with a primary address and both the latitude and longitude fields should be populated.

The Drupal view that’s shipped with this module can be configured to filter on a particular contact subtype i.e. schools.

Search results will provide you with schools within a 5 mile radius of the entered postcode along with distance.

Distance is calculated by road (or as the road winds or as the crow walks etc.) and NOT as the crow flies.

New features includes an option to display a map –

And also an option for your device to use your location which will populate the postcode field (works best with mobile devices for accuracy) –

CiviCRM extension page - https://civicrm.org/extensions/civicrm-contact-distance-search

Full installation steps available on the Drupal module page - https://www.drupal.org/project/civicrm_contact_distance_search

Apr 20 2018
Apr 20

This is a first blog post about how we build the team portal for Roparun.

But first what is Roparun? The Roparun is a relay race of over 500 kilometres from Paris and Hamburg to Rotterdam, where people in teams, take part in an athletic event to raise money for people with cancer. It’s also called an adventure for life. This is also clear from the motto, which for years has been: ‘Adding life to days, when days often can’t be added to life’.

So each year Roparun organizes this race and around 400 teams participate in the event. The first part of the project was to setup donation functionality and that is working right now.

The next part of the project is to create a new portal for team captains where they can manage their team data, (e.g. name of the team, start location and the individual team members). We have chosen to have this in a separate Drupal website.

In CiviCRM each team captain is registered as a participant to the Roparun event with the role team captain. The team captain can login into the portal as soon as he has been registered as a team captain and till the event is over.

The first part of this project is that we wanted the team captains being able to login and we have created a module called CiviMRF User Sync. This module build on top of the CiviMRF framework.

This user sync module uses the CiviCRM api to create drupal user accounts. See screenshot below for the configuration.

What you can see is that we use a custom api to retrieve the team captains. This custom api returns the email, contact id and the team id of the team captain. We store the e-mail address as the username and at the email field at the user level.

As soon as a new team captain is registered a new user record is created and the team captain receives an e-mail with a link to create a password.

As soon as an existing team captain is removed from CiviCRM the user account is cancelled and the team captain receives an email indicating that his account is disabled.

We have also created a drupal module to store the team id at the drupal user record and use this team id in the view (see https://github.com/CiviCooP/roparun_team_portal)

So the first bit is done, meaning a team captains can log in. The next bit is to build the portal with Drupal Views and Webforms. The building blocks we are going to use for that is CiviMRF Webform, CiviMRF Views and at the CiviCRM site the form-processor. I will keep you posted on the developments of the next steps.

Dec 06 2017
Dec 06

We had a scenario where client runs a cluster of events, and folk sign up for these, and usually the registrants signs up for all events, but then they might invite mum to the Dinner, and brother John to the Talk, etc etc.

We wanted to achieve this on a single form with a single payment. We explored both CiviCart and Drupal Commerce but in the end concluded we could achieve this in a much lighter way with good old webforms.

The outcome is that up to 6 people can be registered for any combination of events, eg 

  • c1 registers for Events A, B, C, D, E and F
  • c2 registers for B, C and D
  • c3 registers for A and B
  • c4 registers for A and F
  • etc

To see the full gory details of the conditionals approach we took, please read the full blog on Fuzion's site.

Nov 13 2017
Nov 13

Today, Skvare has released a new version of CiviCRM Entity, 2.0-beta11.  This release contains a new feature, an admin configuration page which allows site administrators to disable exposure of entity types to Drupal.

CiviCRM Entity is a Drupal module which exposed CiviCRM API entity types as native Drupal entity types, providing Views, Rules, Entity Reference field integration, and Drupal native pages and forms for each. It supports both CiviCRM 4.6 LTS and CiviCRM 4.7.

Previous versions of CiviCRM Entity allowed developers to control access to Drupal based pages and forms for entity types, but there was no way for administrators to control what entity types were available in Views, Rules, or Entity Reference fields. As CiviCRM Entity has evolved over the past 2 years, over 45 entity types have been supported, including all the major financial record types. There are cases where many of these entity types are not used in Views, Rules, etc.., and admins may not want to make data of these entity types available to be used in Views by lower-ranking administrative users.  Disabling an entity type in CiviCRM Entity does not affect the Core Views integration. However it will not make any of the additions that CiviCRM Entity provides, and for types not supported by CiviCRM Core, integration can now easily be toggled on/off.

Having all entity types enabled can affect performance in some aspects. Generally, this does not affect cached page load for normal users, but anytime you clear the cache, or the Views cache, having 45 entity types can cause cache rebuild to be intensive, not to mention all the additional menu paths that are generated for the Admin menu. Disabling entity types that you do not use will streamline admin user performance, and make the site in total that much faster by reducing memory footprint.

For existing users of CiviCRM Entity, the module can be upgraded as per Drupal standard, and there are no necessary config changes to make.  There are updates that need to be run by going to "/update.php" or running "drush updatedb" from the command line. These updates simply set up a configuration variable, and do not affect the CiviCRM database.  All available entity types are enabled by default for new or upgraded installations.

All submodules that are packaged with the CiviCRM Entity will automatically enable entity types that are required by the submodule and will enforce that these entity types remain enabled as long as the submodule is enabled.

Usage

You will find an admin configuration page at "admin/structure/civicrm-entity/settings". A user with a role with 'administer CiviCRM Entity' permission is required to access and manage the settings on this page.

It is important to remember to enable all entity types used by your site's configuration and 3rd party modules. This page does not check if an entity type is required by an existing View, Rule, Entity Reference field. Disabling an entity type will break functionality in any rule, view, or field that requires it, so proceed with caution.  However, re-enabling will restore functionality for those entities.

For developers of 3rd party modules or custom modules making use of CiviCRM Entity, you are responsible for ensuring an entity type is always available. This requires only a hook_form_alter() implementation to disable the necessary configuration page checkbox, or adding a validation or submit handler to the form.

Future

We consider CiviCRM Entity for Drupal 7 to be feature complete, and it has been quite some time since there was a major bug found. We plan to release a non-beta stable 2.0 version at the end of this year. This upcoming stable release will be regarded as a Long Term Support release, and any major changes or updates will move to a 3.x branch. The primary focus of new development will now shift to the Drupal 8 version development. We will continue to support the 7.x-2.x branch throughout the life of Drupal 7 for bug fixes and minor feature updates.  We will support for CiviCRM 4.6 LTS for its lifetime, and most likely beyond.

Oct 24 2017
Oct 24

With the funding environment for nonprofits in Serbia becoming increasingly more fragmented and the choice of technology tools to aid fundraising and advocacy becoming more diverse, Catalyst Balkans saw an opportunity to fill an open niche for a localized CRM targeted to the nonprofit sector in the Western Balkans. With Catalyst Balkans already having used CiviCRM for several years for its own communication and contact management needs, the localization of CiviCRM was a natural choice.

With virtually zero strings translated into Serbian on Transifex and a very limited budget, Catalyst used a combination of existing staff resources and volunteers to plug away at the translation effort over a period of months. The final 1500 strings were done with the help of a translation professional who also went through and polished the entire translation file.

Many coffees were spent in conversation about the best (and shortest) translation of a string.  Concepts like a ‘pledge’ or acronyms like LYBNTY proved to be a huge challenge to get right. And it also gave our staff coffees a whole new linguistic flavor (and made some of us wish we had a little extra nip of something to slip into the coffee).

However, after nearly 7 months of effort, we completed the translation and were thrilled with the results as we installed it onto a Drupal implementation. Then we broke out the drinks and made coffee hour into happy hour.

Subsequently, we have continued with the translation of several extensions, including the Mosaico mail extension. With the translation complete, we have worked with 9 nonprofits to set up instances in Serbian to beta test the translation and provide us feedback on improvements that could be made.

With this experience in hand, we are launching an effort to provide full translations of CiviCRM and key extensions into Albanian, Bosnian, Croatian, and Macedonian over the next year.

This will allow CiviCRM to access a market of more than 130,000 nonprofit organizations across the 7 countries where there will now be a fully localized CRM solution for them to use and a service provider who will provide hosting, support and training in using CiviCRM for improved fundraising, more effective advocacy and increased constituent engagement.

Nathan Koeshall,

Director and Co-Founder of Catalyst Balkans

Sep 15 2017
Sep 15

A big thank you to all our CiviCON UK Sponsors. Here's a special post from Gold Sponsor Yoti:

Doing things differently: Registrations in seconds, logins without passwords and minimising data.

Over the last 15 years I’ve probably been responsible for around 50 or so websites or microsites that in some way or another have tried to gather people’s data.  Either to enter into an event, join a forum or buy something.  And like most other marketeers I’ve been obsessed by two things.  Funnels and Data. i.e how easily are people signing up and how much do I now know about my customers.  I’ve always known that by asking people for more information there was a danger people would drop out of my acquisition funnel but we marketeers are hungry for data. We want it all and we want it now.

I’ve now come to realise that less is more.  I still want the customers, and loads of them, but I want them to join me on their terms. If I ask people less about themselves I’m more likely to get an answer. 

I always try to liken marketing situations with personal situations.  When you first meet someone in a bar you don’t tend to ask them everything about themselves in the first sentence.  You might introduce yourself and ask them their name.  And perhaps ask them where they’re from.  You certainly wouldn’t ask their address or birthday.  If you did, you can be fairly sure they’d either not tell you or they’d give you a fake one and then avoid you for the rest of the night.  So why do we do it in business?  

So we can send them emails they’ll never read?  Send them mail that’ll clog up their recycling bin. Profiling our user base?  Most businesses can’t afford this kind of wastage.  We can’t afford to be sending emails that damage our brand by clogging up people’s inboxes or sending people mail that costs us thousands of pounds.  And just how much effective user profiling can be done if the data they are giving us isn't actually real in the first place?

I want to suggest a revolutionary new idea.  Why not just ask for less, just their first name? Or maybe, not even ask them anything at all. Imagine if you could give them access to a secure user profile on your website even though you didn't even have a name or email address for them.  And they could return to that profile whenever they want to tell you a bit more about themselves. Imagine if you could give them a special key to your world.

A special key?  What like a password?  Surely not...No, not like a password.  Just let them use their phone and their biometrics.  Let them then build their trust with you by adding more details when they want to.  When they need to.  Let them feel special by showing them all the awesome stuff they can now see that nobody else can, and then when you need their address to send them something they've bought you can ask for it.  You could even give them the option of giving you their phone number or email so you can tell them when you’ve got something new and cool to sell. Or you could just say ‘come and stop by whenever you fancy. You’ve got the keys and you can’t lose them so come and have a browse whenever you fancy it.’

It feels a little scary doesn't it? Having anonymous customers that are in control of what information they give you and when.  But there's also all sorts of other interesting stuff it means you could do. You can create chat forums where people just prove they are from a certain city but otherwise remain anonymous.  You could build web forums for children that only allowed people under a certain age or over another age to register.

However, I know there are plenty of us who do actually need all the data up front, just because that’s how business works and it may take a while for my utopian customer-centric world to exist.  And plenty of businesses need to be absolutely sure they have the right name and address to eliminate fraud that’s costing us billions of pounds every year.

But these businesses still want to get people signed up quickly and they’re still scared about getting their data hacked.  And a lot of businesses do end up giving their customers passwords, which their customers usually forget.

We created Yoti to try and let people and businesses do all of the above. We want to encourage businesses to ask for less, but give you the confidence people are who they claim to be.  We want to make it frictionless for you to sign up new verified customers. We don't want anyone to ever hate your website just because they forgot their password.

Yoti is an easy to use digital identity app for consumers and secure digital identity attribute exchange platform for organisations. Our phones help us connect, make payments and board planes, it’s time that they helped us prove our identity.

People create their Yoti by downloading our free app, prove they’re a real person and match their selfie to their passport or driving licence photo. In less than 5 minutes, out pops their verified digital identity that they control for life. Yoti uses AES 256 encryption to store and share user data in such a way that only they have access to their personal details. Yoti cannot see or access any personal information once the accounts have been verified.

Once people have Yoti they can then prove any element of their identity in seconds to businesses and organisations that accept Yoti.  They simply scan a QR code if they are on a desktop, or press ‘Use Yoti’ if on their mobile.  They’ll then be told what information is being requested and they allow the information to be shared.  Registered in seconds.  They can then log back in using their Yoti. No password is required meaning they’ll never forget them and the business need never worry about whether or not they chose a secure password.  

We are building an ecosystem of places where people can use Yoti in their everyday life. They will use Yoti to prove their age at the supermarket self checkout, apply for jobs online, sign up to new web services, prove their age on nights out instead of having to carry their passport.

We’re in our ‘pre-launch’ phase at the moment (we launch in November) but people seem interested.  Over 65,000 people have installed the app already and some great businesses and organisations like Reed, Worldpay and NSPCC are already working with Yoti.

To make it as easy as possible for businesses and organisations to use Yoti we've developed a number of SDKs in seven different coding languages and plugins for Wordpress and Drupal 7, with Joomla and Drupal 8 in development right now.

If you'd like to find out more about Yoti please visit www.yoti.com

Jul 21 2017
Jul 21

CiviCooP and Systopia and Palasthotel have been working together on CiviProxy and CiviProxy. This blog is a round up of what we have achieved in the last couple of days. The first thing we have achieved is that we had fun and a very good work atmosphere. We made long days and made lots of progress.

What are CiviProxy and CiviMcRestFace?

CiviProxy is a script to act as an application firewall for CiviCRM. It could be used to put your civicrm in secure network. CiviProxy is the gatekeeper to which external systems, such as your website, connect (this is for example when a user signs a petition on your website and the website submits this data to your CiviCRM). CiviProxy will make sure the call is from the right place (ip-adress) and is only doing what allowed to do. 

CiviMcRestFace (CiviMRF) is a framework to be used in other systems (such as your external website) to connect to CiviCRM. The framework itself is divided in three parts: the abstract core (CMS/System independent), the core implementation (e.g. a Drupal 7 implementation), and lastly the module who is doing the actual submission (for example the cmrf_webform module which provides the functionality to submit a webform to CiviCRM).

What we have achieved:

  • Completed the documentation on CiviProxy: https://docs.civicrm.org/civiproxy/en/latest
  • Got a working drupal 7 module with CiviMcRestFace:
    • Completed screens for set up connection profiles (you can also provide the connection credentials through your module with an api; so that you can store them somewhere outside the database)
    • Completed screen for the call log (a call is submission to CiviCRM through CiviMcRestFace)
    • Added functionality to queue calls and run them in the background and added functionality to retry failed calls
    • Added a basic webform integration module to submit a webform to the CiviCRM Api
    • Added a rules integration module so that you can perform additional actions when a call succeeds or fails. Probably a use case is when a call fails you want to send the data by e-mail to the CiviCRM Administrator so that he or she can enter the data manually.
    • Added an example module so you can see how you could use the cmrf_core module in your durpal projects
    • Code: https://github.com/CiviMRF/cmrf_core/tree/7.x-dev
  • Got a start with the drupal 8 module for CiviMcRestFace: https://github.com/CiviMRF/cmrf_core/tree/8.x-dev
Jul 17 2017
Jul 17

If you are a Drupal developer coming new to CiviCRM, it can be a bit of a "culture shock" to realize that CiviCRM is not your typical Drupal module.

CiviCRM has a separate and independent evolution and ecosystem, and its standard practices and APIs reflect that. From installation of the module itself, to creating customizations and modifications of its standard behavior, you are entering into a different "world" when you implement and develop client solutions with CiviCRM.

CiviCRM Entity can help a Drupal developer make the transistion by enabling them to use some of the standard Drupal API features they have grown accustommed to, while still providing insight into the data structures and interconnections of CiviCRM.

For people who spend the majority of their time developing in CiviCRM, it can feel the same way, in reverse. For all-day CiviCRM developers, CiviCRM Entity can be an opportunity to better leverage your Drupal CMS for customizations and new features.  So this introduction to Developing with CiviCRM Entity and the Drupal API is for you too.

How's it work?

The premise behind CiviCRM Entity is really quite simple, though its ramications are profound. What CiviCRM Entity does is automate the process of exposing CiviCRM API entities and actions as Drupal entity types. Basically a Drupal entity type is the standard Drupal data model for a database table.  You map metadata to columns, and this provides one consistent way to store, retrieve, and manipulate database table data for Drupal Core and the entire ecosystem of contributed modules that can go with it. Because CiviCRM does not by itself engage its data with Drupal's Entity API, the majority of Drupal modules are not aware of CiviCRM's data and cannot act on it.

CiviCRM Entity implements all the necessay hooks to define the CiviCRM data as Drupal entity types.  It registers the entity types with hook_entity_info(), sets up the metadata with a hook_entity_property_info_alter() implementation, and extends the default Entity API objects and controllers.  Inside the controller responsible for load, save, and delete, instead of using Drupal's standard PDO SQL query operations, CiviCRM API calls are used.  This makes CiviCRM Entity a "remote entity" module, but specifically designed to work with CiviCRM only. 

This bit alone does the most important thing. It makes Drupal think CiviCRM data is Drupal data.  You can attach Drupal fields, to CiviCRM data.  You can use Drupal's entity_metadata_wrapper. All the rest of the code in the module and its submodules dealing with specific integration enhancements is just gravy.

Using the metadata wrapper, we built up one bit of Drupal Form API code, that woks with all the entities, and provides Drupal standard CRUD forms. Now you got Manage Fields and Manage Display pages for each entity type. Now the Rules module will play nice. Now Drupal developers can build cool custom stuff, using their familar tools.

Entity Field Query

EntityFieldQuery is Drupal 7's standard programmatic way to query the database tables exposed as entities. So let's say we want to find all the Home location type addresses for a particular contact. If there are results there will be an array with key of the entity type name containing objects keyed by id.

$contact_id = 3099;
$query = new EntityFieldQuery();
$address_ids = $query->entityCondition('entity_type', 'civicrm_address')
  ->propertyCondition('contact_id', $contact_id)
  ->propertyCondition('location_type_id', 1)
  ->execute();
if (!empty($address_ids['civicrm_address'])) {
// do something
}

Load Drupal Entity objects

Following the example above, we have a query result, and now want to load the entity objects.

if (!empty($address_ids['civicrm_address'])) {
  // entity_load returns an array of entity objects keyed by id
  $address_entities = entity_load('civicrm_address', array_keys($address_ids['civicrm_address']));
 
  // maybe you just want the individual entity objects...
  foreach ($address_ids['civicrm_address'] as $id => $result) {
    $address_entity = entity_load_single('civicrm_address', $id);
    // get the city of the address
    $city = $address_entity->city;
  }
}

Saving entities

Now lets make sure our city in our address has every word capitolized, and save the address. The data as you see it in the CiviCRM admin backend will immediately reflect the changes.

$address_entity->city = ucwords($address_entity->city);
entity_save('civicrm_address', $address_entity);

Deleting Entities

If you want to delete an entity, you can use entity_delete().  Remember that these functions eventually get to the controller, which is a wrapper around CiviCRM API calls.  This matters especially for contacts, because by default deleting contacts sends them to the CiviCRM "trash", instead of completely deleting them.

entity_delete('civicrm_address', $address_entity->id);

The Entity Metadata Wrapper 

If you start getting serious about programmatically manipulating entities, you want to start using the Entity Metadata Wrapper. This object encapsulates all these operations in an object oriented way.  It becomes especially useful when you are manipulating multi-lingual fields. It also can use entity level validation based on the entity metadata for each property of the entity type.  I would encourage its use in favor of manipulating the entity object directly, or using the entity_X functions. The code is much more readable and easier to write, and with validation it is much safer. There is a great article about the benefits of the wrapper which goes into detail.

You can pass the entity_metadata_wrapper function the entity object, or simply the id of the entity, and it will lazy load the object. If all you have is the id to start, no need to load the entity object first.

$address_wrapper = entity_metadata_wrapper('civicrm_address', $address_id);
$city = $address_wrapper->city->value();
if($address_wrapper->city->validate(ucwords($city))) {
  $address_wrapper->city = ucwords($city);
  $address_wrapper->save();
}
// get the updated entity object
$updated_address_entity = $address_wrapper->value();

// nevermind, lets just delete the entity
$address_wrapper->delete();

Custom Rules Action Example

A very practical use case of using the Drupal API for CiviCRM is creating custom Rules conditions or actions. Lets say we want to encapsulate this logic of automatically making the city of an address have uppercase words.  We may want to encapuslate functionality like this and pass it on to our site builders or clients who can use it when they need it. Once you find out how easy it is to create custom Rules actions, you'll have a powerful tool in your toolbox. There's lots of documentation on the web for doing this. 

Lets put this in a little module, I'm calling it civicrm_custom. Create a directory in your sites/all/modules directory named civicrm_custom

To read the rest of the article, and to get the code to build the Rules action, visit the Skvare.com blog.

May 05 2017
May 05

CiviCRM Entity Reference Field is a submodule of the CiviCRM Entity project. One of the many advantages of installing the CiviCRM Entity module is the ability to use Drupal’s Entity Reference module to reference CiviCRM data from nodes, terms, or other entity types. Many people are using the Inline Entity Form module, which provides field widgets that allow you to create, edit, or delete a referenced entity from the parent form.

If you reference CiviCRM contacts via an Entity Reference field and use Inline Entity Form, you’ll often want to include the ability for the user to create or edit subsidiary CiviCRM entity types, such as the email, phone, and address entities. This can get tricky with CiviCRM integration. A regular entity reference field stores a target entity id in a Drupal field table of the Drupal database.  CiviCRM Addresses are stored in the CiviCRM database, and can be created by different types of users and in many different ways. In addition, Drupal and CiviCRM reference data in opposite ways.  Drupal’s field model, “forward references,”  which means the entity stores the ids of the child entities. CiviCRM generally uses “backreferences,” meaning the child entity will store the parent entity’s id.  To make the situation even more interesting, this pattern is not consistently followed in CiviCRM, and you’ll occasionally have a “forward reference”. For example, events store a location block id, and the location block references addresses.

We want the convenient and familiar interface of the Inline Entity Form/Entityreference combination, but we want to use the existing data from the CiviCRM tables, and not store target ids in Drupal field tables, while at the same time being flexible enough to go both ways. We want to make a square peg to fit into a round hole. What we needed was a “remote reference field”.

The Drupal Field API is very powerful, and it allows you to make field types that are disconnected from the standard Drupal field tables. The solution is CiviCRM Entity Reference Field, which adds a new field type that can be added to any CiviCRM entity type.

We’ve successfully used this module for a variety of use cases, including:

  • Referencing emails, addresses, phones from contacts
  • Referencing participants from events
  • Referencing relationships from contacts
  • Referencing activities from contacts
  • Referencing contacts from activities
  • Referencing contacts from relationships

It is also possible to have CiviCRM Entity Reference (CER) fields on entities referenced by CER fields. For example, In CiviCRM, Events reference location blocks, which in turn reference addresses. To edit profiles on Events, you need to reference UFJoin from Events.  The UFJoin entity type needs a UFGroup reference field which needs to reference UFField. That case is especially interesting as that “line of reference” runs both forward and backwards from the UFGroup entity.

That’s the what and the why, to see an example of how to use it, please continue reading this article on Skvare.com.

May 02 2017
May 02

As of CiviCRM Entity 2.0-beta4 the sub module called CiviCRM Entity Price Set Field provides a Drupal field type for the Event entity type.  In this article we’ll review the features of this submodule and discuss how to configure and customize it to fit your needs.

event-edit-financial.png

Event Registration on the Event view page

When configured to display on the Event view pages, this field generates a registration form that supports:

  • Registering multiple Participants
  • Uses the event’s price set and all price fields of any type
  • Pay later or credit card transactions utilizing CiviCRM’s payment processing
  • Profiles
  • Default values for the profile fields corresponding to the logged in user’s contact information
  • Customizable Ajax-fied confirmation and thank you panes
  • Utilizes the event’s settings such as “Is paid event?” etc..
  • Test or Live transactions

Field widget for the Event Edit Form

A “simple” field widget is provided by default for this module.  At the time of this writing, only the first price field can be edited via this widget.  

civicrm-entity-event-view-register.png

Getting Started

CiviCRM Entity and CiviCRM Entity Profile are dependencies for CiviCRM Entity Price Set Field. Go to the Drupal module page and enable all three modules and enable CiviCRM Entity Price Set Field, CiviCRM Entity, and CiviCRM Entity Profile.

Once enabled, you can add the Price Set Field to the Event Entity Type.

  • Go to the Event Manage Fields form at “/admin/structure/civicrm-entity/civicrm_event”
  • Scroll to the “Add New Field” section, enter a Label, and select the ‘CiviCRM Entity Price Set’ field type, for this example select the “Simple -- one price field” widget
  • There’s no special field or field instance settings, so just click save until you’re back to the Manage Fields page
  • Now go to the Manage Display Full Content form at “/admin/structure/civicrm-entity/civicrm_event/display/full
  • Set the new field to display
  • There is a field formatter setting to optionally submit test transactions
  • Pat yourself on the back, you’re setup to take registrations from the Drupal based Event view pages at /civicrm-event/[id]

Please note that the registration form takes into account the different settings on the CiviCRM Event.  For instance it will only enable CC transactions and render a billing block if the “Paid Event?” checkbox is checked. The form conforms to registration start and end dates, only renders if Online Registration is enabled.  The form checks to see if Max Participants has been reached, even when adding additional participants. Additional participants can only be added if “Allow Multiple Registrations” is enabled.

To learn more about registration form, how transactions work with the Contribution API, and how it can be customized, please continue reading on Skvare.com.

Apr 28 2017
Apr 28

Add Rules action Assign Contact to Group
Add Rules action Remove Contact from Group
Add Integration for IM entity
Add Integration for Website entity
Add integration for Contribution Recur
Add Rules event for CiviCRM Price Set Field 'After Successful CC Transaction'
CiviCRM Price Set Field , improved support for price fields with multiple checkboxes
Fix issues with CiviCRM Core Contribution Recur Views integration
Enable CiviCRM Entity Reference field on parent entity 'add', Inline Entity Form Single widget, for Contacts
Expose civicrm_world_region table to Views

Views Relationships:

  • Membership -> Membership Payment
  • Participant -> Participant Payment
  • Participant Payment -> Participant
  • Explicit Relationship from Address -> Country
  • Country -> World Region

Overhaul of CiviCRM Entity Inline, inline entity form widget FAPI generation and handling
Minor bug fixes for CiviCRM Entity, CiviCRM Price Set Field, CiviCRM Entity Discount, CiviCRM Entity Reference Field

Apr 25 2017
Apr 25

CiviCRM Entity is a contributed module for tightly integrating and extending CiviCRM with Drupal. This module exposes CiviCRM API entities as proper Drupal entity types. This is HUGE as it allows you to make CiviCRM data available within your favorite Drupal tools such as Rules, Views, and EntityReference. I’d like to present another advantage of Drupal entity types, and that is Drupal fields.

By enabling CiviCRM Entity, you can add Drupal fields and associate with CiviCRM entity types such as Contacts and Events. In fact, any of the hundreds of Drupal field types can be used with CiviCRM Entity.  You may be asking yourself, “Shouldn’t I use a CiviCRM custom field? Why would you want to use Drupal fields?” The correct answer is, you should choose the right tool for the job.

CiviCRM is great at having the business logic and infrastructure to support event registrations. CiviCRM has price sets, price fields, and custom fields for collecting information from users when they register for events, as well as the logic and structure that goes with payment processing and financial accounting. You would want to use a CiviCRM custom field to collect data for a specific user. This will be helpful because they data can be accessed via Reports.

Drupal, on the other hand, is much better at organizing and presenting content than CiviCRM. There are tons of modules that can be leveraged to give you the functionality you desire. Need a mobile responsive slideshow? Add a Drupal Image field, configure it to work with Flexslider slideshow. Images are perfect example of picking the right tool for the job. The images in the slideshow for an event is not related to any data that someone may need for reporting, so there is no point to make it a custom field in CiviCRM, so let Drupal do what it does well, store and present this kind of data.

What do I get out of the box?

When you install CiviCRM Entity, you get Drupal based view page and edit forms for all exposed entities. We will limit our examples to Events in this article, but the same applies for all entity types. You can create, update, and display CiviCRM Events, and never leave Drupal. The view page has the path “<site_root>/civicrm-event/[id]” where [id] is the event id, and <site_root> is the base url of your Drupal website. The edit form is “/civicrm-event/[id]/edit” and new events can be created at “/civicrm-event/add”.

This view page and edit form are “standard Drupal”, meaning that all fields and properties that are displayed can be managed in typical Drupal fashion.  We highly recommend installing the Display Suite (DS) module, and its submodule Display Suite Forms to take full advantage of the possibilities, as you can configure field groups, layouts, and use the provided Display Suite field formatters for many of CiviCRM’s properties. With DS Forms you can also choose to hide properties from the edit form, although you will be forced to include any API required properties.

For more information on how to add and manage fields and managing the Display with CiviCRM Entity, please continue read the full article on Skvare.com.

Download CiviCRM Entity

Feb 24 2017
Feb 24

In this blog I want to explain the round up we have done around the refactoring of the acl_contact_cache. In the previous sprints we discovered that a lot of the performance was slowed down by the way the acl_contact_cache was used (or rather not used at all). See also the previous blog post: https://civicrm.org/blog/jaapjansma/the-quest-for-performance-improvements-5th-sprint

At the socialist party they have 350.000 contacts and around 300 users who can access civicrm. Most of the users are only allowed to see only the members in their local chapter.

In the previous blog we explained the proof of concept. We now have implemented this proof of concept and the average performance increase was 60%.

We created a table which holds which user has access to which contacts. We then fill this table once in a few hours. See also issue CRM-19934 for the technical implementation of this proof of concept.

Performance increase in the search query

In the next examples we are logged in as a local member who can only see members in the chapter Amersfoort. We then search for persons with the name 'Jan'. And we measure how long the query for searching takes.

The query for presenting the list with letters in the search result looks like

SELECT count(DISTINCT contact_a.id) as rowCount  
FROM civicrm_contact contact_a 
LEFT JOIN civicrm_value_geostelsel geostelsel ON contact_a.id = geostelsel.entity_id  
LEFT JOIN civicrm_membership membership_access ON contact_a.id = membership_access.contact_id  
WHERE  ((((contact_a.sort_name LIKE '%jan%'))))  
AND (contact_a.id = 803832 
  OR ((((
    ( geostelsel.`afdeling` = 806816 OR geostelsel.`regio` = 806816 OR geostelsel.`provincie` = 806816 )
    AND (
      membership_access.membership_type_id IN (1, 2, 3) 
      AND (
        membership_access.status_id IN (1, 2, 3)
        OR (membership_access.status_id = '7' AND (membership_access.end_date >= NOW() - INTERVAL 3 MONTH))
      )
    )
  ) 
  OR contact_a.id = 806816
 )) 
 AND (contact_a.is_deleted = 0)
))
ORDER BY UPPER(LEFT(contact_a.sort_name, 1)) asc;



As you can see that is quite a complicated query and includes details about which members the user is allowed to see.  Only executing this query takes around 0.435 seconds and the reason is that mysql has to check each record in civicrm_contact (which in this case is around 350.000 and growing)

After refactoring the acl cache functionality in CiviCRM Core the query looks different:

SELECT DISTINCT UPPER(LEFT(contact_a.sort_name, 1)) as sort_name  
FROM civicrm_contact contact_a 
INNER JOIN `civicrm_acl_contacts` `civicrm_acl_contacts` ON `civicrm_acl_contacts`.`contact_id` = `contact_a`.`id`  
WHERE  (((( contact_a.sort_name LIKE '%jan%' ))))  
AND  `civicrm_acl_contacts`.`operation_type` = '2' 
AND `civicrm_acl_contacts`.`user_id` = '803832' 
AND `civicrm_acl_contacts`.`domain_id` = '1' 
AND (contact_a.is_deleted = 0)    
ORDER BY UPPER(LEFT(contact_a.sort_name, 1)) asc

The query now takes around 0,022 seconds to run (20 times faster).

Explanation

How does this new functionality works:

1. Every time an ACL restriction is needed in a query civicrm core only does an inner join on the civicrm_acl_contacts table and that is all

2. The inner join is generated in the service 'acl_contact_cache'  that service also checks whether the the civicrm_acl_contacts table need to be updated or not.

3. When an update of civicrm_acl_contacts table is needed depends on the settings under administer --> System Settings --> Misc --> ACL Contact Cache Validity (in minutes)

So how does this look like in code?

Below an example of how you could use the acl_contact_cache service to inject acl logic into your query:

// First get the service from the Civi Container
$aclContactCache = \Civi::service('acl_contact_cache'); // The $aclContactCache is a class based on \Civi\ACL\ContactCacheInterface
// Now get the aclWhere and aclFrom part for our query
$aclWhere = $aclContactCache->getAclWhereClause(CRM_Core_Permission::VIEW, 'contact_a');
$aclFrom = $aclContactCache->getAclJoin(CRM_Core_Permission::VIEW, 'contact_a');

// Now build our query
$sql = "SELECT contact_a.* FROM civicrm_contact contact_a ".$aclFrom." WHERE 1 AND ".$aclWhere;
// That is it now execute our query and handle the output...

The reason we use a service in the Civi Container class is that it is now also quite easy to override this part of core in your own extension.

The \Civi\ACL\ContactCache class has all the logic to for building the ACL queries. Meaning that this class contains the logic to interact with the ACL settings in CiviCRM, with the permissioned relationship etc.. All those settings are taken into account when filling civicrm_acl_contacts table which is per user and per operation once in the three hours.

Jan 27 2017
Jan 27

email imageWhen email was first designed, security was not considered important and up until fairly recently it was still possible to send an email from any address and get away with it.

However, as spam, phishing and spoofing attacks by email have become increasingly common there have been various attempts to make email more secure.  In the last year or so the major providers (AOL, Google, Microsoft etc.) have all seriously tightened their security and authentication requirements for validating and receiving email.  The result of this is that a lot of legitimate email is now being classified as spam or rejected by those providers.  In order to ensure that your email continues to be marked as legitimate and received by these larger providers it is now almost essential that you implement SPF, DKIM and DMARC on your domains otherwise many of your recipients will never see the emails you are sending!

As an implementer of CiviCRM I've had to learn all this pretty quickly...  There's a lot of detailed information out on the web but hopefully this post covers what you actually need to know to make sure your clients emails are delivered and read!

Read the rest of my blog post here.

Jan 26 2017
Jan 26

The last two days we spent another sprint at socialist party to improve performance.  And again we used a guy with database knowledge to analyze our queries. We have optimized a few things so far one of the major areas to improve next was the ACL part of queries. That is the part of the query to decide which contacts you are allowed to see. So at the socialist party they have local chapter administrator who are only allowed to see the active members in their local area.

To decide which contacts you are allowed to see a search query is rebuild to include this ACL part. So there is a join on the membership table and a join on the chapter-structure table and then a where expression is to filter on the chapter and to filter on active memberships. Meaning that the MySQL will scan all records in the table civicrm_contact which at the socialist party is around 350.000.

We have worked on a proof-of-concept solution so far which is to create a table with the user_id and which contact_ids the user is allowed to see. We then do a join on this table resulting that only that subset of contacts is scanned by mysql. Which in fact improved the search query from running a few seconds to just a few milliseconds.

When a user logs in we check how long ago it was when we last updated this table and if it is longer than 24 hours we will delete the entries for that user and re-insert the entries.

We have also reported this at the CiviCRM issue Queue see: https://issues.civicrm.org/jira/browse/CRM-19934

The next thing we are going to work on is to make the proof-of-concept stable and do some refactoring of the core civicrm_acl_contact_cache table.  As this proof-of-concept looks like a bit what civicrm_acl_contact_cache should do but does not do.  Also we want to add a setting in the user interface where site-administrators could set the value for the validity time.

We are also wondering what your thoughts are about our performance quest so far and about our proposed solution.

Jan 09 2017
Jan 09

JMA Consulting is pleased to welcome Jon Goldberg as our new Director of Operations effective today.

After a brief stint as a political organizer, Jon spent 13 years working in various capacities at a non-profit legal organization, primarily in IT.  In 2010 he co-founded Palante Technology Cooperative and started their CiviCRM department, where he worked for 7 years.  Outside of work, Jon can be found engaging in queer community organizing, (dis-)assembling electronics, and training parrots.

"I'm really excited to have Jon join us given his keen appreciation of how to help progressive organizations achieve their missions using CiviCRM. He's got a deep and wide knowledge of CiviCRM. I appreciate how he gives back to the community like through StackExchange, where he is the top ranked CiviCRM contributor," said Joe Murray, President of JMA Consulting and co-author of Using CiviCRM.

Jon's new role will include leading client engagements, developing standardized service offerings, overseeing internal business administration, and continuing to contribute to the CiviCRM ecosystem.

JMA Consulting is a leading provider of CiviCRM services including:

  • CiviCRM for advocacy and donor management,
  • software development for extensions and core CiviCRM,
  • data migration from NationBuilder and other CRMs, and
  • custom integrations with Drupal and Wordpress sites.
Jan 09 2017
Jan 09

Last week we had a fourth sprint to improve CiviCRM performance at the socialist party. 

During this sprint we started with looking at why the screen for adding and editing memberships loaded slow. The issue reported was that it took some time before the end date field jumped from the right side of the screen to the middle of the screen. It turned out that as long as the field was displayed at the right side the screen was still loading.  Timing this gave a time of about 18 seconds before the screen was fully loaded.

We discovered a few causes:

  • The javascript files which the browser had to download are over 2MB in size
  • Every request in CiviCRM was also logged in Google Analytics and in Piwik.
  • The PHP function getGroupsHierarchy in CRM/Contact/BAO/Group.php took around 900ms to execute (see the issue: https://issues.civicrm.org/jira/browse/CRM-19831)
  • On that same screen the custom fields are retrieved with an AJAX request and in each request all contributions linked to the membership are also retrieved.

The getGroupsHierarchy function is performing slow because it needed to loop three times through all groups in the database in this case there are around 2.687 groups. In addition one of the loops uses the php function array_merge to merge the result of the loop. The array_merge function is known for performing slow when an array contains more than 40 elements.

After rewriting the function we reduced the execution time from 900ms to 20ms.  At https://github.com/civicrm/civicrm-core/pull/9633/files you can find the patch of the rewritten code.  In that patch we use for loops and not foreach as we discovered that a for loop is performing a lot faster than a foreach. If we replace the for to a foreach the total time of execution of that function will drop to 2.360 ms a factor of 100 slower.

We have also turned off Google Analytics and Piwik for CiviCRM pages as there is no need for this. Nobody really knew why it was turned on in the first place.

After fixing the getGroupsHierarchy function we reduced the total loading time in the browser from 18  seconds to 10 seconds. Which is still slow!

Read the previous blog posts:

Dec 19 2016
Dec 19

Last week we had our third sprint at the socialist party to improve the performance.  In the previous blogs I have explained what we have done so far. You can read them here and here.

Since the previous sprint a guy with a lot of database knowledge has done some analysis of the queries and he came up with the following observations. Most queries are build in CRM_Contact_BAO_Query class and that class adds a join on the tables civicrm_group_contact and civicrm_group_contact_cache and a where clause with an or on both tables.  See example query below.

SELECT contact_a.id AS contact_id  FROM civicrm_contact contact_a  LEFT JOIN civicrm_group_contact `civicrm_group_contact-2304` ON (contact_a.id = `civicrm_group_contact-2304`.contact_id AND `civicrm_group_contact-2304`.status IN ("Added"))   LEFT JOIN civicrm_group_contact_cache `civicrm_group_contact_cache_2304` ON contact_a.id = `civicrm_group_contact_cache_2304`.contact_id   WHERE  ( ( ( `civicrm_group_contact-2304`.group_id IN ( 2304 ) ) OR ( `civicrm_group_contact_cache_2304`.group_id IN (2304) ) ) )   GROUP BY contact_a.id;

Such a query is performing slow because it is looking up all records in civicrm_contact, all records in civicrm_group_contact_cache and all records civicrm_group_contact. Rebuilding the query to use an IN statement will increase the performance. See the query below:

SELECT contact_a.id AS contact_id  FROM civicrm_contact contact_a 
WHERE contact_a.id IN (SELECT contact_id FROM civicrm_group_contact WHERE group_id IN (2304) AND status IN ("Added"))
OR contact_a.id IN (SELECT contact_id FROM civicrm_group_contact_cache ON groupId IN (2304))
GROUP BY contact_a.id;

Luckily we could do this in an extension because CRM_Contact_BAO_Query provides functionality to change and tweak the query.  The extension we have developed could be found on github (https://github.com/CiviCooP/org.civicoop.groupperformance)

In our test environment the extension has a visible effect on the performance. Searching for contact went down from 15 seconds to 3 seconds. We have also installed the extension in the production environment and we are waiting on feedback from end users whether they experience a noticble increase in performance.

Read more

Dec 01 2016
Dec 01

Three weeks ago I wrote about our quest for performance at the Socialist party. This week we had a follow up sprint and I want to thank you for all the comments on that blog.

During this sprint we have been looking into the direction of the amount of groups (+/- 2.700) and whether the amount of groups slowed down the system. We developed a script for deleting a set of groups from all database tables and we deleted around 2.400 groups from the system and we saw that this had an positive impact on the performance.

Before deleting the groups adding a new group took around 14 seconds. After removing 2.400 groups, adding a new group took around 3 seconds. So that gave us a direction in which we could look for a solution.

We also looked what would happened when we delete all contacts who have not a membership from the database and that also had a positive impact but not as huge as the reducing the amount of groups. The reason we looked into this is that around 200.000 contacts in the system are not members but sympathizers for a specific campaign.

We also had one experienced database guy (who mainly knows Postgres) looking into database tuning; at the moment we don't know what the outcome is of his inspection.

From what we have discover by reducing the groups we have two paths to follow:

  1. Actually reducing the amount of groups in the system
  2. Developing an extension which does functional the same thing as groups but with a better structure underneath and developed with preformance in mind. (no civicrm_group_contact_cache; no need for nesting with multiple parents; no need for smart groups).

Both paths are going to be discussed at the socialist party and in two weeks we have another sprint in which we hope to continue the performance improvements.

Read more

Nov 11 2016
Nov 11

After the socialist party upgraded civicrm to version 4.6 a month ago they are experiencing performance issues. In this blog I will round up our quest for performance improvements. But before that some facts about their installation.

  • +/- 350.000 contacts
  • +/- 300 users who can access civicrm and see members in their local chapter
  • +/- 2700 groups
  • There are several campaign websites linked to CiviCRM and one of their successfully campaigns leads to 7500 new contacts per week in the system
  • Running  on a VPS with 4 CPU cores and 8GB of RAM
  • around 40 active extensions

Since yesterday we have added New Relic as a monitoring tool. With New Relic we can monitor and look back in the history. We can also see all the details in each request. So we can analyze the performance. 

Above is the screenshot of the monitoring of the last 24 hours.  The red/rose squares indicates when the overall performance is poor (more than 2 seconds per request). We also see that MySQL has a big part in the  largest peaks.

The screenshots above shows the slowest queries and the slowest mysql operations. One observation is that it looks like that the MySQL delete statements are slowing the system down.

It is not clear what exactly those delete statements are or what is causing those to be slow. That is one of the questions to look into next.

Another thing we want to look into is the tuning of the MySQL database configuration and we also want to get familiar with New Relic.

Do you have any thoughts? Or did you do any performance improvements you did? We can use any help and we will keep posted about our quest for performance.

Read more

Jul 09 2016
Jul 09

Someone asked me to post this here - so that he can give it a try!

I did a Lightening Talk at CiviCON 2016 showing how you can put your Contacts on a Leaflet Map. It only takes a few minutes to put your CiviCRM Contacts on a Leaflet Map if you're using Drupal. Leaftlet is an open-source JavaScript Library for interactive maps. In addition what's really cool is that you can color the PIN based on the value of a CiviCRM custom field!

I've posted the details in a QA format including some of my slides from my CiviCON Lightening Talk on CiviCRM's StackExchange site:

http://civicrm.stackexchange.com/questions/12862/how-to-put-your-civicrm...

Give it try!

Jul 09 2016
Jul 09

Someone asked me to post this here - so that he can give it a try!

I did a Lightning Talk at CiviCON 2016 showing how you can put your Contacts on a Leaflet Map. It only takes a few minutes to put your CiviCRM Contacts on a Leaflet Map if you're using Drupal. Leaftlet is an open-source JavaScript Library for interactive maps. In addition what's really cool is that you can color the PIN based on the value of a CiviCRM custom field!

I've posted the details in a QA format including some of my slides from my CiviCON Lightning Talk on CiviCRM's StackExchange site:

http://civicrm.stackexchange.com/questions/12862/how-to-put-your-civicrm...

Give it try!

May 27 2016
May 27

Anyone who has tried to login to their CiviCRM database via their phone knows the feeling: utter helplessness. You would even be forgiven for thinking CiviCRM is actively hostile to the small screen.

This initial experience of the un-initiated CiviCRM user on the phone will probably remain until the eventual adoption of the Bootstrap framework (a CSS framework with built-in mobile/responsive elements).

What may be surprising to many, however, is that CiviCRM today is quite mobile friendly in all the important areas. You just have to put the pieces together.

After a long discussion on the Civi Partners list of the myriad ways CiviCRM developers have been integrating Boostrap into their projects, Allan Dixon started a page documenting them.

The Bootstrap page includes both the civicrm_bootstrap theme (nice work CiviCRM team - looks great on mobile!) as well as PTP's own Drupal Bootstrap base theme called civiresponsive

The civiresponsive base theme was developed for PTP's PowerBase project. PowerBase is a hosted CiviCRM database services for political organizing groups - however, it will work for any group that is maintaining their CiviCRM installation in a Drupal site that is separate from the organization's main web site (a highly recommended setup that allows web site and CiviCRM upgrades/general maintenance to happen independently of each other and even by separate people or organizations).

The theme's goal is to facilitate designing Drupal 7 themes that look and feel like an organization's home page while displaying all of the CiviCRM public facing pages (e.g. contribution, event registration, petition, etc.).

This approach gives users a seamless experience going from an organization's home page to the various contribution/event/petition pages.

By using civicresponsive as your base theme, you can quickly develop sub-themes that are already responsive. The only changes you need are to modify the CSS elements to match the look of the original web page.

Here's an example of the only changes we had to make to a recent group's theme:

body { background-color: black; }

/*
* The bootstrap theme displays this toggle when the screen is
* small. However we have nothing in this menu so we never want
* it displayed.
*/
.navbar-toggle { display:none; }

.main-container { background-color: white; margin-top: 0px; }

.footer { background-color: #e7bd15; margin-top: 0px; }

.main-container { padding-top: 20px; padding-left: 40px; padding-right: 40px; padding-bottom: 20px; }

#navbar { background-color: black; margin-left: 0px; border: none; }

#navbar .logo { display: block; float: none !important; margin-left: auto !important; margin-right: auto !important; width: 959px; }

.navbar-header { float: none; }

#block-block-1 img { display: block; margin-left: auto !important; margin-right: auto !important; }

/* The next two lines were changed to center the banner. */
.navbar.container { max-width: none; }
.container { width: 100%; }

Next, we use the Drupal theme key module to specify the URL paths that should use the custom theme.

In addition to using the mobile-friendly theme for public facing CiviCRM pages, we have had a lot of luck designing custom intake forms using webform civicrm. The combination of the mobile theme and the layout control provided by webforms allows for an excellent user experience.

If you don't have a mobile-friendly theme, you can still have a great mobile-friendly contribution page using Simple Donate. This extension allows you to pick one contribution page and display it in a mobile-oriented way (it still exists at the regular, public-facing contribution page - with this extension you have two ways to access it).

While having well-designed public facing web pages is important, let's face it: most people interact with us via the email messages we send.

And, starting a few versions ago, CiviCRM core started shipping sample responsive design email templates, which are quite good. If you are not using an email template tested on mobile devices I would strongly encourage you to adapt one of these core templates to your look and feel. It makes a huge difference.

And, if you are frustrated adapting one of these templates, you might want to checkout the Email Template Builder extension which radically redefines how we can build email messages and templates. It does not yet fit in with the normal CiviMail workflow, but it is an exciting development and one that will hopefully become better integrated in the future.

All of these developments are focused on members of the public. However, there are times when the database admins need to access their database via their phones. The times are rare since most of us don't want to do data entry on the tiny phone keyboard or contend with the long scrolling lists. However, those rare times are well covered in the CiviMobile extension, which provides an excellent mobile interface for searching for contacts, event attendance and even completing surveys. You simply login to your CiviCRM site and add /civicrm/civimobile to the address bar and you are ready to go (you probably want to book mark it for future use).

These ready-to-use projects are all very exciting... but what's to come is even more exciting. If you are a developer, you might be interested in learning how to develop using the new CiviCRM angular tools.

CiviCRM is such a prolific community that it's impossible to keep tabs on everything, so please scroll down to the comments where you are sure to learn about the many mobile CiviCRM projects that didn't mention in this blog post!

Apr 07 2016
Apr 07

This weekend the Skvare team attended TexasCamp, the first state-wide DrupalCamp! The Dallas conference attracted users and developers from all over the state to learn and network. Skvare showed community support with a Gold-level sponsorship. Since CiviCRM and Drupal work so well together, we shared some presentations that touched on the intersection of both.

Texas Camp 2016

Mark Hanna shared information on a new open-source online learning system powered by CiviCRM and Drupal. This week's release of Skvare's Tin Can compliant Learning Record Store is a Drupal-native API that expands Drupal capabilities for flexible, open-source Learning Management Software. This open-source solution can significantly lower the barrier to entry for organizations seeking to implement an online learning program.

View the video that goes with the slides below.

Feel free to play around with the alpha release of Tin Can Learning Record Store code and let us know what you think.

Kate Shaw presented an intro to CiviCRM as software for nonprofits, discussing use cases for managing membership, special events, and crowdfunding.

Peter Petrik shared information on hosting and performance improvements and Personal URL's.

Thank you to LevelTen Interactive for organizing and hosting, and to our fellow sponsors, Pantheon, Amazee Labs and Acquia.

If you'd like to learn more about CiviCRM, now's your chance

Register to win a free ticket to CiviCon 2016 by April 15. Sessions for CiviCon 2016 are taking shape. Whether you're new to CiviCRM or a seasoned user there is something for you at CiviCon Colorado 2016! If you’re an active user of CiviCRM but lack the resources to attend, here’s one barrier that’s been removed to help get you to Fort Collins this June. Head on over to our registration page to enter the drawing!

Apr 05 2016
Apr 05

Girls on the Run's mission is to inspire girls to be joyful, healthy and confident using a fun, experience-based curriculum. Teams are paired with coaches who teach valuable life skills over a 24-lesson curriculum that culminates with a 5K run at the end of each season. Every 5K event requires the registration and coordination of thousands of team coaches and over 800 volunteers. Also, as part of sharing their strengths with the community, each Girls on the Run team plans and implements a community service project, which allows them to discover the unimaginable strength that comes from helping others.

With so many volunteers and activities, Girls on the Run St. Louis had a few big questions to solve:

  1. How could they migrate their data away from DonorPerfect, a CRM that was becoming increasingly cumbersome and cost prohibitive the larger their organization grew?

  2. How could they integrate their donation, volunteer and event forms into their website to provide a more seamless user experience for their constituents?

  3. How could they free up their volunteers’ and coaches’ time to spend more time helping the organization with its mission?

  4. How could they gain better insights into volunteer and donor activities with a centralized database?

Girls on the Run St. Louis turned to Spry Digital for help. Right away, Spry Digital understood that GOTRSTL needed a robust, open-source constituent management system. One that would centralize their data to provide better insights into the activities of thousands of volunteers and donors. One that would not increase in cost based on the number of records they added. One that could be integrated with their front-end website, provide forms and make it easier for site visitors to donate. Spry Digital recommended CiviCRM to solve for all of these concerns and more. After evaluating numerous solutions, GOTRSTL decided on CiviCRM due to its flexibility and cost. As Maggie Siemers, Operations Director, puts it, “The price was very enticing.”

Spry Digital implemented CiviCRM in two phases between 2013 and 2014. While work was underway to plan Girls on the Run St. Louis’ new website, Spry Digital setup CiviCRM (with Drupal) as a standalone application and began migrating their content. Instantly, GOTRSTL saw the benefits of having a central database and began improving efficiencies within the organization while gaining insights into their donor and volunteers activities.


With their data and activity tracking now in CiviCRM, Spry Digital designed and developed an entirely new website, using Drupal 7 with CiviCRM powering donations, events, and volunteer registration and management.

HOW ARE THEY DOING?

With an integrated Drupal/CiviCRM solution and seamless user experience, the organization saw vast improvements to their fundraising and volunteer management efforts. Girls on the Run St. Louis were able to streamline operational efficiencies while growing the number of volunteers and coaches that they could effectively manage.

Over the next few years, Girls on the Run witnessed the extraordinary payoff using CiviCRM:

  • Between 2013 and 2014 (before the launch of the new public website, using CiviCRM only) GOTRSTL increased donations by 46%

  • Between 2014 and 2015 (with a new website with integrated Drupal/CiviCRM solution) they saw their overall donations increase by 106% with a revenue increase of 64%

  • By Fall, 2015, GOTRSTL managed the memberships of 935 coaches followed by 1352 in the Spring of 2016

  • GOTRSTL is now managing 2000 coaches and 800 volunteers for every 5K race

  • An overall increase in donations and grants of 70% since implementing CiviCRM

Girls on the Run St. Louis continue to explore and test the features of CiviCRM to see how they can make better use of the platform while they continue to grow their organization. CiviCRM has allowed GOTRSTL to grow with improved processes and better insights while limiting their operational costs.
 

Mar 29 2016
Mar 29

There are two powerful modules used in the Drupal world for creating fast custom searches. Search API is a framework which provides an interface for site builders to create custom searches on any entity known to Drupal. It supports several search backends, including Apache Solr and native database search. It has a flexible API so developers can easily extend, customize, and alter aspects of the search process. Many additional contrib modules are available. Using these techniques, searching millions of records becomes fast and efficient.

The Facet API module allows site builders to easily create and manage faceted search interfaces. In addition to the UI components that standard, themers and module developers can build their own widgets, or alter the out-of-the-box widgets.

This one-two punch of powerful modules brings extremely powerful, flexible, and as you will see, easy to use custom search functionality to Drupal. Oftentimes all that is required is configuration through the admin interface, although the development API is extensive and the imagination is the limit for what can be done.

Creating custom searches for CiviCRM has always been a developer task, and the facilities are limited from both a site builder and a developer perspective. Wouldn't it be great to use Drupal's powerful search modules with CiviCRM data? Well folks, you can, with a little help from CiviCRM Entity.

CiviCRM Entity is a Drupal module which exposes many CiviCRM entities as true Drupal entities. That means that almost any module that can use Drupal entities, can access and manipulate CiviCRM data, Drupal style. This includes many commonly used modules such as Views, Rules, Entity ReferenceEntityqueue, and many more.

Create a Custom Search and Display

In this article I'll outline the basic instructions and methods that can be used create custom searches on nearly 20 different CiviCRM entities. As an example, we'll build a custom Contact search, with a faceted block interface for Contact Type. This is the simplest example of what can be done. There are tons of options and configurations -- too many to outline fully in one blog article.

We'll be using native database search indexing in this article, but this methods applies to Apache Solr as well as several other available backend service classes.

The interface for the search will be created using blocks and a view, which will integrate into your theme. If your theme is responsive, then your custom search will be, too!

Step-by-Step

The full step by step instructions are detailed on Skvare's blog.

  1. Install and Configure System Requirements
  2. Create Search Server Instance
  3. Create a Contact Search Index Instance
  4. Select Fields to Index
  5. Configure Filters
  6. Facets
  7. Index the Contacts
  8. Create the Search View
  9. Configure View fields and Filters
  10. Contact Type Facet Block
  11. The Results

Conclusion

You can have multiple facets per view, and using one of the many contrib modules that extend these modules, there is hardly a limit to what you can achieve. All of this can be achieved with zero additional code, pure site building. There is a strong developer api for customizing and extending this functionality even further.

Also, there are no CiviCRM permissions stopping users from visiting the view. You can control permissions with standard Views permissions. If you want to expose CiviCRM data to users, but not give those users any permission to access the CiviCRM backend, then it can be so. If your theme is responsive, then your search view can be, too.

This is a very simple example of what is possible, and I hope you can see the vast potential involved with these modules and techniques. Really the limit is your imagination.

Have fun building your own specific implementations!

Skvare is a leader in Drupal and CiviCRM integration. Contact us to get questions answered and for consultations to make the most of your Drupal and CiviCRM installation.

Mar 18 2016
Mar 18

It's becoming a common request from our clients to find user-friendly ways to integrate CiviCRM data with the rest of their Drupal website functionality. Oftentimes content creators without direct user access to CiviCRM need to do simple things, such as create, update, and delete contacts in simple, specific ways. 

Example Use Case

A hypothetical organization advertises various community service projects that they organize and coordinate. Each service project can have it's own page, created by adding a Project content type to display a description, images, videos, slideshows or other information for each project. You'll probably use a View to show multiple Project listings on a page. All that is standard Drupal site building content and functionality. No problem.

But what if the organization wants to display to the user the Project Coordinator(s), which they also want to store in CiviCRM as contact(s)? Their Drupal website content creators need to be able to select, create, and update CiviCRM contacts for the coordinator on the Project node edit form in a simple, user friendly way. The website content creators are not given access to the CiviCRM back end. The organization wants the form to be styled like the rest of the Drupal site and be fully responsive.

Challenge

You have CiviCRM data that you need to use with Drupal's standard field and node display and theme structure. What to do?

The Solution

CiviCRM Entity to the rescue!

CiviCRM Entity is a Drupal module which exposes many CiviCRM entities as true Drupal entities. That means that almost any module that can use Drupal entities can also access and manipulate CiviCRM data, Drupal style. This includes many commonly used modules such as Views, Rules, Search APIEntityqueue, and many more.

You can use a standard Drupal Entity Reference field and integrate nearly 30 different CiviCRM entities in the standard Drupal way, into any of your node or entity infrastructures. Contacts, Addresses, Emails, Phone numbers, Events, Contributions, Memberships, Activities, and many more CiviCRM entities can be referenced in a consistent manner.

System Requirements

Drupal 7 and CiviCRM 4.4 or higher 

Install the following Drupal modules and their dependencies:

How To Guide

Read the full set of instructions and screenshots on how to create such an interface for this very real use case on Skvare's blog.

After following the instructions content creators will be able to select, create, update, or delete CiviCRM contacts with ease and assign them as Project Coordinator(s) for the Project nodes! These content creators need not ever see the CiviCRM backend. You can follow the same procedure for CiviCRM Events, and nearly 30 other entities. Cool, yeah?

Have fun with your own specific implementations!

Who are We?

Skvare is a leader in Drupal and CiviCRM integration and project lead for CiviCRM Entity. Contact us to get questions answered and for consultations to make the most of your Drupal and CiviCRM installation.

Mar 15 2016
Mar 15

Hello everybody. I am assessing the feasibility of a CiviCRM integrated 'Community Shares' extension on behalf of my client, the Kindling Trust.

Community Shares are an investment mechanism available to Co-operative and Community Benefit Societies in the UK - see CommunityShares.org.uk if you want more info.

Kindling are a not for profit organisation based in Northern England, who have been using Civi for a number of years, and are planning on launching a community share offer in the future, in order to raise funds for a community owned farm, to develop sustainable food practices, and regional food security.

I anticipate that the new extension would build upon the existing CiviContribute and CiviMember components, and have attached an outline document describing how I would see it functioning.

We are interested in talking to members of the Civi partner ecosystem, who are interested in assisting us with developing the Extension, and providing estimates for the cost of development. We are currently at the stage of raising finances to undertake the work. Please email me here if you are interested.

Mar 10 2016
Mar 10

CiviCRM Entity is a Drupal module which greatly enhances CiviCRM integration with Drupal. This module exposes many CiviCRM entities as true Drupal entities. That means that almost any module can use Drupal entities. As a result, these modules can access and manipulate CiviCRM data directly from within Drupal via Drupal’s Entity API. This includes many commonly used modules such as Views, Rules, Search API, Entityqueue, Entity Reference, and many more.

CiviCRM Entity was originally developed by Eileen McNaughton from Fuzion. Skvare was an early adopter of version 1.0 where we implemented the Rules integration and quickly realized that this was a powerful and essential tool for most CiviCRM implementations that integrate with Drupal. We reached out and collaborated on the development towards the 2.0 branch, with features that were important to Skvare’s clients.

Released last year, the 2.0 branch of CiviCRM Entity has many new features including:

  • almost 20 CiviCRM entities, such as Contact, Membership, Event, Contributions, exposed as Drupal entities

  • Native Drupal create/edit/delete forms for CiviCRM data

  • Native Drupal view pages of CiviCRM data

  • Standard Drupal Manage Fields and Manage Display forms

  • Ability to add Drupal fields to CiviCRM data

  • Display Suite integration

Many of the CiviCRM Entity features are being actively used with large-scale CiviCRM deployments such as the Salvation Army Echelon or the University of Minnesota Crowdfunding Platform.

Future Development

Skvare is now an active developer and maintainer of the project. We are in the process of coordinating feature requests that will be added to the 3.0 branch.

We use, build, and develop solutions with CiviCRM Entity on most of our client websites, and we’ve gotten a lot of feedback. We also keep our ear to the ground for how other people are using it, and this blog article is part of that. The feedback we’ve gotten is, “this is awesome, can I have more, and better!”

What clients now see, love, and want more of, is the ability to create Drupal native themed, responsive view and edit forms. No more figuring out where to edit that one line of custom css code in a CiviCRM-specific template. No secondary system interface for end-users to learn. It’s as simple as building a Drupal webform and concurrently more powerful than the out-of-the-box CiviCRM features. All accessible as a site-building task without the need for any custom code.

A common client request is for volunteers and employees for a CiviCRM form create or edit CiviCRM data without user access to the CiviCRM backend. Oftentimes these volunteers are updating data such as participant attendance statuses at events, with a mobile phone or a tablet. Similar to the built-in profiles, but with consistent theming, out of the box responsive design, broader data access, and much tighter integration of the underlying business logic.

This year, Skvare is excited about adding more features and further improving the integration between CiviCRM and Drupal.  We have many ideas, we have many requests, and as we look into the future, we see this module being one of the most important CiviCRM-related modules for both implementors and end users.

Proposed Features include:

  • Improve Forms with better validation, widgets and formatters for CiviCRM FK reference fields, improved metadata, inline entity forms for FK reference fields, default layouts

  • Better custom field support -- be able to edit multivalued groups, and organize them on edit forms and in Rules in their field groups.

  • Open up more CiviCRM entities, such as Country, CustomValue, and those provided by Extensions, as Drupal entities

  • Improved Views support -- field, filter, sort, relationship handlers for all entities not exposed by CiviCRM Core

  • CiviCRM Profile Integration -- make them entities!

  • Native Drupal Contribution pages

  • Native Drupal Event registration pages

  • Native Drupal CiviCRM configuration pages

  • Deeper Drupal Commerce Integration, allow contributions, memberships, and event registrations to happen with standard Drupal Commerce workflow

We’ve seen the impact and success in real-world implementations. We’re motivated and  want to solicit feedback from the community.

  • Have you used the module before?

  • In what configuration?

  • Do you like a specific feature?

  • Are there specific features you wish the module had?

  • What ideas for the future do you have?

How can you help?

Try it out.

Give feedback on this blog article.

Join the discussion in the Future Development issue

Develop yourself, fork Eileen’s github repo.

Hire Skvare to add the specific feature you want, now. We’ll put it back into the general release and continue building more functionality. Clients have already committed funds to further extend this project. Those who fund development guide the priorities, milestones, and timelines.

Talk to Mark Hanna, CiviCRM Entity maintainer and new development lead.

Want to see a Drupal 8 version sooner rather than later, contact us to make it happen.


Be on the lookout for a set of blog articles showcasing features and site building techniques that Skvare has implemented, and the use cases that we’ve gathered as we aim to further extend this module. We’ll also be presenting on this topic at the upcoming CiviCon 2016.

Nov 18 2015
Nov 18

We in NFPservies have some client using secure.Worldpay as their payment processor and it was always an issue in making it work and now worldpay has introduced online.worldpay gateway with API reference which are easy and quicker than before

Earlier in secure.worldpay gateway we use to redirect from CiviCRM to secure.worldpay website and process all the card details and then it will be passed over to CiviCRM website and we process the details. But, In Online.Worldpay we are using API for creating Tokens and Card Authorize. so no more redirection also the processes is straight forward and quicker

REQUIREMENT

This Extension is based on Online.Worldpay.com - https://online.worldpay.com/

Please make sure you have an account with https://online.worldpay.com/

INSTALLATION INSTRUCTIONS

To install the uk.co.nfpservice.module.worldpay, move the uk.co.nfpservice.module.worldpay directory to your civicrm Custom extension folder directory and Install the extension.

HOW TO USE

Please obtain the service key and client key from your online.worldpay.com account setting

This extension uses https://api.worldpay.com/v1/ api for accessing the website. This link might change from time to time so please make sure you are using the correct URL given by your Online Worldpay Account.

For more documentation regarding the API please refer to https://online.worldpay.com/docs

FUTURE REQUIREMENT

  1. Recurring Payment.
  2. Webhook Integration.


CURRENT RELEASE

This module was developed based on CiviCRM 4.6 and Drupal 7.x. - hasn't been tested in any other version.
Please give a test and if you want to use it in lower or higher version. Please feel free to contact me if you have any issues. You can find the contact details at the footer of this document.

Currently tested in CiviCRM 4.6 / Drupal 7.x with Webform CiviCRM Integration 7.x - 7.14
 

DOWNLOAD URL

you can download the Extension from CiviCRM Extension folder or from GitHub https://github.com/rameshrr99/uk.co.nfpservice.onlineworldpay

Nov 18 2015
Nov 18

We in NFPservies have some client using secure.worldpay as their payment processor and it was always an issue in making it work and now worldpay has introduced online.worldpay gateway with API reference which are easy and quicker than before

Earlier in secure.worldpay gateway we used to redirect from CiviCRM to secure.worldpay website and process all the card details and then it will be passed over to CiviCRM website and we process the details. But, in online.worldpay we are using API for creating Tokens and Card Authorize. so no more redirection. Also the processes is straight forward and quicker

REQUIREMENT

This Extension is based on online.worldpay.com - https://online.worldpay.com/

Please make sure you have an account with https://online.worldpay.com/

INSTALLATION INSTRUCTIONS

To install the uk.co.nfpservice.module.worldpay, move the uk.co.nfpservice.module.worldpay directory to your civicrm Custom extension folder directory and Install the extension.

HOW TO USE

Please obtain the service key and client key from your online.worldpay.com account setting

This extension uses https://api.worldpay.com/v1/ api for accessing the website. This link might change from time to time so please make sure you are using the correct URL given by your Online Worldpay Account.

For more documentation regarding the API please refer to https://online.worldpay.com/docs

FUTURE REQUIREMENT

  1. Recurring Payment.
  2. Webhook Integration.


CURRENT RELEASE

This module was developed based on CiviCRM 4.6 and Drupal 7.x. - hasn't been tested in any other version.
Please give a test and if you want to use it in lower or higher version. Please feel free to contact me if you have any issues. You can find the contact details at the footer of this document.

Currently tested in CiviCRM 4.6 / Drupal 7.x with Webform CiviCRM Integration 7.x - 7.14
 

DOWNLOAD URL

you can download the Extension from CiviCRM Extension folder or from GitHub https://github.com/rameshrr99/uk.co.nfpservice.onlineworldpay

Oct 19 2015
Oct 19

Salvation Army Echelon, the official young adult auxiliary of The Salvation Army, is the product of an idea that began in Dallas, Texas in 2010. Echelon is the next generation of community action for the Salvation Army. Men and women age 21 to 35 engage in networking, fundraising and community service in chapters across the U.S.

After the network of chapters expanded from one club in Dallas to several clubs across the nation, Salvation Army Echelon wanted a solution to house the chapters under one umbrella while allowing each club to customize their own site. They chose Skvare based on their expertise in CiviCRM and the added benefit of having all web-related services - web development, hosting and maintenance - provided by one company.

Angie Harmon, Chair of PR & Communications of the Echelon Peoria Advisory Board says,

“It’s enabled us to gather attendees and volunteers for our meetings and service events. The biggest benefit so far is probably the ability to allow people to register through the website for our paid events. Before this, we were using individual event pages and each chapter was using a different third-party provider.”

Salvation Army Echelon Dallas Fort Worth

Features

One of the primary goals for Echelon was to give each chapter the ability to share unique content in a sophisticated and accessible way while keeping the Salvation Army branding consistent throughout. It was important to make website changes simple for chapter members who have minimal technical experience. In addition, each chapter website needed to perform a variety of tasks, including:

  • track memberships,
  • send out e-newsletters to members or donors,
  • set up events and discount codes,
  • recruit, schedule and track volunteers, and
  • show a calendar of events, meetings and volunteer opportunities.

Permissions Set for Success

One of the main challenges was to give each chapter full control over their local membership information. Skvare’s developers created groups and modified the ACL’s (Access Control Lists) to segment the access, editing, and deletion of chapter-specific contacts. Reporting benefits from this setup, too: though the queries are shared, the reports are configured to only display results for the local chapter. This protects member data from other locations but provides a predictable, repeatable process across chapters, saving time and overhead costs. Skvare also delivered a custom solution for Salvation Army Echelon members to apply their member discount at the time of event registration.

The new website automates many administrative tasks, combining the power of a central system with localized options for each chapter. Ongoing training, website maintenance and hosting from Skvare leave Salvation Army Echelon members free to do important community work.

To learn more about this project or other CiviCRM solutions, please visit Skvare

Aug 28 2015
Aug 28
The Sustainable Furnishings Council is an educational and marketing organization, promoting healthy environments, inside and outside.  The council promotes sustainable practices among furniture manufacturers, retailers, and consumers alike. 

As we're seeing with many of our member driven clients, the Sustainable Furnishings Council wanted to create a visually stunning, mobile friendly website emphasizing member promotion as a way to add exposure and value to their members.    Member promotion drives value because the organization's website often has relatively high traffic compared to an individual member website and thus acts as an important referral source.  To do this effectively,  choosing the right combination of Website tool (CMS) and backend database (CMS/AMS) is essential.  Almost always, the CiviCRM/Drupal combination is an excellent tool set choice because of their powerful integration.  

CiviCRM has good integration with WordPress and Joomla!, but the Drupal integration is particularly strong because of the deep integration with Drupal Views, webforms, roles, and permissions. (Click here to read more)  

For the Sustainable Furnishings Council, Views integration was extensively utilized.  All of the elements in the membership search, search results, and member detail pages including name, description, logo, and member level are stored in CiviCRM.    With all of their critical data in one place, The Council's staff can utilize this data in CiviCRM to create groups, send mass emails, and manage their organization without having to worry about doing any double entry of data in a separate member directory or use a process to move the data to website.  

[]

For more informtion please contact:

Paul Keogan
Principal and Founder of BackOffice Thinking
[email protected]

Jul 30 2015
Jul 30

Cividesk is one of the leading CiviCRM service providers. Giving Back is a cornerstone of our company culture, and we proudly support many charitable and humanitarian organizations with pro-bono or reduced-cost services. While our Giving back program is usually full, we still have a few Drupal openings this summer and would therefore be glad to support nonprofits that need Drupal work with pro-bono services.

Criteria for being part of the program are simple: being a primarily volunteer-run non-profit organization providing free and secular services to the underprivileged (every word is important!).

If your organization fits these criteria, please drop us a line at [email protected] with your wishes and we might turn them into reality!

Jul 21 2015
Jul 21

Drupal 8 is likely to be released around September. Fuzion have been driving the initiative to have CiviCRM ready to roll when Drupal 8 is released and there have already been some generous sponsors for our work to help get us this far. We have CiviCRM currently working on latest beta, and old friends like Views are working fine and you can give it a try out at http://civid8.fudev.co.nz/.

But there is still lots to do to get Rules, Entities and Webform working. A Make It Happen has just been launched, and you can help progress this work by chipping in here https://civicrm.org/civicrm/contribute/transact?reset=1&id=71, or you can get in touch with us about setting you up your own test suite.

If you are happy trying out your own installation, details can be found here https://civicrm.org/blogs/torrance/give-civicrm-drupal-8-test-out

Jun 10 2015
Jun 10

This has been my approach (together with CiviCoop) to load test a big site with CiviCRM where most visitors where expected to login.
Let me know if you would agree with this approach or if you have a better alternative.

Every big drupal site needs load testing before going live.

These are the key questions you should have answered in the final stages before deployment:

  • How does your infrastructure handle the expected amount of visitors?
  • how does it perform with maximum amount of visitors?
  • and at what amount of visitors does it start to crumble?

For anonymous load testing there are a number of tools available.
For logged in users there are not so many available.

But what about the sites where logging in is secured using unique tokens per user visit like drupal?

How do you load test those?

The problem is that you often can record or script what you need to post during login.

But sites like drupal secure their login pages with an unique token so you do not know what you will need to post beforehand.

With LoadImpact.com that problem is solvable.

LoadImpact automates load testing and gives graphs like:

Example of LoadImpact graphs

To setup a load test on Load Impact for logged in users you can do:

Step 1: Record one or more user scenario's with the Load Impact chrome plugin: https://chrome.google.com/webstore/detail/load-impact-user-scenario/comn...

Step 2: Export user scenario to Load Impact.

Step 3: Look into the generated LUA code and find the GET request to the login page.

Step 4: Change it so the form token is gathered and placed in a variable:

For example:

http.page_start("Page 1")
local pages = http.request_batch({
    {"GET", "https://www.domain.com/user", response_body_bytes=10240}
})

local body = pages[1]['body']
local token = string.match(body, 'input type="hidden" name="form_build_id" value="(.-)"')

Step 5: find the POST request to the drupal login page and change the "form_build_id" with the token value.

if token ~= nil then
http.request_batch({
    {"POST", "https://www.domain.com/user", headers={["Content-Type"]="application/x-www-form-urlencoded"}, data="form_build_id=" .. token .. "&form_id=user_login&name=<username>op=Log%20in&pass=<password>", auto_decompress=true}
})
else
  log.error("failed to find token" .. body .. "");
end

And you're done. Now load tests can be performed with thousands of concurrent logged in users on your drupal site.

If your user scenario contains other form submissions you can repeat this for the other forms as well.

Using CiviCRM as an example: someting similar is needed if CiviCRM searches are performed.

CiviCRM adds a session dependent qfKey to every search. Without the right qfKey a search will not be executed properly, harming the load test.

To solve this you have to execute the following steps in the Load Impact user scenario.

Step 1: Find the GET page for the search and place the qfKey in a variable

local pages = http.request_batch({
    {"GET", "https://www.domain.com/civicrm/contact/search?reset=1", response_body_bytes=102400}
})

local body = pages[1]['body']
local token = string.match(body, 'input type="hidden" name="qfKey" value="(.-)"')

Step 2: find the POST request to the search page and replace the qfKey with the token

if token ~= nil then
http.page_start("Page 5")
http.request_batch({
    {"POST", "https://www.domain.com/civicrm/contact/search", headers={["Content-Type"]="application/x-www-form-urlencoded"}, data="_qf_Basic_refresh=Search&_qf_default=Basic%3Arefresh&contact_type=&entryURL=https%3A%2F%2Fwww.domain.com%2Fcivicrm%2Fcontact%2Fsearch%3Freset%3D1&group=&qfKey=" .. token .. "&sort_name=&tag=", auto_decompress=true}
})

http.page_end("Page 5")
else
  log.error("failed to find token" .. body .. "");
end

And you can also do proper CiviCRM searches in your Load Impact user scenario and load test your Drupal+CiviCRM site before deployment.

Originally posted on http://orgis.com/en/blog/web/professional-load-testing-drupal-and-civicr...

Jun 05 2015
Jun 05

For those of you in the New York City area, 16-19 July 2015 is NYC Drupal Camp (pronounced "nice camp"), an annual grassroots non-profit conference run by volunteers. The event covers a broad range of topics related to Drupal. As part of the camp, the developers of the Aegir hosting system have organised the first Aegir Summit, 16-17 July.

For those not familiar with Aegir: it is a control panel based on Drupal and Drush to help automate the installation of Drupal, typically in a multi-site architecture (1 code base, many independant sites). With the provision_civicrm module, Aegir can also automate the installation of CiviCRM. This means that with a few clicks, you can create a new database, install Drupal and CiviCRM, configure the web server and optionally manage the SSL certificate. It also helps to automate other tasks, such as backups, upgrades and cloning. Need to create a new testing site for a client? Two clicks and it's ready. If you often create the same types of sites: create a model site, then clone it everytime you need a new instance.

There are also plans to support WordPress in Aegir. I have been working on a prototype that uses the command line tool wp-cli instead of Drush. If you would like to try it, please keep it mind that it is highly experimental and requires patching Aegir 3 (which is still in beta, although there are Debian/Ubuntu packages). The code is available here: hosting_wordpress (please read the 'Readme' file for installation notes).

If you are into Docker and other types of farming, you might find the Aegir Summit interesting as well. There has been a lot of talk about moving Aegir to a more Docker/container-friendly architecture in the next phase, for example.

Long story short: if you are in the NYC area, it would be great to see you there. If you cannot make it and you are interested in any of the above, feel free to leave a comment on this blog post or contact me by e-mail: mathieu at symbiotic.coop.

If you are a CiviCRM hosting provider and you would like to provide a self-serve online form so that your future users can test your services and create a new CiviCRM instance, that's possible too. Hopefully we will have a demo on time for the Aegir Summit, but in the mean time, I will leave the following teaser below (and yes, the sign-up form is a CiviCRM form!) :-)

May 27 2015
May 27

Thanks to generous contributions from Fuzion and Beat Schnyder from basx GmbH, I was able to put in a few days work over the last month into the Drupal 8 integration module, allowing Drupal 8 and CiviCRM to interoperate.

I had done a large chunk of work of the initial work for this during Google Summer of Code last year but as Drupal 8 has been under heavy development, so much had changed that the module needed to be carefully picked over to bring back into a working state. Over the several days of funding I had, I managed to get both the module and install working against the latest Drupal 8 beta (at the time, beta-9 but now 10 too) and Civicrm 4.6.3 in time for the Denver CiviCon sprint.

You can check out the current state of things (along with all the bugs and so on) here: http://civid8.fudev.co.nz/ (login: demo password: demo). If you would like to explore deeper, such as testing how it works with Views, and later on with other Drupal goodies that aren't yet available in D8, then just ask us for a user account with more permissions. Fuzion is asking for contributions to help support this work - they are encouraged but not compulsory. Funds will go towards Drupal 8 support.

Any bugs you come across, please feel welcome to open an issue on JIRA and assign to me (Torrance).

For the adventurous at heart, I’ve documented the installation procedure, so you can have a go at installing a local copy to play with (and I will update this if it's not clear enough or there are issues in the process).

There’s still plenty of work to do, but the more eyes looking, the shallower the bugs. :)

 
May 24 2015
May 24

When preparing an email newsletter, one part of it that is time consuming is gathering together all the content that is needed. In my experience, virtually all the content already exists elsewhere, such as in the local CMS, in CiviCRM, or on a blog, or some other online source.    So I was thinking how can I make this process easier.  What I did: I created mail merge tokens for CiviCRM that autofill a list of recent blog posts, stories, or any other type or category of CMS content.  So the end-user sees a list of tokens, one for each content type, each term/category, each aggregator feed,  and for each date range. Such as "Content of type 'blog' created in the last 7 days" .  What is particulary powerful about this approach, is that if you are also using a CMS aggregator (such as the aggregator module in Drupal core) then virually any external RSS feed is turned into CMS content, which is now available as a CiviCRM token. (The original blog post about this extension is at: https://civicrm.org/blogs/pogstonesarahgladstone/easier-creation-email-newsletters-content-tokens )

Thanks to community involvement (specifically thanks to https://github.com/jorich-2000), there is a new version of the Content Token extension.  This version now supports Joomla, in addition to Drupal7, Drupal6, and WordPress.

The lastest version of the "Content Token" extension can be downloaded from: https://civicrm.org/extensions/content-tokens

If you like this extension, you will also likely enjoy my other extension: the "Fancy Token" extension which provides tokens for upcoming events, contribution pages, profiles, and WebForms. 

I am looking forward to getting feedback on this.

Pages

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web