Using open source Minio and the Flysystem S3 module to handle Drupal files

We are in the process of transforming the way we host our applications to a docker based workflow. One of the challenges we face is the file storage. At the heart of our business are open source technologies and tools, therefore  we have looked into in using Minio (more or less the same as Amazon S3 for file storage) instead of local filesystem (or Amazon S3).

We are going to use the Drupal module Flysystem S3 - that works both with Amazon S3 and Minio (compatible with the Amazon S3).

Flysystem is a filesystem abstraction library for PHP which allows you to easily swap out a local filesystem for a remote one - or from one remote to another.

For a new site it is pretty straight forward, for a legacy site you need to migrate your files from one storage to another - that I am going to look into in the next blog post.

Minio container

First we need Minio up and running. For that i am using docker, here is an example docker-compose.yml:

services:
  minio:
    image: minio/minio:edge
    container_name: minio
    hostname: minio
    ports:
      - "8001:9000"
    volumes:
      - "./data:/data"
    environment:
      - "MINIO_ACCESS_KEY=AFGEG578KL"
      - "MINIO_SECRET_KEY=klertyuiopgrtasjukli"
      - "MINIO_REGION=us-east-1"
    command: server /data


Settings

When you have installed the Flysystem S3 module (and the dependency - the module Flysystem), we need to add the settings for Minio to our settings.php file (there is no settings for this in Drupal. Yet.):

$schemes = [
    's3' => [
        'driver' => 's3',
        'config' => [
            'key'    => 'AFGEG578KL', 
            'secret' => 'klertyuiopgrtasjukli',
            'region' => 'us-east-1',
            'bucket' => 'my-site',
            'endpoint' => "http://minio.mysite.com:9000",
            'protocol' => "http",
            'cname_is_bucket' => false,
            "cname" => "minio.mysite.com:8001",
            'use_path_style_endpoint' => TRUE,
            'public' => true,
            'prefix' => 'publicfiles',
        ],
        'cache' => TRUE, 
        'serve_js' => TRUE,
        'serve_css' => TRUE,
    ],
];
$settings['flysystem'] = $schemes;

Endpoint is for communicating with Minio, cname is the base URL that files is going to get on the site. Serve_js and serve_css is for Minio to store aggregated CSS and JS.

Create a field

You now need to define which fields are going to use the S3 storage, for this, I create a new image reference field, and use “Flysystem: s3” as the Upload destination.

Surf over to Minio - our example is on http://minio.mysite.com:9000 - add the defined bucket, my-site, and make sure that drupal can write to it (edit policy in Mino and make sure it has read and write on the prefix - or for wildcard prefix - *)

And you are done

And that is it - now we are using Minio for storing the images. Try to upload a file on the field you created - and you should see the file in Minio. Also on the site - you should of course see the image - but now with the URL used in the settings for CNAME, in our case, minio.mysite.com:8001.

We have put some time and effort into the Flysystem S3 module together with other contributors, and we hope you will test it out and report any feedback. Have fun!

Original Post: 

About Drupal Sun

Drupal Sun is an Evolving Web project. It allows you to:

  • Do full-text search on all the articles in Drupal Planet (thanks to Apache Solr)
  • Facet based on tags, author, or feed
  • Flip through articles quickly (with j/k or arrow keys) to find what you're interested in
  • View the entire article text inline, or in the context of the site where it was created

See the blog post at Evolving Web

Evolving Web