Releasing with confidence

One of the things that I work on at Mollie is maintaining the Mollie payment gateway for Spree Commerce. If you’re interested in reading about the technical challenges that I faced during its development, you can read the blog post that I wrote about a year ago.

Illustration by Sven Franzen.

Although I’m not working on the Spree Commerce gateway on a daily basis, we receive pull requests every now and then. Since stability is of the essence when dealing with payments, we need to make sure all of our code changes are properly tested. This allows us to keep iterating without introducing unintended breaking changes.

In this short technical post I want to highlight some steps that I’ve taken that allow us to run the test suite of the Mollie gateway against all of our target platforms using Travis CI & Appraisal.

Setting up a test environment

Since the gateway is a Ruby gem that is built on top of Spree Commerce, we need a working Spree installation when running our tests. Luckily, Spree comes with a Rake task that takes care of setting up a full test environment.

Every time we run rake spec we want to verify whether we need to create the new Rails app (although Travis will obviously create a new Spree installation inside every container it spins up).

Let’s see what our Rakefile looks like:

require ‘bundler’
Bundler::GemHelper.install_tasks
require 'rspec/core/rake_task'
require 'spree/testing_support/extension_rake'
RSpec::Core::RakeTask.new
task :default do
  if Dir['spec/dummy'].empty?
    Rake::Task[:test_app].invoke
    Dir.chdir('../../')
  end
  Rake::Task[:spec].invoke
end
desc 'Generates a dummy app for testing'
task :test_app do
  ENV['LIB_NAME'] = 'spree_mollie_gateway'
  Rake::Task['extension:test_app'].invoke
end

A new test environment will now be generated in the folder spec/dummy when running the following Rake commands:

$ bundle exec rake test_app
$ bundle exec rake spec # Or simply run `bundle exec rake`

Installing multiple Spree versions

The version of Spree that you want to test against is usually defined in your Gemfile. You can obviously specify only one version of a dependency. Luckily, there's a great gem called Appraisal. It allows you to generate multiple Gemfiles using different dependency versions.

We can do so by creating a file called Appraisals in the root of our project:

appraise 'spree-3-5' do
  ENV['USE_LEGACY_BUNDLER'] = 'true' # We'll get to this
gem 'spree', '~> 3.5.0'
end
appraise 'spree-3-6' do
  gem 'spree', '~> 3.6.5'
end
appraise 'spree-3-7' do
  gem 'spree', '~> 3.7.2'
end

Now, when we run the following commands, three different Gemfiles will be created in the folder gemfiles:

$ bundle exec appraisal install
$ cd gemfiles
$ ls -lsah
total 24
0 drwxr-xr-x   5 vernon  staff   160B Apr 11 11:12 .
0 drwxr-xr-x  24 vernon  staff   768B Apr 11 11:12 ..
8 -rw-r--r--   1 vernon  staff   150B Apr 11 11:12 spree_3_5.gemfile
8 -rw-r--r--   1 vernon  staff   150B Apr 11 11:12 spree_3_6.gemfile
8 -rw-r--r--   1 vernon  staff   150B Apr 11 11:12 spree_3_7.gemfile

Something that’s really cool about working with Gemfiles is that it allows you to have multiple Gemfiles extend from a single Gemspec (which you should use to specify your Gem’s dependencies). An example can be found in spree_3_7.gemfile :

# This file was generated by Appraisal
source "https://rubygems.org"
gem "mollie-api-ruby", "~> 4.1.3"
gem "spree", "~> 3.5.0"
gemspec path: "../"

Setting up Travis CI

We use Travis CI as our continuous integration platform for all of our open source projects. Setting it up is easy: simply add the Travis CI to your Github repository using the Github Marketplace.

The configuration for Travis is done through a simple YAML file called .travis.yml

After a lot of tweaking, the final configuration file looks like this:

sudo: false
dist: trusty
language: ruby
cache: bundler
rvm:
  - 2.3.8
  - 2.4.4
  - 2.5.1
gemfile:
  - gemfiles/spree_3_5.gemfile
  - gemfiles/spree_3_6.gemfile
  - gemfiles/spree_3_7.gemfile
# Use legacy bundler (1.x) when using Ruby 2.3.x
before_install:
  - if [ "USE_LEGACY_BUNDLER" != "false" ]; then gem uninstall -v '>= 2' -i $(rvm gemdir)@global -ax bundler || true; fi
  - if [ "USE_LEGACY_BUNDLER" != "false" ]; then gem install bundler -v '< 2'; fi
# Allow failing tests against Spree 3.7.x for now
matrix:
  allow_failures:
    - gemfile: gemfiles/spree_3_7.gemfile
script:
  - bundle exec rake test_app
  - bundle exec rake spec
deploy:
  provider: rubygems
  gem: spree_mollie_gateway
  on:
    tags: true
    repo: mollie/spree-mollie-gateway
    rvm: 2.5.1
  gemfile: gemfiles/spree_3_6.gemfile
  api_key:
    secure: N3+LevY4HZd57cmwfIy6a+E/wX/jCD9Pnqq41/kr6I6iLIMz5xWoU0iDEZEXGhTWmId5O0zxTl2o3w4PiVkH4Onmufv3oLOONexrJeIF5cx/TdWpY36RycA2euVvOGVYMNWGIhslCern4zc8pOTwGpmm8DD8qPTQAV1JPXzbPEiXkxJX0KYm1Vb6v4GTW0q2ghvqY3fzgNaJKVQoEXVrOaXMd/dJDQRjBzbzfF8V2Z779kfOfV5PV/jqheq4bXQYMDBKQUHuTvRMNx0tF4mGgHANp1AXMS4GVbYNyAQNmyd3QnkJHSdKthTTe6nwQoH7oloY4Dbd89fG7Yx7I/kt2Aip3pHMts+4/oVN6QPqe0xe11TeC6yRhi/357I+DCzIk1aRGJ9lkpGQnM+oB54N0yw+htvsd7WE2ac57ixVG0ni39IQciQJ4p3aJzho8z3ui3CNj0p3Y4B4moEIJ3JV3Gg+TEXbzViu9G8j7YIlqw0OpGaaczavfuRSTYkE0QKA4BSqGOG+qsY8ArdvW3KgVohSxHzFR6ubhgT81JvzF6hpS7h7PalOUKA5cr7ha/dB0qL1PYNU6xHQFWiM/ILW5eF6ttQDgzA294U8FoRykG6T9swdVZA5Vd6DZafcu6ofqgyRFGQXzlhq/Hdkocb2W60l5BR4BSq4uoIvmMHXd+8=
env:
  matrix:
    secure: ftw4Vv/ri+oxmuKcKX3MElYurqR/HHSuChyx8RM7ixfP9+l87jr772OGtvg0v0neW2CCz9ot+4SXL+dewFELTr6+8rstT0C1Lcyn7ANTpBp80bnMpHZrpJFkDUYne17r8kA+OUkW71OC1RrTTfa9mfSTd9g1wKSbfbKsQvZ1Q/mmwAAX7b1YyR0k83tJO23vr1Yx4GU7RNhuv+uTTNz0kuO4ezbv5WR1wjKQQsJwcc72WDsDyqZq5o9SVxA0fFe4666v1yheBZsX5gcWSugwtRQqD8fp1Era1T+11mJDk6S/4wZu/vRrOQCEkaFYdYtY3owVg8X3JVnAgtsiTDLt+s+tkZPTfTq56f3EqIEpbZQ6x906zQfGBosVBpWO1Bn+cKMGFrURWwr+19dOHU+jXYUfvV5EE12aJq6EV82IvveK2YhZ5+axP7JYkQDjwYJcuQd4ONQvagHf1YGeqLiGB5SuXEYBRv1521276inp2DOqUNjfquQwqWSy3Q2iQ/0yrAKcjYgYBk4yS8HkpaLsYd1EzgxW5IllJ+E040ViSddIQLHpDI9cmg9/uPFaVyHxWDMNOBMOAhq7znPAEtDfG6KnRmXswqkVfR7XM7ObIX8kdpGwc/cHGZwPD1595BHCcc0y5FkyRMY9KzTvj13yNdO61icIJa0PoQzfypaKuQQ=

Let’s go through a few interesting part in this configuration:

rvm:
  - 2.3.8
  - 2.4.4
  - 2.5.1

Since I want our test suite to be ran against multiple Ruby versions, I specified three versions, which will result in three different Travis builds. Travis will then install the specified Ruby version using RVM (Ruby Version Manager).

gemfile:
  - gemfiles/spree_3_5.gemfile
  - gemfiles/spree_3_6.gemfile
  - gemfiles/spree_3_7.gemfile

You can specify which Gemfiles should be used to install the dependencies by providing them to Travis. Without this option, Travis will simply use the Gemfile in your project's root.

# Use legacy bundler (1.x) when using Ruby 2.3.x
before_install:
  - if [ "USE_LEGACY_BUNDLER" != "false" ]; then gem uninstall -v '>= 2' -i $(rvm gemdir)@global -ax bundler || true; fi
  - if [ "USE_LEGACY_BUNDLER" != "false" ]; then gem install bundler -v '< 2'; fi

A few months ago, a new version of the dependency manager for Ruby was introduced: Bundler 2. While this works great with newer versions of Rails, my build kept failing when Bundler tried to install the dependencies for Ruby 2.3.8.

The solution for this feels a bit awkward. Since Travis now always uses Bundler 2, I decided to uninstall Bundler 2 and install Bundler 1 instead when the environment variable USE_LEGACY_BUNDLER is set. This env variable is set in my Appraisals file.

Since three different Gemfiles are now used to install the dependencies using three different Ruby versions, 9 builds will be ran on each push. Since I’m still working on the compatibility with Spree 3.7.x, I’ve marked these failing builds as “allowed failures” in travis.yml.

Our final build pipeline in Travis CI.

Automatic releases

We distribute our payment gateway via the popular Rubygems, as this is the default dependency source that is set in the Gemfile when building a Ruby app.

While manual deployments are as simple as running the commands below, this requires every developer to have an account at Rubygems, while it’s more efficient to deploy from a single account.

$ gem build spree_mollie_gateway.gemspec # This will create a Gem called spree_mollie_gateway-3.0.4.gem
$ gem push spree_mollie_gateway-3.0.4.gem # Push to Rubygems

Therefore, we’ve decided to automatically deploy to Rubygems every time a developer created a new git tag.

Luckily, Travis can easily do this for us. For each new tag, we’ll now push a new version of the gem to Rubygems. To prevent multiple deploys for the same gem, we’ll only run the deployment steps in the container running Ruby 2.5.1 with Spree 3.6.x. Authentication is done by providing the encrypted Rubygems API key of our company account.

deploy:
  provider: rubygems
  gem: spree_mollie_gateway
  on:
    tags: true
    repo: mollie/spree-mollie-gateway
    rvm: 2.5.1
  gemfile: gemfiles/spree_3_6.gemfile
  api_key:
    secure: N3+LevY4HZd57cmwfIy6a+E/wX/jCD9Pnqq41/kr6I6iLIMz5xWoU0iDEZEXGhTWmId5O0zxTl2o3w4PiVkH4Onmufv3oLOONexrJeIF5cx/TdWpY36RycA2euVvOGVYMNWGIhslCern4zc8pOTwGpmm8DD8qPTQAV1JPXzbPEiXkxJX0KYm1Vb6v4GTW0q2ghvqY3fzgNaJKVQoEXVrOaXMd/dJDQRjBzbzfF8V2Z779kfOfV5PV/jqheq4bXQYMDBKQUHuTvRMNx0tF4mGgHANp1AXMS4GVbYNyAQNmyd3QnkJHSdKthTTe6nwQoH7oloY4Dbd89fG7Yx7I/kt2Aip3pHMts+4/oVN6QPqe0xe11TeC6yRhi/357I+DCzIk1aRGJ9lkpGQnM+oB54N0yw+htvsd7WE2ac57ixVG0ni39IQciQJ4p3aJzho8z3ui3CNj0p3Y4B4moEIJ3JV3Gg+TEXbzViu9G8j7YIlqw0OpGaaczavfuRSTYkE0QKA4BSqGOG+qsY8ArdvW3KgVohSxHzFR6ubhgT81JvzF6hpS7h7PalOUKA5cr7ha/dB0qL1PYNU6xHQFWiM/ILW5eF6ttQDgzA294U8FoRykG6T9swdVZA5Vd6DZafcu6ofqgyRFGQXzlhq/Hdkocb2W60l5BR4BSq4uoIvmMHXd+8=

Conclusion

After a lot of tweaking, I’m happy with the end result. Every time a new pull request comes in, the new CI setup allows us to review pull request with ease & release new versions of the gateway with confidence.

Not only does this testing strategy allow us to move faster, it also allows us to release new versions of our Spree gateway without having to manually test the changes using different Spree versions. I’d definitely recommend playing around with Travis, as it will greatly help you with a lot of automation efforts.

We’re always looking for self-driven and intuitive talents who appreciate the art of technology and can help us with shipping disruptive payment products.

Check out our current job openings.
Read our other articles here.
Follow me on Github.
Sign up at our website, or find us on Twitter.