• Sur Nantes depuis 2018
• Création de sites et applications web avec Rails

@_pantographe - https://pantographe.studio

Nicolas Brousse

• CDNL 2012 à Gobelins, Paris
• Développeur Backend historiquement PHP et Ruby/Rails depuis 4 ans

@nicolas_brousse

CI/CD en image

On utilise GitLab et GitLab CI

Pipelines et jobs

• Pipeline → ensemble de jobs regroupés par stage
• Stage → un groupe de jobs (exemples : tests, securité)
• Job → exécution d'une ou plusieurs commandes. Si une commande échoue, le job échouera

Un pipeline type à Pantographe

Un pipeline de déploiement sur staging et production

Précisions sur nos process

• Merges Requests (MR/PR) pour chaque features
• Merge sur master : deploy auto. sur qa (pour tests internes)
• Merge sur production : deploy auto. sur staging puis manuellement sur production

Features

• Deploiment sur Heroku et sur Dokku
• Execute les migrations
• Notification sur outils externes (chat, Sentry)

Aperçu du script de base

#!/usr/bin/env sh
# Variables
apply_migration=${AUTO_MIGRATE_ON_DEPLOY:-false}
start_time=$(date +%s)
end_time=$(date +%s)
SENTRY_RELEASE_VERSION=$(sentry-cli releases propose-version)
 
export GIT_REMOTE_NAME="deployment"
 
 
# Functions
headers() {
  echo ""
  echo "=====> Infos"
  echo ""
  echo "-----> Sentry: v$SENTRY_CLI_VERSION"
  echo "-----> Git: $(git --version)"
  echo "-----> Git lfs: $(git lfs --version)"
  echo ""
 
  . /opt/deploy-scripts/headers.sh
}
 
# Configure
configure() {
  echo ""
  echo ""
  echo "=====> Configure for $PROVIDER_NAME provider"
  echo ""
 
  if [ -z "$APP_NAME" ]; then
    >&2 echo "-----> Set APP_NAME environment variable"
    exit 1
  fi
 
  . /opt/deploy-scripts/configure.sh
}
 
# Deploy
deploy() {
  echo ""
  echo ""
  echo "=====> Deploy"
  echo ""
  echo "-----> Deploy from revision: $CI_COMMIT_SHA"
  echo ""
 
  . /opt/deploy-scripts/deploy.sh
 
  if [ "$apply_migration" = "true" ]; then
    echo ""
    echo "=====> Apply migrations"
    echo ""
 
    . /opt/deploy-scripts/migrate.sh
  fi
 
  end_time=$(date +%s)
}
 
notify_mattermost() {
  echo ""
  echo ""
 
  if [ -n "$DEPLOY_MATTERMOST_WEBHOOK" ]; then
    echo "=====> Notify on Mattermost"
    echo ""
 
    if [ "$1" != "failure" ]; then
      color="#42c5c5"
      fallback="🚀 __[${APP_NAME}](${CI_ENVIRONMENT_URL}) ${CI_ENVIRONMENT_NAME}__ env is now deployed"
      title="New deployment"
      text="🚀  New deployment on **[${APP_NAME}](${CI_ENVIRONMENT_URL})**"
    else
      color="#b30011"
      fallback="🚀 __[${APP_NAME}](${CI_ENVIRONMENT_URL}) ${CI_ENVIRONMENT_NAME}__ env failed to deployed"
      title="New deployment failed"
      text="💥  Deployment on **[${APP_NAME}](${CI_ENVIRONMENT_URL})** failed!"
    fi
 
    # Use: https://docs.mattermost.com/developer/message-attachments.html
    read -r -d '' payload << EOM
Some JSON code
EOM
 
    curl -sS -i -X POST --data-urlencode "$payload" "$DEPLOY_MATTERMOST_WEBHOOK" > /dev/null 2>&1
  else
    echo "-----> INFO: Configuration missing to notify on Mattermost (ENV: \$DEPLOY_MATTERMOST_WEBHOOK, \$DEPLOY_MATTERMOST_CHANNEL)"
  fi
}
 
# Catch errors
trap "script_failed" 2
 
script_failed()
{
  signal=$?
 
  notify_mattermost "failure"
 
  echo ""
  echo ""
  echo "=====> 💥 fail to deploy the application"
  echo ""
 
  exit $signal
}
 
# Main script
main() {
  headers
  configure
 
  if [ -n "$SENTRY_AUTH_TOKEN" ]; then
    echo -n "-----> "
    sentry-cli releases new "$SENTRY_RELEASE_VERSION" 2>&1
    sentry-cli releases set-commits "$SENTRY_RELEASE_VERSION" --auto 2>&1
  fi
 
  deploy
 
  if [ -n "$SENTRY_AUTH_TOKEN" ]; then
    sentry-cli releases finalize "$SENTRY_RELEASE_VERSION" 2>&1
    sentry-cli releases deploys "$SENTRY_RELEASE_VERSION" new \
     --env "$CI_ENVIRONMENT_NAME" \
     --url "$CI_ENVIRONMENT_URL" \
     --time "$((end_time - start_time))" 2>&1
   fi
 
  notify_mattermost
 
  echo ""
  echo ""
  echo "=====> 🚀 Application successfully deployed"
  echo ""
}
 
main

Inconvénients

• Gros fichiers compliqués à lire
• Difficile à maintenir
• Difficile à faire évoluer
• Difficile à tester

Principaux éléments

• Logger
• Event Manager
• Layers

Les différents évènements

configure, pre_deploy, deploy, post_deploy, deploy_succeeded , deploy_skipped et deploy_failed

4 types de layers

• Providers: code relatif au provider (ex: Heroku)
• Applications: code relatif à l'application à déployer (ex: Rails)
• ThirdParties: code relatif à d'autres outils (ex: Sentry).
• Notifiers: code pour les notifications (ex: Slack, Mattermost)

Arborescence

$ tree lib
lib
├── deployer
│   ├── cli.rb
│   ├── command_runner.rb
│   ├── event_manager.rb
│   ├── exceptions.rb
│   ├── http.rb
│   ├── layers
│   │   ├── applications
│   │   │   └── rails_application.rb
│   │   ├── base.rb
│   │   ├── notifiers
│   │   │   ├── base.rb
│   │   │   └── mattermost_notifier.rb
│   │   ├── providers
│   │   │   ├── base.rb
│   │   │   ├── concerns
│   │   │   │   ├── git.rb
│   │   │   │   └── ssh.rb
│   │   │   ├── dokku.rb
│   │   │   └── heroku.rb
│   │   └── third_parties
│   │       └── sentry.rb
│   ├── layers.rb
│   ├── logger.rb
│   ├── tasks
│   │   ├── deploy_task.rb
│   │   └── layers_task.rb
│   ├── tasks.rb
│   ├── utils.rb
│   └── version.rb
└── deployer.rb
 
8 directories, 24 files
$ tree lib
lib
├── deployer
│   ├── cli.rb
│   ├── command_runner.rb
│   ├── event_manager.rb
│   ├── exceptions.rb
│   ├── http.rb
│   ├── layers
│   │   ├── applications
│   │   │   └── rails_application.rb
│   │   ├── base.rb
│   │   ├── notifiers
│   │   │   ├── base.rb
│   │   │   └── mattermost_notifier.rb
│   │   ├── providers
│   │   │   ├── base.rb
│   │   │   ├── concerns
│   │   │   │   ├── git.rb
│   │   │   │   └── ssh.rb
│   │   │   ├── dokku.rb
│   │   │   └── heroku.rb
│   │   └── third_parties
│   │       └── sentry.rb
│   ├── layers.rb
│   ├── logger.rb
│   ├── tasks
│   │   ├── deploy_task.rb
│   │   └── layers_task.rb
│   ├── tasks.rb
│   ├── utils.rb
│   └── version.rb
└── deployer.rb
 
8 directories, 24 files
$ tree lib
lib
├── deployer
│   ├── cli.rb
│   ├── command_runner.rb
│   ├── event_manager.rb
│   ├── exceptions.rb
│   ├── http.rb
│   ├── layers
│   │   ├── applications
│   │   │   └── rails_application.rb
│   │   ├── base.rb
│   │   ├── notifiers
│   │   │   ├── base.rb
│   │   │   └── mattermost_notifier.rb
│   │   ├── providers
│   │   │   ├── base.rb
│   │   │   ├── concerns
│   │   │   │   ├── git.rb
│   │   │   │   └── ssh.rb
│   │   │   ├── dokku.rb
│   │   │   └── heroku.rb
│   │   └── third_parties
│   │       └── sentry.rb
│   ├── layers.rb
│   ├── logger.rb
│   ├── tasks
│   │   ├── deploy_task.rb
│   │   └── layers_task.rb
│   ├── tasks.rb
│   ├── utils.rb
│   └── version.rb
└── deployer.rb
 
8 directories, 24 files
$ tree lib
lib
├── deployer
│   ├── cli.rb
│   ├── command_runner.rb
│   ├── event_manager.rb
│   ├── exceptions.rb
│   ├── http.rb
│   ├── layers
│   │   ├── applications
│   │   │   └── rails_application.rb
│   │   ├── base.rb
│   │   ├── notifiers
│   │   │   ├── base.rb
│   │   │   └── mattermost_notifier.rb
│   │   ├── providers
│   │   │   ├── base.rb
│   │   │   ├── concerns
│   │   │   │   ├── git.rb
│   │   │   │   └── ssh.rb
│   │   │   ├── dokku.rb
│   │   │   └── heroku.rb
│   │   └── third_parties
│   │       └── sentry.rb
│   ├── layers.rb
│   ├── logger.rb
│   ├── tasks
│   │   ├── deploy_task.rb
│   │   └── layers_task.rb
│   ├── tasks.rb
│   ├── utils.rb
│   └── version.rb
└── deployer.rb
 
8 directories, 24 files
$ tree lib
lib
├── deployer
│   ├── cli.rb
│   ├── command_runner.rb
│   ├── event_manager.rb
│   ├── exceptions.rb
│   ├── http.rb
│   ├── layers
│   │   ├── applications
│   │   │   └── rails_application.rb
│   │   ├── base.rb
│   │   ├── notifiers
│   │   │   ├── base.rb
│   │   │   └── mattermost_notifier.rb
│   │   ├── providers
│   │   │   ├── base.rb
│   │   │   ├── concerns
│   │   │   │   ├── git.rb
│   │   │   │   └── ssh.rb
│   │   │   ├── dokku.rb
│   │   │   └── heroku.rb
│   │   └── third_parties
│   │       └── sentry.rb
│   ├── layers.rb
│   ├── logger.rb
│   ├── tasks
│   │   ├── deploy_task.rb
│   │   └── layers_task.rb
│   ├── tasks.rb
│   ├── utils.rb
│   └── version.rb
└── deployer.rb
 
8 directories, 24 files

Base du script

module Deployer
  module Tasks
    class DeployTask < Base
      def run!
        manage_exceptions do
          Deployer.runtime(:run!) do
            check_env_vars!
            load_layers!
 
            print_headers!
 
            apply :configure
            apply :pre_deploy
            apply :deploy
            apply :post_deploy
 
            logger.el
            logger.title "🚀 Application has successfully been deployed"
            logger.el
 
            apply :deploy_succeeded, title: "Notify"
          end
        end
      end
 
      private
 
      def manage_exceptions
        yield
      rescue ::Deployer::EnvVarMissingError => e
        logger.l e.message
 
        exit 1
      rescue ::Deployer::AlreadyDeployed => e
        notify_skip!(e) { logger.error e.message }
 
        exit 0
      rescue ::Deployer::CommandRunner::ExitStatusError => e
        notify_failure! e do
          logger.error e.message
          e.stderr.each { |err| logger.error err }
        end
 
        exit 1
      rescue ::StandardError => e
        notify_failure! e
 
        raise e
      end
 
      ...
    end
  end
end
module Deployer
  module Tasks
    class DeployTask < Base
      def run!
        manage_exceptions do
          Deployer.runtime(:run!) do
            check_env_vars!
            load_layers!
 
            print_headers!
 
            apply :configure
            apply :pre_deploy
            apply :deploy
            apply :post_deploy
 
            logger.el
            logger.title "🚀 Application has successfully been deployed"
            logger.el
 
            apply :deploy_succeeded, title: "Notify"
          end
        end
      end
 
      private
 
      def manage_exceptions
        yield
      rescue ::Deployer::EnvVarMissingError => e
        logger.l e.message
 
        exit 1
      rescue ::Deployer::AlreadyDeployed => e
        notify_skip!(e) { logger.error e.message }
 
        exit 0
      rescue ::Deployer::CommandRunner::ExitStatusError => e
        notify_failure! e do
          logger.error e.message
          e.stderr.each { |err| logger.error err }
        end
 
        exit 1
      rescue ::StandardError => e
        notify_failure! e
 
        raise e
      end
 
      ...
    end
  end
end
module Deployer
  module Tasks
    class DeployTask < Base
      def run!
        manage_exceptions do
          Deployer.runtime(:run!) do
            check_env_vars!
            load_layers!
 
            print_headers!
 
            apply :configure
            apply :pre_deploy
            apply :deploy
            apply :post_deploy
 
            logger.el
            logger.title "🚀 Application has successfully been deployed"
            logger.el
 
            apply :deploy_succeeded, title: "Notify"
          end
        end
      end
 
      private
 
      def manage_exceptions
        yield
      rescue ::Deployer::EnvVarMissingError => e
        logger.l e.message
 
        exit 1
      rescue ::Deployer::AlreadyDeployed => e
        notify_skip!(e) { logger.error e.message }
 
        exit 0
      rescue ::Deployer::CommandRunner::ExitStatusError => e
        notify_failure! e do
          logger.error e.message
          e.stderr.each { |err| logger.error err }
        end
 
        exit 1
      rescue ::StandardError => e
        notify_failure! e
 
        raise e
      end
 
      ...
    end
  end
end
module Deployer
  module Tasks
    class DeployTask < Base
      def run!
        manage_exceptions do
          Deployer.runtime(:run!) do
            check_env_vars!
            load_layers!
 
            print_headers!
 
            apply :configure
            apply :pre_deploy
            apply :deploy
            apply :post_deploy
 
            logger.el
            logger.title "🚀 Application has successfully been deployed"
            logger.el
 
            apply :deploy_succeeded, title: "Notify"
          end
        end
      end
 
      private
 
      def manage_exceptions
        yield
      rescue ::Deployer::EnvVarMissingError => e
        logger.l e.message
 
        exit 1
      rescue ::Deployer::AlreadyDeployed => e
        notify_skip!(e) { logger.error e.message }
 
        exit 0
      rescue ::Deployer::CommandRunner::ExitStatusError => e
        notify_failure! e do
          logger.error e.message
          e.stderr.each { |err| logger.error err }
        end
 
        exit 1
      rescue ::StandardError => e
        notify_failure! e
 
        raise e
      end
 
      ...
    end
  end
end
module Deployer
  module Tasks
    class DeployTask < Base
      def run!
        manage_exceptions do
          Deployer.runtime(:run!) do
            check_env_vars!
            load_layers!
 
            print_headers!
 
            apply :configure
            apply :pre_deploy
            apply :deploy
            apply :post_deploy
 
            logger.el
            logger.title "🚀 Application has successfully been deployed"
            logger.el
 
            apply :deploy_succeeded, title: "Notify"
          end
        end
      end
 
      private
 
      def manage_exceptions
        yield
      rescue ::Deployer::EnvVarMissingError => e
        logger.l e.message
 
        exit 1
      rescue ::Deployer::AlreadyDeployed => e
        notify_skip!(e) { logger.error e.message }
 
        exit 0
      rescue ::Deployer::CommandRunner::ExitStatusError => e
        notify_failure! e do
          logger.error e.message
          e.stderr.each { |err| logger.error err }
        end
 
        exit 1
      rescue ::StandardError => e
        notify_failure! e
 
        raise e
      end
 
      ...
    end
  end
end

Exemple de layer : RailsApplication

module Deployer
  module Layers
    module Applications
      class RailsApplication < Deployer::Layers::Base
        on :pre_deploy, if: :sidekiq_installed? do
          logger.title "Set sidekiq as quiet…"
 
          quiet_sidekiq!
        end
 
        on :post_deploy, if: :apply_migrations? do
          logger.title "Migrations status…"
          migrations_status { |stdout| logger.indent(stdout) }
          logger.el
 
          logger.title "Applying migrations…"
          apply_migrations! { |stdout| logger.indent(stdout) }
          logger.el
        end
 
        on :deploy_failed, if: :sidekiq_installed? do
          logger.title "Start sidekiq…"
 
          restart_sidekiq!
        end
 
        private
 
        def apply_migrations?
          ENV["AUTO_MIGRATE_ON_DEPLOY"] == "true"
        end
 
        def migrations_status(&block)
          provider.run "rails db:migrate:status", &block
        end
 
        def apply_migrations!(&block)
          provider.run "rails db:migrate", &block
        end
 
        def sidekiq_installed?
          provider.run "rails runner 'Sidekiq::VERSION'"
        rescue ::Deployer::CommandRunner::ExitStatusError
          false
        else
          true
        end
 
        def quiet_sidekiq!
          provider.run "rails runner 'Sidekiq::ProcessSet.new.each(&:quiet!)'"
        end
 
        def restart_sidekiq!
          provider.process :worker, :restart
        end
      end
    end
  end
end
module Deployer
  module Layers
    module Applications
      class RailsApplication < Deployer::Layers::Base
        on :pre_deploy, if: :sidekiq_installed? do
          logger.title "Set sidekiq as quiet…"
 
          quiet_sidekiq!
        end
 
        on :post_deploy, if: :apply_migrations? do
          logger.title "Migrations status…"
          migrations_status { |stdout| logger.indent(stdout) }
          logger.el
 
          logger.title "Applying migrations…"
          apply_migrations! { |stdout| logger.indent(stdout) }
          logger.el
        end
 
        on :deploy_failed, if: :sidekiq_installed? do
          logger.title "Start sidekiq…"
 
          restart_sidekiq!
        end
 
        private
 
        def apply_migrations?
          ENV["AUTO_MIGRATE_ON_DEPLOY"] == "true"
        end
 
        def migrations_status(&block)
          provider.run "rails db:migrate:status", &block
        end
 
        def apply_migrations!(&block)
          provider.run "rails db:migrate", &block
        end
 
        def sidekiq_installed?
          provider.run "rails runner 'Sidekiq::VERSION'"
        rescue ::Deployer::CommandRunner::ExitStatusError
          false
        else
          true
        end
 
        def quiet_sidekiq!
          provider.run "rails runner 'Sidekiq::ProcessSet.new.each(&:quiet!)'"
        end
 
        def restart_sidekiq!
          provider.process :worker, :restart
        end
      end
    end
  end
end
module Deployer
  module Layers
    module Applications
      class RailsApplication < Deployer::Layers::Base
        on :pre_deploy, if: :sidekiq_installed? do
          logger.title "Set sidekiq as quiet…"
 
          quiet_sidekiq!
        end
 
        on :post_deploy, if: :apply_migrations? do
          logger.title "Migrations status…"
          migrations_status { |stdout| logger.indent(stdout) }
          logger.el
 
          logger.title "Applying migrations…"
          apply_migrations! { |stdout| logger.indent(stdout) }
          logger.el
        end
 
        on :deploy_failed, if: :sidekiq_installed? do
          logger.title "Start sidekiq…"
 
          restart_sidekiq!
        end
 
        private
 
        def apply_migrations?
          ENV["AUTO_MIGRATE_ON_DEPLOY"] == "true"
        end
 
        def migrations_status(&block)
          provider.run "rails db:migrate:status", &block
        end
 
        def apply_migrations!(&block)
          provider.run "rails db:migrate", &block
        end
 
        def sidekiq_installed?
          provider.run "rails runner 'Sidekiq::VERSION'"
        rescue ::Deployer::CommandRunner::ExitStatusError
          false
        else
          true
        end
 
        def quiet_sidekiq!
          provider.run "rails runner 'Sidekiq::ProcessSet.new.each(&:quiet!)'"
        end
 
        def restart_sidekiq!
          provider.process :worker, :restart
        end
      end
    end
  end
end

Merci

• pantographe-keynotes.netlify.app
• pantographe.studio/blog/how-we-improved-our-deployment-tool