Trebuchet/Design

Basic design

Git repositories holding our code (such as MediaWiki core) sit on the deployment system behind a web server. Users initiate a deployment using the git-deploy script (eg: git deploy start). start writes out a lock file to only allow a single deploy at a time and adds a tag to the local repository as a rollback point. At this point the user either:

• updates the repo as necessary and deploy the change
• aborts the deploy

Once the deploy is ready, the user completes the deploy by using git deploy sync. git-deploy will write out a sync tag then trigger a sync script and adds a .deploy file to the repository root, which describes the currently deployed code. The sync script updates the local repository and its submodules so that the application servers can fetch properly. Once the code has been updated and marked, git-deploy calls a salt run for fetch, then another salt run to checkout the deploy git tag. The sync script will report success or failure. After the sync script is run, git-deploy removes the lock file.

To abort a deploy, use the git deploy abort command. The script rolls back the repo to the previously set tag and removes the lock file.

Salt deploy runner

• Location (on the salt master): /srv/runners/deploy.py
• Managed in the puppet deployment module

A salt runner is a script that runs on the salt master and can combine many salt calls into a single function.

The salt deploy runner can be called from the deployment server via sudo salt-call publish.runner deploy.<function>. It is called by the git-deploy sync hook. It has two functions:

deploy.fetch(repo)

calls fetch (via a salt module) on all application servers for the specified repo

deploy.checkout(repo)

calls checkout (via a salt module) on all application servers for the specified repo

Each function returns a report in json on which minions returned successfully, failed, or didn't return.

Salt deploy module

• Location (on the salt master): /srv/salt/_modules/deploy.py
• Managed in the deployment puppet module

A salt module lives on every salt minion and can be called from the salt master or from any peer which is allowed access.

The salt deploy module is called via salt <matching-criteria> deploy.<function>. It has the following functions:

deploy.sync_all

sync all repositories configured. This will also fully clone repositories, if they are missing.

deploy.fetch(repo)

do a git fetch based on the repo location (repo_locations) and url (repo_urls) defined via salt pillars.

deploy.checkout(repo,reset=False)

do a checkout of a repo based on the repo location (repo_locations), and url (repo_urls) from salt pillars, and .deploy file defined on the deployment host. Checkout will also modify the .gitmodules file based on sed configuration defined in salt pillars (repo_regex).

Salt deployment pillars

• Location (on the salt master): /srv/pillars
• Managed in the puppet repo: role::deployment::salt_masters::production

Salt pillars are a set of configuration data available on every salt minion (via salt-call pillar.data). Pillars are managed on the master and are distributed to all minions on update.

Reporting

The salt-runner returns a report about the number of minions that returned successfully, timed out, or failed. For minions that timeout or fail, the runner returns a list of the minions. For minions that fail, it also returns a failure status for the minions.

In addition to the report, all minions update redis with information about the current tag deployed, and the status (and timestamp) of fetch and checkout. The status of what's deployed on the cluster can be found using the deploy-info script.

Deployment methods

Git

TODO: document process

JVM Application Deployment

Proposal

1. Jenkins builds release jars.
2. Jenkins pushes jars to a Maven Repository Manager (Nexus, Artifactory, etc.) via the ArtifactDeployer plugin.
3. Trebuchet will support deployment via Maven Repo Manager via a config file.
   1. config file will specify exact jars and versions to be deployed.
4. Jars are deployed via a new deploy salt module.
   1. deploy.fetch
      1. wget/rsync files listed in config file
   2. deploy.checkout
      1. link to specified files

To Do:

• Choose Maven Repository Manager and deploy in production somewhere.
• Move existing artifacts it http://nexus.wmflabs.org to production.
• Make deploy.fetch generic, and call specific functions based on a configuration option.