How to deploy Wikidata code

This page explains procedures for how to make configuration changes, prepare a new Wikidata build (Wikibase + dependencies) and get code updated on Wikidata. See also how to deploy code that is not specific to Wikidata.

Deployment schedule

Wikidata is on the 'deployment train', meaning that Wikidata is updated the same time as MediaWiki core is updated during the general deployment window. A new Wikidata deployment build (Wikibase branch + dependencies) is normally made every two weeks, so that wikidata.org is updated every two weeks, which means we don't add a new branch every other deployment train cycle.

Occasionally small bug fixes or config changes are needed outside the normal deployments, normally these are done within a SWAT. Rarely a special deployment window is scheduled for something related to Wikidata.

How to make a new Wikidata build for branches / production deployment

Make a new deployment branch of Wikibase

On command line with latest master of Wikibase, replacing branch name in example with appropriate branch name:

git checkout -b wmf/1.25wmf1
git push origin wmf/1.25wmf1

Note: It's also possible to make a branch of Wikibase using Gerrit interface.

Make a new Wikidata build

The Wikidata build uses https://github.com/wmde/WikidataBuildResources as the base for running composer install. To make a build from scratch with this, do:

1) Obtain WikidataBuildResources:

Clone https://github.com/wmde/WikidataBuildResources
Run npm install (so we can use grunt convenience scripts)
grunt branch --branchName="wmf/1.25wmf8"

2) Make new branch on gerrit:

Clone / checkout previous deployment branch of https://git.wikimedia.org/summary/mediawiki%2Fextensions%2FWikidata (e.g. wmf/1.25wmf7)
push this as a new deployment branch to gerrit: git push origin wmf/1.25wmf8

3) Then copy the new build into the Wikidate directory:

rm -Rf * in Wikidata
cp -R ../WikidataBuildResources/* . into the Wikidata directory.
then review and submit the changes to the deployment branch of Wikidata in gerrit.

If the contents of https://github.com/wmde/WikidataBuildResources are unchanged since last deployment build, then you can just do:

Clone / checkout previous deployment branch of https://git.wikimedia.org/summary/mediawiki%2Fextensions%2FWikidata
push this as a new deployment branch to gerrit: git push origin wmf/1.25wmf8
Run npm install (so we can use grunt convenience scripts)
grunt branch --branchName="wmf/1.25wmf8"
git review wmf/1.25wmf8 to submit the new deployment branch to gerrit, so that jenkins runs tests etc. and you can view the diff in gerrit.

Document stuff

Ahead of time, document important changes and bug fixes that are in the new branch on mw:Wikidata deployment. This helps keep a record of what changed and when, things to announce to the community, and be on look out for any issues after deployment related to those changes.
On mw:Wikidata deployment, important to document any deployment notes (e.g. config changes needed).

To help search for things to document, you can search commit messages via git log, with the --grep option:

git log --grep='DEPLOY'
git log --grep='bug'

Update an existing build

Cherry pick the relevant change in Wikibase to the appropriate deployment branch (e.g. wmf/1.27.0-wmf.2) and then +2 code review (a deployer needs to do this), if Jenkins approves and it looks sane.
Clone Wikidata.git from gerrit
Run composer update --prefer-dist -o --ansi or if you want to just update a specific component (e.g. wikibase), then composer update --prefer-dist -o --ansi wikibase/wikibase. Note: You might need to wait 5 minutes or so for packagist / composer to find the new change in Wikibase.
Check the diff and composer.lock file look sane.
Submit the change to review: git review wmf/1.27.0-wmf.2 Do not merge until swat or ready to deploy, since a core submodule update is automatically done (, but see exceptions where automatic submodule updates don't work). This might cause trouble or confusion if someone else is deploying something unrelated before us.

Then poke someone (aude, hoo, or Jan) or sign up for a SWAT deploy to have Wikidata updated.

If the backport involves i18n messages, then ask to have "scap" run which rebuilds localisation cache. If this is not done, then messages may break at a later time. (e.g. when LocalisationUpdate runs, which pulls in / deploys new translations from translatewiki)

For when the deployment actually happens, see #What to do before / during deployment time.

During deployment

Make sure all config changes and other todos in the deployment notes section of mw:Wikidata deployment (for the relevant branch) are done.
Wikibase etc. automatically gets deployed along with new branches of MediaWiki core at the regularly scheduled core deployment times. See above and mw:Wikidata deployment and Deployments for schedules and details. Mukunda (20after4) or another WMF person will deploy, same time as he/she deploys the new core branch. (Wikidata is a submodule)
When the wikis get switched to new version of core + Wikibase / Wikidata build, then check the wikis to be sure things are good.
At deployment time, hang out on #wikidata and #wikimedia-operations IRC channel in case problems occur with our stuff.
On mw:Wikidata deployment, mark the deployment as done. (for new code deployment to test.wikidata / test2 / wikidata / wikipedias)
Inform Lydia that deployment is done and she will handle announcements and communications to the community. (particularly for deployment of new code)

Deploy a hotfix

Hot fixes are critical bug fixes / fix regressions and are to be avoided, if possible, or kept to a minimum.

Procedures

If a hot fix is needed for deployment, then follow these steps:

cherry pick changes into Wikibase.git deployment branch (hint: Gerrit has a cherry pick button)
wait for WikidataJenkins to approve (and manually verify also, as needed, to be sure!)
+2 code review and WMF Jenkins will merge
Then update Wikidata.git deployment branch, per steps above, but don't merge it yet.
Poke aude, hoo, or another deployer (+ GregG) if it is an urgent bug fix. If it is semi-urgent, then sign up for a SWAT deploy (where someone will deploy for you). SWAT deploy windows are twice a day (15:00 UTC in the summer, and 23:00 UTC), Monday to Thursday.
Once deployed, double check test.wikidata / test2.wikipedia/ wikidata etc. to verify the hotfix
Hang out on #wikimedia-operations and #wikidata IRC channels in case problems occur

Note: there no deploys on Fridays (and weekends) except for emergencies.

Configuration changes

Wikidata configuration is in https://git.wikimedia.org/summary/operations/mediawiki-config.git

wmf-config/Wikibase.php
wmf-config/InitialiseSettings.php
wmf-config/CommonSettings.php

People listed on https://gerrit.wikimedia.org/r/#/admin/groups/21,members can approve changes and deploy them.

Normally aude or hoo handles these, but one can also ask Reedy, Greg G. or sign up for a SWAT deploy (where someone will deploy for you).

There no deploys on Fridays (and weekends) except for emergencies.

Schema changes

Schema changes should be reviewed by the WMF's database engineer, Sean Pringle (springle on irc) or at least he be informed if it's something simple.

Also, important to note, schema changes are not done at the same time as code deploys and update.php is never run in production. If suitable, schema changes might be done ahead of time or some time later. This means code must be written in a backwards compatible way to handle the new column, table or whatnot not existing yet. This might be done with a feature flag or some other way.

As well, it is important for tables to have a single column primary key, so that the Online Schema Changes (OSC) tool can be used to do a hot deploy of the schema change without putting Wikidata in read-only mode.

To have the schema change done:

File a ticket on bugzilla under the "Wikimedia" product, "Wikidata" component and with the "schema-change" keyword.
Send an email to the database engineer (Sean) and cc folks like Robla and Reedy.

After schema change is done:

Add table to dumps: Add the table to the dump python scripts (https://git.wikimedia.org/summary/operations%2Fdumps.git), by either poking Ariel (apergos) or making a patch yourself + poke Ariel. The 'ariel' branch of the dumps git repo is currently (as of July 2014) being used, so make the patch there. See 71087 for an example. A good description of the table is important. If the table contains any private information (e.g. ip addresses), then adding it to the dumps is more complicated and it would be best for Ariel to do it.
Add table to toollabs: The scripts for managing database replications on labs are in https://git.wikimedia.org/summary/operations%2Fsoftware.git. An example of how to add tables is https://gerrit.wikimedia.org/r/#/c/118582/ and how to have a column added is https://gerrit.wikimedia.org/r/#/c/125369/. Then poke Coren and Sean Pringle (springle) to review / deploy. If the table involves private information (e.g. ip addresses), then it is best just to poke Coren and Sean and let them handle it.

Further details:

Schema changes

Update cron jobs

Dispatch changes and some other stuff is run as a cron job. Cron jobs are configured in puppet, in "manifests/misc/maintenance.pp".

https://git.wikimedia.org/blob/operations%2Fpuppet.git/55e76106e0e976c5d56ccb2a54921ad90bbe764d/manifests%2Fmisc%2Fmaintenance.pp

Any cron jobs that you want to stop need to be marked as "absent" and do not get removed from puppet manifest.

To change a cron job, add a new cron job and mark the old one as "absent".

How to make a Wikidata build for master / beta deployment

Builds for beta are generated daily on a labs instance, which is in the Wikidata-build project. Both WikidataJenkins and wmf-jenkins will run tests on any new builds. Once they both approve, then someone needs to +2 code review and then Jenkins will merge the new build. Builds for master branch of Wikidata.git will be deployed automatically to beta.