How to perform security fixes

From Wikitech

This is a brief guide on how to perform security fixes. From commit draft to minor release.

Steps

1: Commit

Locally (on your workstation) check out the currently relevant wmf branch(es). Write the fix there and test it. Then commit to your local topic branch. You also may want to check if this commit applies cleanly to the other active wmf branch (if there is two, there is usually two, e.g. wmf/1.20wmf9 and wmf/1.20wmf10), so you're prepared to handle the merge conflict later on.

Note: Commits (especially security related ones) should, of course, always add release notes!

Prefix the commit message with "SECURITY:" to make them easy to see by others on tin.

It's also useful to make sure that a task exists in Phabricator. If one doesn't exist already, please create one. You can use the TXXXXX number for the filename. This makes tracking of patches much easier

# Create a topic branch
jdoe@laptop:~/Dev/mediawiki/core$ git checkout -b TXXXXX master

# Fix the security bug

# Add the file (or files) to the staging area
jdoe@laptop:~/Dev/mediawiki/core (TXXXXX)$ git add path/to/file.php RELEASE-NOTES-1.20

# Confirm your stage contains what you want to do commit (no more, no less)
jdoe@laptop:~/Dev/mediawiki/core (TXXXXX)$ git status

jdoe@laptop:~/Dev/mediawiki/core (TXXXXX)$ git commit

2: Apply on tin

# Convert the commit to a git patch file
jdoe@laptop:~/Dev/mediawiki/core (TXXXXX)$ git format-patch HEAD^ --stdout > ~/TXXXXX.patch

# Copy it to your home directory on tin (scp defaults: protocol=ssh, username=local username, target directory=home directory)
jdoe@laptop:~/Dev/mediawiki/core (TXXXXX)$ scp ~/TXXXXX.patch tin.eqiad.wmnet

# Log into tin and go to the git dir
jdoe@laptop:~/Dev/mediawiki/core$ ssh tin.eqiad.wmnet
jdoe@tin:~$ cd /srv/mediawiki-staging/php-1.20wmf10
# Sign off uses your .gitconfig (on tin) to determine name and e-mail address.
jdoe@tin:/srv/mediawiki-staging/php-1.20wmf10 (wmf/1.20wmf10)$ git am --signoff < ~/TXXXXX.patch

# Move the .patch file to /srv/patches/<branch>/core/<order>-TXXXXX.patch (for core patches)
# or /srv/patches/<branch>/extensions/ExtensionName/<order>-TXXXXX.patch (for extension patches,
# create the directory if it doesn't exist already). <order> should be chosen to be the next, sequential
# number for files in the given directory.
jdoe@tin:/srv/mediawiki-staging/php-1.20wmf10 (wmf/1.20wmf10)$ mv ~/TXXXXX.patch /srv/patches/php-1.20wmf10/core/01-TXXXXX.patch
jdoe@tin:~$ cd /srv/patches/
jdoe@tin:/srv/patches (master)$ git add .
# The security patches directory is a git repository of its own, to track patches.
jdoe@tin:/srv/patches (master)$ git commit -a -m 'Add php-1.20wmf10/core/01-TXXXXX.patch'

  • If there are other active branches, apply the patch there as well (see roadmap).
  • You may want to verify at this point that the bug is fixed on test.wikipedia.org, if possible (i.e. testing it might require adding specific permissions to your account which could expose which code is vulnerable).

3: Deploy

See also How to deploy code. If the issue is not public, make sure you set the DOLOGMSGNOLOG=1 to prevent it from being automatically publicly logged. When done, you should go to #wikimedia-operations connect and say "!log Deployed patch for Txxxxx" to log it manually without revealing potentially sensitive details such as file names.
jdoe@tin:/srv/mediawiki-staging$ sync-file php-1.20wmf10/includes/PathToFile.php 'Deploy security fix'

4: Push to gerrit & make release

Todo: These could be moved to a generic "Backport and do a minor release" how-to page, then put a link here.

Make release

If the vulnerability is not public, send a pre-release announcement at least 24 hours before release, announcing the availability of the update. Coordinate with Red Hat's security team to assign CVE numbers if needed.

For each patch in your release, port and commit the patch to the release branches.

jdoe@laptop:~/Dev/mediawiki/core$ git checkout -t gerrit/REL1_19
# Check to make sure the patch applys, if not you'll need to backport
jdoe@laptop:~/Dev/mediawiki/core (REL1_19)$ git apply --check < ~/TXXXXX.patch
jdoe@laptop:~/Dev/mediawiki/core (REL1_19)$ git am ~/TXXXXX.patch

Update and commit RELEASE-NOTES and DefaultSettings.php version number

Tag the release

jdoe@laptop:~/Dev/mediawiki/core (REL1_19)$ git tag -a '1.19.6' -m "Security Release 1.19.6"

Copy the current version of the release branch'es extensions into extensions/ TODO: This is painful and needs to be done better

Get make-release from mediawiki/tools/release.git. Run make-release:

jdoe@laptop:~/Dev/mediawiki/tools/make-release$ ./make-release --git-root ~/Dev/mediawiki/core/ 1.19.6
Was 1.19.5 the previous release? [y/n] y
 Cloning core...

Decrypt your private key when prompted to sign tarballs. Keep the output from make-release for the release announcement email.

Copy the build branch to tin

jdoe@laptop:~/Dev/mediawiki/tools/make-release$ cd build/
jdoe@laptop:~/Dev/mediawiki/tools/make-release/build$ scp -r 1.19 tin.eqiad.wmnet:.

Have someone with root move the files to download.wikimedia.org (dataset2:/data/xmldatadumps/public/mediawiki). Warning: Once the files are copied, they are publicly available, since the directory index listings are available on the server.

Send release announcement emails.


Push to gerrit

# Commit to the wmf branch in gerrit. Especially important because the clone on tin is
# now 1 commit ahead of the gerrit repository
jdoe@laptop:~/Dev/mediawiki/core (TXXXXX)$ git push gerrit HEAD:refs/for/wmf/1.20wmf10
# Creates new gerrit change, self-merge this one through the web interface
# Okay, since it is already deployed anyway.

# Now for the release. First to master.
jdoe@laptop:~/Dev/mediawiki/core (TXXXXX)$ git push gerrit HEAD:refs/for/master

# Make note of the git commit hash 
jdoe@laptop:~/Dev/mediawiki/core (TXXXXX)$ git show HEAD --stat

# Then to the other release branch(es).
jdoe@laptop:~/Dev/mediawiki/core (TXXXXX)$ git checkout -b TXXXXX/REL1_19 gerrit/REL1_19
jdoe@laptop:~/Dev/mediawiki/core (TXXXXX/REL1_19)$ git cherry-pick 9023aa2d830029da5745e92212f03ddbc71da4c2
# Make sure to update the release notes properly (probably causes a conflict, either way update the right file)
jdoe@laptop:~/Dev/mediawiki/core (TXXXXX/REL1_19)$ git add RELEASE-NOTES-1.19
jdoe@laptop:~/Dev/mediawiki/core (TXXXXX/REL1_19)$ git commit --amend
jdoe@laptop:~/Dev/mediawiki/core (TXXXXX/REL1_19)$ git push gerrit HEAD:refs/for/REL1_19

# Creates a new gerrit change for each push. Merge as appropriate.

See also