Help:Git

Wikimedia Labs documentationThis is a documentation page for Wikimedia Labs. If you can improve it, please do that. You can also request help on #wikimedia-labs ^connect. For all documentation pages, please see the List of all documentation pages for labs. If you are seeking a help with a specific project, please contact the project owners.

Git/Gerrit and the repositories

Restrictions and Anonymous access

Note: Push access is currently limited to staff developers and operations engineers. This will change soon. To pull the repository anonymously for read access use the following command:

git clone https://gerrit.wikimedia.org/r/p/<repository>

For example:

git clone https://gerrit.wikimedia.org/r/p/operations/puppet

Browsing only via the web

Public repositories can be browsed from the web:

https://gerrit.wikimedia.org/r/gitweb?p=operations/puppet.git;a=summary

Set up your ssh key in Gerrit

Log into the web interface for gerrit. Click on Settings then SSH Public Keys then add your key.

You can validate whether authentication works by running

ssh -p 29418 <LABSLDAPUSERNAME>@gerrit.wikimedia.org

Which should result in a message like this:

  ****    Welcome to Gerrit Code Review    ****
 Hi USERNAME, you have successfully connected over SSH.
 Unfortunately, interactive shells are disabled.
 To clone a hosted Git repository, use:
 git clone ssh://LABSLDAPUSERNAME@gerrit.wikimedia.org:29418/REPOSITORY_NAME.git

If you get the following error:

 Agent admitted failure to sign using the key.
 Permission denied (publickey).

... just run:

 ssh-add

Checking out the repositories

We have the following main repositories and branches:

operations/puppet
- production (HEAD)
  - used in production
labs/private
- master (HEAD)
  - only used in testlabs project

To check out the master (HEAD) branch of a repo, you can clone it like so (if they're different, shell-username is your unix shell name, not the wiki username):

git clone ssh://<shell-username>@gerrit.wikimedia.org:29418/<repository-name>.git

To check out the operations/puppet repo, you can clone it like so:

git clone ssh://gerrit.wikimedia.org:29418/operations/puppet.git

To check out the operations/software repo, you can clone it like so:

git clone ssh://<username>@gerrit.wikimedia.org:29418/operations/software.git

To check out the labs/private repo, you can clone it like so:

git clone ssh://gerrit.wikimedia.org:29418/labs/private.git

Updating your local repository

Don't 'pull' or 'merge'.

After you do the initial repository clone, git will automatically set up a remote tracking branch for you. You simply need to fetch and rebase:

git fetch
git diff origin/<branch>
git rebase origin/<branch>

Making changes

After cloning the repo, you can make changes the normal git way:

<make changes>
git add <newfiles>
git commit -a

Then push your changes using git-review:

git review

Amending a change

Occasionally your change may fail a lint test, or someone may review your change and ask you to fix something. In this case, you'll need to amend your change. To do this:

Checkout the change (you can find this line in Gerrit, on the change, in Download -> checkout, ssh):
git fetch ssh://<username>@gerrit.wikimedia.org:29418/operations/puppet <ref> && git checkout FETCH_HEAD
Make changes
Commit the change, ensuring you are amending the commit
git commit --amend -a
Push the change
git review

Fixing a path conflict

To fix a path conflict, you can rebase your change against the tip of the branch. Here's a basic example:

Ensure your branch is up to date:
git pull
Checkout your change
git fetch ssh://<username>@gerrit.wikimedia.org:29418/operations/puppet <ref> && git checkout FETCH_HEAD
Rebase it against origin/production
git rebase origin/production
If there are conflicts, fix them, and run git rebase --continue
Push the change
git review

Making the changes live in puppet

Labs

Labs changes are automatically pulled. There's nothing you need to do after the merge.

Production

Once changes are merged on Gerrit, an Op needs to take one last step to apply the change on production. Merging instructions for the main repo are here and for the private repo here.

Migration from svn

If you previously worked on a project with svn (e.g. from the toolserver), you can convert your commit history to git and import it in gerrit. You'll need to install git-svn locally for the conversion.

Conversion

First, import the svn repository and convert it to git:

git svn clone svn_repository_address new_git_dir/

You now have a git repository in new_git_dir/. To check, cd in new_git_dir/ then "git log". You should see all the commit history converted from svn. However, you may want to replace the name and mail address of the contributors. To do that, first get a list of all contributors, like "git log | grep Author | uniq", and create a file (authors.txt) in the form:

svn_name1 = git_name1 <mail_address1>
svn_name2 = git_name2 <mail_address2>

The easier is then to reimport the svn repository:

git svn clone svn_repository_adress new_git_dir/ -A authors.txt

You now have a git repository with all your svn commit history and the authors should be properly defined (for more details, check man git svn, especially if your repository has some special features, like ignored files).

Import

Now, to import your git repository to gerrit, you need to set gerrit as your remote.

git remote add origin your_gerrit_repository_address

NB: origin is just a name for the remote repository, you could as well call it gerrit for more clarity.

The last thing you need is to push your git repository (represented here by your only branch master) to gerrti (origin):

git push origin master

If it doesn't work, you may need to force push. What it means is generally that you already have a repository which is not compatible with the one you want to upload: gerrit doesn't want to lose the commit history it may have. Hence, considering that your repository was just created and is empty, you can force push.

To do that, you need to have the rights to do so. Go on the gerrit page of your project and find the Access page, then Edit and add the permission Push to your group. Don't forget to check the box Force. You should now be able to:

git push origin master --force

Your project should now be in gerrit and ready to to be used. You can delete the new_git_dir/ if you want. Don't forget to set your user name and address if you want to commit to it and send it for review.

As a precaution, you should revoke the Push rights, as you only need them for the initial import.

Windows

These are the instructions for setting up on a Windows dev machine. Please improve these instructions if you find them to be inaccurate / incomplete.

Installation with Cygwin

If you're comfortable with Cygwin, then use it to install git, ssh, wget and python. Install pip, and use pip to install git-review:

 $ wget http://peak.telecommunity.com/dist/ez_setup.py
 $ python ez_setup.py
 $ easy_install pip
 $ pip install git-review

Now, clone the repository and set up git-review:

 $ git clone ssh://<USERNAME>@gerrit.wikimedia.org:29418/mediawiki/core.git
 $ cd core; git review -s

You should be able to use git and git-review normally.

Installation

Install the following packages:

msysgit (http://code.google.com/p/msysgit/) Afterward, restart Windows because the changes to the path seem to only be actuated when that is done.
turn on Pageant (comes with putty), and load your private key
install git-review (pip install git-review)

Optional packages:

TortoiseGit (http://code.google.com/p/tortoisegit/) if you want to have integration with Windows Explorer.

'IMPORTANT': Add the git directory to your path.

Configuration

Command line:

Clone the repository: git clone ssh://<username>@gerrit.wikimedia.org:29418/<main_project>/<project>.git

UI:

Right click in Windows Explorer and select "Git clone"
Fill in Url (ssh://<username>@gerrit.wikimedia.org:29418/analytics/reportcard.git) and Directory (some not-yet-existing or empty directory). Select "Load Putty key" and navigate to where your private key resides.

From then on, use a command prompt for further commands. First navigate to the location of the repository, then:

git checkout -b mytestbranch (or another name)
...edit...
git add <filename> (depends on whether you added new files)
git commit -a
git review
(if necessary) git checkout master

Books, guides, tutorials, documentation, etc.

Shared repositories on labs

On some Labs instances, multiple users commit to or pull into the same repository. Before changing such a repository, navigate to the root of the directory and run:

sudo chgrp -R sharedgroup .
sudo chmod -R g+w .
git config core.sharedRepository group

where sharedgroup is some group that includes everyone using the repo.