Version Control Using Git

From Koha Wiki

Jump to: navigation, search
Git > Git
Home > Development
Home > Documentation

Contents

Version Control for Koha using Git

So you want to contribute code to Koha, that's great! First off, welcome to the community, we're glad to have you.

First I want to say a word about version control. Why use version control at all? It may be tempting if you download koha and install it for your library to just edit the files to customize it for your local needs outside the context of a version control system.

But spending a few minutes learning how to use a version control system will open up a whole new set of ways you can interact with the koha community, and will make it easy to update your production system, easy to test revisions, and easy to contribute those back to the main project. So if you're going to write or customize Koha code, take a few minutes to familiarize yourself with the primary tool us Koha developers use for version control.

This should be considered a working document, as we all get used to using git, and we get more developers on board, our workflow is likely to change.

Our goal is to make it as easy as possible to integrate patches from the community, and to make it as easy as possible for all developers to work with the project repositories as well as the features they are developing for their libraries/customers

motto "Every time a patch falls on the floor, a kitten dies"


Roles

Release Manager: responsible for feature freezes, scheduling a release

QA Manager: responsible for defining coding practices and ensuring that code conforms to coding practices

Developers: write code, document it, push it out :-)

Public: do you want to be updated with the last changes without writing code

Get Git

On Debian 6:

sudo apt-get install git-core

On Ubuntu (git-core has been superceded since 10.10) or Debian 7 (Wheezy):

sudo apt-get install git git-email

For your documentation needs, Scott Chacon over their way wrote this:

http://progit.org/

Browse Git

The project's public repository, including commit logs, diff viewing, and searching, can be browsed on the Web at http://git.koha-community.org/

Getting Started with Git

Fill some parameters

Set your name & mail address
git config --global user.name "your NAME"
git config --global user.email "your@mail.com"


Set up a SMTP server

If necessary, set up a SMTP server

git config --global sendemail.smtpserver "smtp.address.com"
git config --global sendemail.smtpuser "login@email.address.com"
git config --global sendemail.smtppass "password"
git config --global sendemail.smtpssl "true"


For example, to set up using your GMail account as your SMTP server:

git config --global sendemail.smtpserver smtp.gmail.com
git config --global sendemail.smtpserverport 587
git config --global sendemail.smtpencryption tls
git config --global sendemail.smtpuser your_email@gmail.com
git config --global sendemail.smtppass your_password


NOTE: This configuration should also work if your email is hosted on a Gmail platform. Just use your hosted email address in the sendmail.smtpuser slot. If your password contains characters that have special meaning to the shell, you may need to escape them with \ when setting sendemail.smtppass (e.g., typing \! is necessary if ! is in your password).


Automatically fix whitespace errors in patches

As Chris pointed out in his blog, the following lines make Git automatically fix trailing whitespaces

git config --global core.whitespace trailing-space,space-before-tab
git config --global apply.whitespace fix
Set Syntax Highlighting/Colour UI

It can be difficult to scan through content in Git when all the text is grey on black. Enabling this option will change the colours of some of the text to make separations of content more meaningful. This is especially noticeable when using commands such as "git show".

git config --global color.ui auto
More information on Git configuration

For more information on customizing your Git configuration, consult sites such as this one: http://git-scm.com/book/en/Customizing-Git-Git-Configuration

Getting Koha using Git

Clone the Public Repository

Clone the public repository:

git clone git://git.koha-community.org/koha.git kohaclone

You're gonna get a bunch of stuff that looks like:

$ git clone git://git.koha-community.org/koha.git kohaclone
  
  remote: Generating pack...
  remote: Done counting 16310 objects.
  remote: Deltifying 16310 objects.
  remote:  100% (16310/16310) done
  Indexing 16310 objects.
  remote: Total 16310, written 16310 (delta 10657), reused 12014 (delta 7834)
  100% (16310/16310) done
  Resolving 10657 deltas.
  100% (10657/10657) done
  Checking files out...
  100% (3429/3429) done

Cloning over HTTP

To clone the repo using the Git protocol (git:), you need to be able to connect to port 9418 on git.koha-community.org. If for some reason you have a restrictive firewall that cannot be changed, and using SSH port forwarding is not an option, it is possible to clone the repo using HTTP:

git clone http://git.koha-community.org/koha.git kohaclone

Other git operations such as git fetch and git rebase will work normally after you clone using HTTP.

Note that because this requires a cronjob (git-update-server-info) that is set to run every 10 minutes, cloning or fetching over HTTP can result in a repo that's slightly out of date with respect to the public repo. As a general note, unless you absolutely have to use HTTP because of restrictive firewall rules, using the Git protocol is more efficient, both for you and the server that hosts the repository.

Doing work on Koha with Git

You will want to create a branch to work on.

If you want to track the current development version, which is on the master branch's HEAD, do this:

$ cd kohaclone
$ git checkout -b mywork origin

If you want to track the maintenance branch of an earlier version of Koha (e.g., 3.0.x), you can do the following:

$ cd kohaclone
$ git branch -r
 origin/3.0.x
 origin/HEAD
 origin/master

The origin/3.0.x branch is the name of the maintenance branch for Koha 3.0, including future bugfix releases such as 3.0.1. To make a local (to your repository) branch to track this (remote, from git.koha-community.org) branch, you can do the following:

$ git branch --track 3.0.x origin/3.0.x
$ git checkout 3.0.x

If, after doing this, you want to create a branch on top of the 3.0.x branch in order to do some work, you can do this:

$ git checkout -b my-3.0-work

If you're still confused read this slightly modified IRC log:

nengard: got a question - how do i checkout a branch from git - i want just the 3.2 branch not head [1:17pm] owen: http://wiki.koha-community.org/wiki/Version_Control_Using_Git [1:19pm] owen: nengard: Although I guess that gets you 3.2.x, not official 3.2 [1:19pm] jcamins: You can also check out by tag. [1:21pm] chris: git checkout -b my_3.2.x origin/3.2.x [1:21pm] owen: And that gets the 3.2.x development branch right? Not the official release [1:21pm] chris: makes it and checks it out [1:21pm] chris: if you want the release, you need to check out the tag [1:21pm] owen: What about: git checkout v3.02.01 [1:22pm] chris: owen that will work, but you wont be on a local branch [1:22pm] chris: which is totally fine [1:22pm] chris: unless you want to make changes and change to another branch

Do some work

OK, now you've got your own clone, you're ready to get to work. Lets say you're a template designer who works for the Nelsonville Public Library and you're working on some improvements to the templates in Koha Version 3.0.

So at this point, you're ready to do some work. You'll be able to work within this 'clone' you've created, with version control, so you'll have a complete history of your work, without affecting the rest of the project; and you'll also have a simple way to notify others of the work you've done so they can test it out.

If you've used CVS before, this next bit is gonna be pretty standard. After you've made some changes to a file, simply go:

$ git commit file 

You can also just commit a bunch of files with:

$ git commit -a 

You'll be asked to enter a comment for your patch. On the first line, please start by the bugzilla number, with something like

"Bug NNNN blabla bla

full description of what my patch does

continued

continued

...

"

If your patch is an ENHancement, please add [ENH] just after the patch number in your patch comment 1st line (example: "Bug 6543 [ENH] I add a very nice feature")

You can see how your recent commit fits into the context of all other commits in this branch/clone with the log command:

$ git log

If your work requires changes in the mySQL database, then look at http://wiki.koha-community.org/wiki/Tutorial_for_Updating_Database_Files

Update your git install of Koha

Sometimes you'll be working on something on a branch that needs to be updated to the latest 'origin'. Running the following commands will grab all the changes since you last pulled and merge them with your current changes:

$ git fetch
$ git rebase origin/master 

If you are working on a branch derived from the 3.0.x branch, you should rebase against origin/3.0.x:

$ git rebase origin/3.0.x
Using git stash

If you have made changes to your copy of Koha and are not ready to commit them just yet you can still update. Simply do this:

$ git stash save
$ git fetch
$ git rebase origin/master
$ git stash apply


Share with the rest of the world

So now we're happy with our work, and we're ready to show it to the world.

The way we do this is the same way as the way the developers for the linux kernel work. We use the tools in git to format a patch and send it to the QA manager. Once that patch has been approved, it is pushed on to the RM Manager who has final say on where the patch will be applied.

Please make sure your install is up to date with current origin (or the release you are working on) before submitting.

To create a patch and send it to the QA Manager, use git-format-patch and issue:

$ git format-patch origin 

If you are working on a branch derived from the 3.0.x branch, you should 'git format-patch' against origin/3.0.x:

$ git format-patch origin/3.0.x

It will produce a numbered series of files in the current directory, one for each patch in the current branch but not in origin/HEAD. Now send it off using git-send-email:

$ git send-email 0001-filename 

and choose koha-patches@lists.koha-community.org for the TO: address. See Set an email alias for Koha patches for a shortcut. (Note: koha-patches@lists.koha-community.org is a mailing list and can be subscribed to at [1]. Discussion of patches occasionally occurs on this list, so it is recommended that active Koha developers subscribe to it).

It is also customary to attach your patch to the relevant bug in the Koha bugs database, particularly if your patch is preliminary and you want people to discuss it. You can do this manually or by using git-bz.

If for some reason git-send-email does not work for you, as an alternative, you can retrieve the patch files from your server, add them as attachments to an email, and send them to the patches list. Please do not copy and paste the text of your patch into an email, as that can break the formatting of the patch and make it impossible to apply.

Delete your Branch

To delete a branch:

git branch -D branchname

To delete multiple branches at once:

git branch | grep "^  bug_" | xargs git branch -D

N.B. Change "bug_" to whatever branchname pattern that you want to match on for branches to delete

Koha Revision Control Overview for RMs

Applying Patches

Ok, the release manager has received some patches and wants to apply them.

Git also provides a tool called git-am (am stands for "apply mailbox"), for importing such an emailed series of patches. Just save all of the patch-containing messages, in order, into a single mailbox file, say "patches.mbox", then run

$ git am -3 -i -s -u patches.mbox 

Git will apply each patch in order; if any conflicts are found, it will stop, and you can fix the conflicts as described in "Resolving a merge". (The "-3" option tells git to perform a merge; if you would prefer it just to abort and leave your tree and index untouched, you may omit that option.)

Once the index is updated with the results of the conflict resolution, instead of creating a new commit, just run

$ git am --resolved 

and git will create the commit for you and continue applying the remaining patches from the mailbox.

The final result will be a series of commits, one for each patch in the original mailbox, with authorship and commit log message each taken from the message containing each patch.


Pushing changes to a public repository

Now they want to update the public repository so the rest of the world can get the new code.

The simplest way to do this is using git-push and ssh; to update the remote branch named "master" with the latest state of your branch named "master", run

$ git push ssh://yourserver.com/~you/proj.git master:master 

or just

$ git push ssh://yourserver.com/~you/proj.git master 

As with git-fetch, git-push will complain if this does not result in a fast forward. Normally this is a sign of something wrong. However, if you are sure you know what you're doing, you may force git-push to perform the update anyway by proceeding the branch name by a plus sign:

$ git push ssh://yourserver.com/~you/proj.git +master 

Note that the target of a "push" is normally a bare repository. You can also push to a repository that has a checked-out working tree, but the working tree will not be updated by the push. This may lead to unexpected results if the branch you push to is the currently checked-out branch!

As with git-fetch, you may also set up configuration options to save typing; so, for example, after

$ cat >>.git/config <<EOF
[remote "public-repo"]
       url = ssh://yourserver.com/~you/proj.git
EOF

you should be able to perform the above push with just

$ git push public-repo master

See the explanations of the remote.<name>.url, branch.<name>.remote, and remote.<name>.push options in git-config for details.

Koha Revision Control Overview for QA Manager

The QA manager will receive patches and will want to either apply them and then push them upstream to the RM's repo or escalate the patch for the RM to view.

Dealing with Patches

The QA manager needs to check if the patch is a bugfix or a new feature. If it is a bugfix they then need to check that it confirms to the coding guidelines. If it does they can then apply the patch (the same way as the RM does). Then they do a git-push to push this back to the RM's repository. If they are new features, the QA Manager will forward them to the Release Manager to look at.

For example

  • Check mail
  • Look at new patch email, its a bug fix, looks good, save to a file to_apply
  • git-am to_apply
  • git push
  • Let the RM know.

OR

 - Check mail
 - Look at new patch email, ooh a neat new feature, forward to RM Manager


Koha Revision Control for the public

Clone the public repository


git clone git://git.koha-community.org/koha.git whateveryouwanttocallyourrepo 

You're gonna get a bunch of stuff that looks like:

git clone git://git.koha-community.org/koha.git kohaclone 
  remote: Generating pack...
  remote: Done counting 16310 objects.
  remote: Deltifying 16310 objects.
  remote:  100% (16310/16310) done
  Indexing 16310 objects.
  remote: Total 16310, written 16310 (delta 10657), reused 12014 (delta 7834)
  100% (16310/16310) done
  Resolving 10657 deltas.
  100% (10657/10657) done
  Checking files out...
  100% (3429/3429) done

To update the files in your machine:

cd /your path/kohaclone

git pull

You're gonna get a bunch of stuff that looks like:

git pull

  Updating a4d6314..3d41470
  Fast forward
  C4/Biblio.pm                                       |  209 +-
  C4/Branch.pm                                       |   16 +-
  C4/Breeding.pm                                     |    7 +-
  C4/Circulation.pm                                  |   48 +-
  C4/Context.pm                                      |    3 +-
  C4/Items.pm                                        |    6 +-
  C4/Koha.pm                                         |  550 +-
  C4/Languages.pm                                    |   38 +-
  C4/Output.pm                                       |   19 +-
  C4/Reports.pm                                      |   51 +-
  C4/SIP/ILS/Transaction/Checkout.pm                 |    5 +-
  C4/SIP/ILS/Transaction/Renew.pm                    |    3 +-
  INSTALL.debian                                     |    7 +-
  Makefile.PL                                        |    3 +
  admin/cities.pl                                    |   12 +-
  admin/letter.pl                                    |   15 +-
  admin/printers.pl                                  |   13 +-
  admin/roadtype.pl                                  |   12 +-
  .../intranet-tmpl/prog/en/includes/help-bottom.inc |   11 +-
  .../intranet-tmpl/prog/en/includes/help-top.inc    |    4 +-
  .../prog/en/includes/reports-menu.inc              |    6 +
  .../prog/en/includes/sysprefs-menu.inc             |    2 +-
  t/Accounts.t                                       |    3 +

After you have successfully pulled a new 'Koha' repo and you want to rewind your repo back to the Release-Candicate-1 tag (as it's currently at HEAD) for a nice safe install, try this...

   git fetch --tags
   git reset --hard v3.00.00-stableRC1

NOTES

If you want to mail the diffs, put the following in the post-receive file:

echo "Summary of changes:"
   git diff-tree --stat --summary --find-copies-harder $oldrev..$newrev
   git diff $oldrev $newrev

French tutorial / doc :

I (Paul) have found that http://wiki.archlinux.fr/howto:production:git is very clear and useful.


NOTE if you get this error:

   Environment problem:
   Your name cannot be determined from your system services (gecos).
   You would need to set GIT_AUTHOR_NAME and GIT_COMMITTER_NAME 
   environment variables; otherwise you won't be able to perform
   certain operations because of "empty ident" errors.
   Alternatively, you can use user.name configuration variable.
   
   fatal: empty ident  <your@email.com> not allowed

You're probably using a slightly older version of git that needs you to export some variables:

$ export GIT_AUTHOR_NAME="whatever"

$ export GIT_COMMITER_NAME="whatever"

This could easily also be called "How to Submit a Patch". I'm mucking about here so that'll come up on a search. ;)

tips and tricks

You can also find some tips and tricks for advanced user at the tips_and_tricks



Further reading


Developer handbook

Personal tools