Release management

From Koha Wiki

Jump to: navigation, search
Home
Home
Koha > Technical > Development
Koha > Technical > Development > Release Management

Note: this guidelines are work in progress, and release mangers have the right to change this to whatever they prefer. For now this will serve as a documentation place for the tasks involved in release management.

Contents

The Release Manager role

Release Managers (RMs) perform the task of maintaining the master branch of Koha, which becomes the next major release of Koha every 6 months. The RM does this by applying bugfixes and enhancements from the community Bugzilla that have been set to "Passed QA" by a member of the QA team.

The role of Release Manager is decided by election after each major release. The newly elected RM will decide what features and bug fixes make it into the next major release of Koha.

A new beginning

The RM will begin his tenure by setting the database minor version to the next number. For example, if the past RM just released 3.22.00.00, you as the new RM will commit a new db version of 3.23.00.000 to denote the beginning of the next unstable version of Koha.

Guidelines on pushing this commit to master are found below in Pushing to Master.

Here is an example of the code in updatedatabase.pl:

$DBversion = "3.23.00.000";
if ( CheckVersion($DBversion) ) {
    print "Upgrade to $DBversion done (The year of the monkey will be here soon.)\n";
    SetVersion ($DBversion);
}

The choice of cheeky comment is left to the new RM.

You will also want to set the last stable branch to 'protected' on gitlab

Daily maintenance workflow

Release Managers start by checking the list of bugs in the community Bugzilla that have "Passed QA". Typically an RM will start by pushing bug fixes before pushing new features. It is traditional for a new RM to push only bug fixes during the first month of the RM's tenure, so that these fixes can easily be backported to the current stable release of Koha.

Applying the patch

Once the RM has selected a good candidate bug for inclusion in master, the RM will apply that bug's patches to a local copy of master. The 'git qa' alias found in Tips and tricks will help an RM do this quickly and easily.

If there is a conflict, You will have to options

  • Solve the conflict
  • Try to solve the conflict and give up

If solving the conflict is not straightforward, set the status of the bug to "Patch doesn't apply" so the author can rebase the patch.

Checking for regressions

Next, you should run some tests. For example, if the patch modifies several templates, you should at least run:

perl xt/tt_valid.t
perl xt/author/valid-templates.t

Do some testing, and keep in mind you should be confident in the QA team.

Updating the DB Revision

Move the db update and set the version

If the patch set makes changes to Koha's database data or database schema, the RM will need to make a DBrev commit.

First, note where the database update is. If it is in a file in atomicupdates, the RM will need to create a new db update in updatedatabase.pl and delete the atomicupdate file ( via git rm ).

Next, the db version needs to be set for this database update. A db version looks like this: 3.23.00.015

3 is the major version number, 23 is the minor version number ( master always has an odd minor version to denote it is unstable ), the 00 is the patch version ( master will always be 00 here ) and the 015 represents the db version.

The RM will also need to set this same db version in Koha.pm

Update the schema files

If the db update changes the database schema, the RM will want to update Koha's Schema files using the script misc/devel/update_dbix_class_files.pl. In kohadevbox, this can be achieved by running the following command as the vagrant user:

$ dbic

This deletes/creates an empty database called dbic in which it loads the kohastructure.sql file. It then uses it to regenerate the schema files.

Warning: The RM should ensure the db schema of his or her test system matches the community schema exactly. Failure to do so will introduce improper Schema files into the community codebase. It can be done like this:

$ mysql -uroot -e 'DROP DATABASE IF EXISTS dbic; CREATE DATABASE dbic'
$ cd /home/vagrant/kohaclone
$ mysql -uroot dbic < installer/data/mysql/kohastructure.sql
$ misc/devel/update_dbix_class_files.pl --db_name dbic --db_user root

Commit the changes

Commit the changes you have made, and give the commit a description such as:

Bug 15443: DBrev 3.23.00.015

Sign the patches

Once everything is solved, and testing proves everything is fine with the applied patches, you should sign all the new commits.

This git alias will allow you to just do git s <number of commits>

s = "!f() { c=`expr $1 - 1`; git filter-branch -f --msg-filter \"cat && echo \\\"\nSigned-off-by: Kyle M Hall <kyle@bywatersolutions.com>\\\"\" HEAD~$c^..; }; f"

Pushing to master

Once everything looks ready to go, it's time to push the new commits to master

git push gitmaster@git.koha-community.org:koha.git HEAD:master

Create a remote for pushing

You could also set a remote for pushing like this:

git remote add koha_gitmaster gitmaster@git.koha-community.org:koha.git

and then push using this shorter version:

git push koha_gitmaster HEAD:master

Update Bugzilla

This is very important: let people know you pushed the patches to master. Add a comment on the relevant bugs like this "This patch has been pushed to master, thanks $author" and change the status to Pushed to Master.

Dependencies changes

It is not uncommon that a patchset adds new dependencies. This is done by adding entries to the C4/Installer/PerlDependencies.pm file. When this happens, the debian/control file needs to be updated to reflect this situation.

The debian/control file is created out of the debian/control.in file, by the debian/update-control script. This script relies on the output of the apt-file command. So dependencies need to be installed using packages, and on a proper repository for it to work [1].

The update-control script needs to be run on Debian 7. The best way to do it, is inside a VM, and probably on kohadevbox. To do it:

vagrant up wheezy
vagrant ssh wheezy
sudo apt-file update
cd kohaclone
debian/update-control 

And then check the changes look correct. And create a proper commit with the changes.

[1] If a dependency is not packaged on a repo (Debian's or Koha's repositories), the Packaging Manager needs to be requested to do it.

Releasing

Schedule

See Release maintenance - Releasing as the steps for Release Maintainers and Release Managers are very similar where it concerns the release process!

Additional tools

During 17.11 and 18.05 release cycle, Jonathan Druart added few tricks to simplify and enhance RM workflow.

git pre-push hook

Add the following content to your .git/hooks/pre-push file:

  1. #!/usr/bin/perl
  2.  
  3. use Modern::Perl;
  4. use List::MoreUtils qw( any );
  5.  
  6. my $RELEASE = '17.12'; # FIXME Adapt this
  7. my $REMOTE = 'RM'; # FIXME Adapt this - it is the remote name you use to push
  8. my $RM = q|Jonathan Druart <jonathan.druart@bugs.koha-community.org>|; # FIXME Adapt this
  9. my $remote = $ARGV[0];
  10. my $url = $ARGV[1];
  11. if ( $remote eq 'RM' ) {
  12.     my $current_branch = `git branch | grep \\* | cut -d ' ' -f2`;
  13.     chomp $current_branch;
  14.     if ( $current_branch ne 'master' ) {
  15.         say 'Current branch is not master';
  16.         exit 1;
  17.     }
  18.     my @commits = `git rev-list $REMOTE/master..HEAD`;
  19.     unless ( @commits ) {
  20.         say 'Hum... no commit to push?';
  21.         exit 1;
  22.     }
  23.     my @errors;
  24.     for my $commit ( @commits ) {
  25.         my $commit_message = `git log --format=%s -1 $commit`;
  26.         if ( $commit_message !~ m|^Bug\s\d{4,5}: | ) {
  27.             push @errors, "Do not start with 'Bug XXXXX: ' - $commit";
  28.         }
  29.         if ( $commit_message =~ m|DBRev|i and not $commit_message =~ m|DBRev $RELEASE|) {
  30.             warn $commit_message;
  31.             push @errors, "DBRev is wrong, should start with 'DBRev $RELEASE'";
  32.         }
  33.         my $body = `git log --format=%b -1 $commit`;
  34.         $body =~ s|\n\s*|\n|g;
  35.         if ( $body !~ m|\n?Signed-off-by: $RM| ) {
  36.             push @errors, "No Signed-off-by line - $commit";
  37.         }
  38.     }
  39.  
  40.     my @atomicupdate_files = `git show HEAD:installer/data/mysql/atomicupdate/|tr -s '\\n' | grep -v '^tree'|grep -v 'README'|grep -v skeleton.perl`;
  41.     chomp for @atomicupdate_files;
  42.     for my $atomic ( @atomicupdate_files ) {
  43.         push @errors, "Atomicupdate file exists - $atomic";
  44.     }
  45.     if ( @errors ) {
  46.         say for @errors;
  47.         exit 1;
  48.     }
  49. }
  50. exit 0;

It will prevent:

  • to push a local branch that is not named "master"
  • to push commits that are not signed off by the RM
  • to push commits that do not start with "Bug XXXXX: "
  • to push DBRev commits that do not correctly formatted ("DBRev $RELEASE")
  • to push atomicupdate files

Mark bug report as pushed to master

You will need to install and setup pybugz: https://wiki.koha-community.org/wiki/Pybugz_configuration

I have modified bugz to support --pushed, see the commit on my github


The following script will update the status of the bug reports that have just been pushed to master.

My workflow is:

  1. git remote -v
  2. # RM      ssh://gitmaster@git.koha-community.org/koha.git (fetch)
  3. # RM      ssh://gitmaster@git.koha-community.org/koha.git (push)
  4. # origin  git://git.koha-community.org/koha.git (fetch)
  5. # origin  git://git.koha-community.org/koha.git (push)
  6.  
  7. git remote update # origin and RM are in sync
  8. git checkout -B master RM/master
  9. git bz apply XXXXX # Apply patches
  10. # Review and test patches
  11. git so X # Sign off patches
  12. git push RM master # Will be rejected if the git hook catches something wrong
  13. perl ../rm-tools/mark_as_pushed.pl

Here is the content of mark_as_pushed.pl:

  1. use Modern::Perl;
  2. use IO::Prompt;
  3.  
  4. my $comment = q|Pushed to master for 18.05, thanks to everybody involved!|; # FIXME Adapt this
  5. my @bug_numbers = `git log origin/master..HEAD --oneline | perl -lape 's/^\\S*\\sBug\\s(\\d+).*/\$1/' | sort | uniq`;
  6. chomp for @bug_numbers;
  7.  
  8. my @existing_authors = `git log origin/master --format="%aN" --reverse|sort|uniq`;
  9. chomp for @existing_authors;
  10. my @current_authors = `git log origin/master..HEAD --format="%aN" --reverse|sort|uniq`;
  11. chomp for @current_authors;
  12.  
  13. my @new_authors = array_minus( \@current_authors, \@existing_authors );
  14.  
  15. if ( @new_authors ) {
  16.     say "\n\n========= NEW AUTHORS ===========";
  17.     say "\t $_" for @new_authors;
  18.     say "=================================\n\n";
  19. }
  20.  
  21. say "Going to mark the following bug as 'Pushed to master'";
  22. for my $bn ( @bug_numbers ) {
  23.     unless ( $bn =~ m|^\d+$| ) {
  24.         say "WARNING - Bug '$bn' is not a valid bug number, skipping";
  25.         next;
  26.     }
  27.     say "- $bn";
  28. }
  29. my $mark_as_pushed = prompt('OK?: ');
  30. exit 0 unless $mark_as_pushed eq 'yes';
  31. print "\n";
  32.  
  33. for my $bn ( @bug_numbers ) {
  34.     unless ( $bn =~ m|^\d+$| ) {
  35.         say "WARNING - Bug '$bn' is not a valid bug number, skipping";
  36.         next;
  37.     }
  38.  
  39.     my $bugz_output = `bugz --connection Koha --skip-auth get -n $bn`;
  40.     my ($status)   = ($bugz_output =~ /\nStatus *: (.*)/);
  41.     if ( $status ne 'Passed QA' ) {
  42.         say "========= ERROR ================";
  43.         say "   Bug $bn is not Passed QA";
  44.         say "================================";
  45.         next;
  46.     }
  47.     say "Marking bug $bn as pushed to master";
  48.     # --pushed is a modified version of bugz
  49.     # Cannot use bugz from debian repo
  50.     # Use 0.14 from git, with python3
  51.     my $c = `bugz --connection Koha modify $bn --pushed -c "$comment"`;
  52.     say $c; # Search for 'Error:'
  53. }
  54.  
  55. sub array_minus { # From Array::Utils
  56.     my %e = map{ $_ => undef } @{$_[1]};
  57.     return grep( ! exists( $e{$_} ), @{$_[0]} );
  58. }

Generate the pre-release notes

I have defined a zsh function release_notes

  1. release_notes () {
  2.         if test $# -ne 2
  3.         then
  4.                 echo "Usage: release_notes commit_from commit_to (`release_notes v17.11.00 HEAD`)"
  5.                 return
  6.         fi
  7.         from=$1
  8.         to=$2
  9.         export koha_release_notes_dir=../koha-tools/koha-release-notes
  10.         ../koha-tools/release-tools/bin/koha-release $from..$to notes
  11.         koha_version=`perl -MKoha -e "print Koha::version();"`
  12.         commit_id=`git rev-parse --short $to`
  13.         commit_message="$koha_version $commit_id"
  14.         mv "misc/release_notes/`ls -tr misc/release_notes/ | tail -n 1`" $koha_release_notes_dir
  15.         git -C $koha_release_notes_dir commit -a -m"$commit_message"
  16. }

You will certainly want to edit some paths.

You may need to install the following perl modules: sudo apt install libtemplate-perl libwww-mechanize-perl libyaml-syck-perl libjson-perl libyaml-perl librest-client-perl libtext-multimarkdown-perl libhtml-tableextract-perl

When something is pushed to master, use it as
release_notes v17.11.00 HEAD

It will generate the release notes from 17.11.00 to HEAD (HEAD should be your current master branch).

A new commit will be added to the koha-release-notes repository For the first commit each release cycle you will need to add the file (it will be untracked) and commit before pushing. After that the commit should be automatic (you still need to push)

To make it works, you will need to clone the [git://git.koha-community.org/release-tools.git release-tools repo], edit etc/config.yaml with the new release team, and fill the kohadocs entry with a local path to the koha-manual repo Note: While running the rolling release notes the version for the team needs to be set to dev version - i.e. 18.06 for 18.11

Personal tools