Release management
Note: this guidelines are work in progress, and release mangers have the right to change this to whatever they prefer. For now this will serve as a documentation place for the tasks involved in release management.
The Release Manager role
Release Managers (RMs) perform the task of maintaining the main branch of Koha, which becomes the next major release of Koha every 6 months. The RM does this by applying bugfixes and enhancements from the community Bugzilla that have been set to "Passed QA" by a member of the QA team.
The role of Release Manager is decided by election after each major release. The newly elected RM will decide what features and bug fixes make it into the next major release of Koha.
A new beginning
The RM will begin their tenure by setting the database minor version to the next number. For example, if the past RM just released 21.11.00.000, you as the new RM will commit a new db version of 21.12.00.000 to denote the beginning of the next unstable version of Koha.
Guidelines on pushing this commit to main are found below in Pushing to main.
See how to write new Database_updates. You can find example in those files:
ls installer/data/mysql/db_revs/|grep 00000.pl
The choice of cheeky comment is left to the new RM.
You will also want to add the new development team to the docs/team.yaml file and set a prospective release date too as part of the above commit.
You will also want to set the last stable branch to 'protected' on gitlab
Daily maintenance workflow
Release Managers start by checking the list of bugs in the community Bugzilla that have "Passed QA". Typically an RM will start by pushing bug fixes before pushing new features. It is traditional for a new RM to push only bug fixes during the first month of the RM's tenure, so that these fixes can easily be backported to the current stable release of Koha.
Applying the patch
Once the RM has selected a good candidate bug for inclusion in main, the RM will apply that bug's patches to a local copy of main. The 'git qa' alias found in Tips and tricks will help an RM do this quickly and easily.
If there is a conflict, You will have to options
- Solve the conflict
- Try to solve the conflict and give up
If solving the conflict is not straightforward, set the status of the bug to "Patch doesn't apply" so the author can rebase the patch.
Checking for regressions
Next, you should run some tests. For example, if the patch modifies several templates, you should at least run:
perl xt/tt_valid.t perl xt/author/valid-templates.t
Do some testing, and keep in mind you should be confident in the QA team.
Updating the DB Revision
Move the db update and set the version
If the patch set makes changes to Koha's database data or database schema, the RM will need to make a DBrev commit.
First, note where the database update is. If it is in a file in atomicupdates, the RM will need to move it to installer/data/mysql/db_revs and name it appropriately (given the new version).
Next, the db version needs to be set for this database update. A db version looks like this: 21.06.00.016
21 is the major version number, 06 is the minor version number (main always has an even minor version to denote it is unstable), the 00 is the patch version (main will always be 00 here) and the 016 represents the db version.
The RM will also need to set this same db version in Koha.pm
Update the schema files
Warning: The RM should ensure the db schema of his or her test system matches the community schema exactly. Failure to do so will introduce improper Schema files into the community codebase.
If the db update changes the database schema, the RM will want to update Koha's Schema files using the script misc/devel/update_dbix_class_files.pl. In kohadevbox, this can be achieved by running the following command as the vagrant user, in ```koha-testing-docker``` it's with the root user:
$ dbic
This deletes/creates an empty database called dbic in which it loads the kohastructure.sql file. It then uses it to regenerate the schema files.
Manual way:
$ mysql -uroot -e 'DROP DATABASE IF EXISTS dbic; CREATE DATABASE dbic' $ cd /home/vagrant/kohaclone $ mysql -uroot dbic < installer/data/mysql/kohastructure.sql $ misc/devel/update_dbix_class_files.pl --db_name dbic --db_user root
Commit the changes
Commit the changes you have made, and give the commit a description such as:
Bug 42424: DBRev 21.06.00.016
Sign the patches
Once everything is solved, and testing proves everything is fine with the applied patches, you should sign all the new commits.
This git alias will allow you to just do git s <number of commits>
s = "!f() { c=`expr $1 - 1`; git filter-repo --message-callback 'return message + b\"\\nSigned-off-by: Full Name <email>\"' --refs HEAD~$c^..; }; f"
You may need to install git filter-repo first.
Pushing to main
Once everything looks ready to go, it's time to push the new commits to main
git push gitmain@git.koha-community.org:koha.git HEAD:main
Create a remote for pushing
You could also set a remote for pushing like this:
git remote add koha_gitmain gitmain@git.koha-community.org:koha.git
and then push using this shorter version:
git push koha_gitmain HEAD:main
Update Bugzilla
This is very important: let people know you pushed the patches to main.
- Add a comment on the relevant bugs like this "This patch has been pushed to main, thanks $author"
- Change the status to Pushed to main
- Add the version to the Version(s) released in list
Dependencies changes
It is not uncommon that a patchset adds new dependencies. This is done by adding entries to the C4/Installer/PerlDependencies.pm file. When this happens, the debian/control file needs to be updated to reflect this situation.
The debian/control file is created out of the debian/control.in file, by the debian/update-control script. This script relies on the output of the apt-file command. So dependencies need to be installed using packages, and on a proper repository for it to work [1].
The update-control script needs to be run on Debian 7. The best way to do it, is inside a VM, and probably on kohadevbox. To do it:
vagrant up wheezy vagrant ssh wheezy sudo apt-file update cd kohaclone debian/update-control
And then check the changes look correct. And create a proper commit with the changes.
[1] If a dependency is not packaged on a repo (Debian's or Koha's repositories), the Packaging Manager needs to be requested to do it.
Releasing
Dependencies
Before each feature release, it would be nice to update any outdated libraries we include.
- Bug 17178 - Mottie Keyboard releases - Download the release, unpack it, replace the content of koha-tmpl/intranet-tmpl/lib/keyboard/ with the content of the 'dist' of the release and finally squash the files within /languages and /layouts (cat *.js > all.min.js)
Schedule
See Release maintenance - Releasing as the steps for Release Maintainers and Release Managers are very similar where it concerns the release process!
Additional tools
During 17.11 and 18.05 release cycle, Jonathan Druart added few tricks to simplify and enhance RM workflow.
git pre-push hook
There is a pre-push hook documented in the 'Tips & Tricks' page.
Mark bug report as pushed to main
You will need to install and setup pybugz: https://wiki.koha-community.org/wiki/Pybugz_configuration
I have modified bugz to support --pushed, see the commit on my github
The following script will update the status of the bug reports that have just been pushed to main.
My workflow is:
1git remote -v
2# RM ssh://gitmainr@git.koha-community.org/koha.git (fetch)
3# RM ssh://gitmain@git.koha-community.org/koha.git (push)
4# origin git://git.koha-community.org/koha.git (fetch)
5# origin git://git.koha-community.org/koha.git (push)
6
7git remote update # origin and RM are in sync
8git checkout -B main RM/main
9git bz apply XXXXX # Apply patches
10# Review and test patches
11git so X # Sign off patches
12git push RM main # Will be rejected if the git hook catches something wrong
13perl ../rm-tools/mark_as_pushed.pl
Here is the content of mark_as_pushed.pl:
1use Modern::Perl;
2use IO::Prompt;
3
4my $comment = q|Pushed to main for 18.05, thanks to everybody involved!|; # FIXME Adapt this
5my @bug_numbers = `git log origin/main..HEAD --oneline | perl -lape 's/^\\S*\\sBug\\s(\\d+).*/\$1/' | sort | uniq`;
6chomp for @bug_numbers;
7
8my @existing_authors = `git log origin/main--format="%aN" --reverse|sort|uniq`;
9chomp for @existing_authors;
10my @current_authors = `git log origin/main..HEAD --format="%aN" --reverse|sort|uniq`;
11chomp for @current_authors;
12
13my @new_authors = array_minus( \@current_authors, \@existing_authors );
14
15if ( @new_authors ) {
16 say "\n\n========= NEW AUTHORS ===========";
17 say "\t $_" for @new_authors;
18 say "=================================\n\n";
19}
20
21say "Going to mark the following bug as 'Pushed to main'";
22for my $bn ( @bug_numbers ) {
23 unless ( $bn =~ m|^\d+$| ) {
24 say "WARNING - Bug '$bn' is not a valid bug number, skipping";
25 next;
26 }
27 say "- $bn";
28}
29my $mark_as_pushed = prompt('OK?: ');
30exit 0 unless $mark_as_pushed eq 'yes';
31print "\n";
32
33for my $bn ( @bug_numbers ) {
34 unless ( $bn =~ m|^\d+$| ) {
35 say "WARNING - Bug '$bn' is not a valid bug number, skipping";
36 next;
37 }
38
39 my $bugz_output = `bugz --connection Koha --skip-auth get -n $bn`;
40 my ($status) = ($bugz_output =~ /\nStatus *: (.*)/);
41 if ( $status ne 'Passed QA' ) {
42 say "========= ERROR ================";
43 say " Bug $bn is not Passed QA";
44 say "================================";
45 next;
46 }
47 say "Marking bug $bn as pushed to main";
48 # --pushed is a modified version of bugz
49 # Cannot use bugz from debian repo
50 # Use 0.14 from git, with python3
51 my $c = `bugz --connection Koha modify $bn --pushed -c "$comment"`;
52 say $c; # Search for 'Error:'
53}
54
55sub array_minus { # From Array::Utils
56 my %e = map{ $_ => undef } @{$_[1]};
57 return grep( ! exists( $e{$_} ), @{$_[0]} );
58}
Generate the pre-release notes
I have defined a zsh function release_notes
1release_notes () {
2 if test $# -ne 2
3 then
4 echo "Usage: release_notes commit_from commit_to (`release_notes v17.11.00 HEAD`)"
5 return
6 fi
7 from=$1
8 to=$2
9 export koha_release_notes_dir=../koha-tools/koha-release-notes
10 ../koha-tools/release-tools/bin/koha-release $from..$to notes
11 koha_version=`perl -MKoha -e "print Koha::version();"`
12 commit_id=`git rev-parse --short $to`
13 commit_message="$koha_version $commit_id"
14 mv "misc/release_notes/`ls -tr misc/release_notes/ | tail -n 1`" $koha_release_notes_dir
15 git -C $koha_release_notes_dir commit -a -m"$commit_message"
16}
You will certainly want to edit some paths.
You may need to install the following perl modules: sudo apt install libtemplate-perl libwww-mechanize-perl libyaml-syck-perl libjson-perl libyaml-perl librest-client-perl libtext-multimarkdown-perl libhtml-tableextract-perl
When something is pushed to main, use it as
release_notes v17.11.00 HEAD
It will generate the release notes from 17.11.00 to HEAD (HEAD should be your current main branch).
A new commit will be added to the koha-release-notes repository For the first commit each release cycle you will need to add the file (it will be untracked) and commit before pushing. After that the commit should be automatic (you still need to push)
To make it works, you will need to clone the release-tools repo, edit etc/config.yaml with the new release team, and fill the kohadocs entry with a local path to the koha-manual repo Note: While running the rolling release notes the version for the team needs to be set to dev version - i.e. 18.06 for 18.11