Release management

From Koha Wiki
Jump to navigation Jump to search

Note: this guidelines are work in progress, and release mangers have the right to change this to whatever they prefer. For now this will serve as a documentation place for the tasks involved in release management.

The Release Manager role

The Release Managers (RM) performs the task of maintaining the main branch of Koha, which becomes the next major release of Koha every 6 months. The RM does this by applying patches for bug fixes and enhancements from Bugzilla that have been set to "Passed QA" by a member of the QA team. The newly elected RM makes the final decision about what bugs will make it into the release. They also set the schedule and announce the deadlines for the release.

The RM is elected some weeks before a major release to serve for the next release cycle.

A new beginning

The RM will begin their tenure by setting the database minor version to the next number. For example, if the past RM just released 24.11.00.000, the new RM will commit a new db version of 24.12.00.000 to denote the beginning of the next development version of Koha.

Guidelines on pushing this commit to main can be found below in Pushing to main.

See how to write new Database_updates. You can find example in those files: 

ls installer/data/mysql/db_revs/|grep 00000.pl

The choice of cheeky comment is left to the new RM.

You will also want to set the last stable branch to 'protected' on gitlab

Daily maintenance workflow

Release Managers start by checking the list of bugs in the community Bugzilla that have "Passed QA". Typically an RM will start by pushing bug fixes before pushing new features. It is traditional for a new RM to push only bug fixes during the first month of the RM's tenure, so that these fixes can easily be backported to the current stable release of Koha.

Applying the patch

Once the RM has selected a good candidate bug for inclusion in main, the RM will apply that bug's patches to a local copy of main. The 'git qa' alias found in Tips and tricks will help an RM do this quickly and easily.

If there is a conflict, You will have 2 options:

  • Solve the conflict if it's straight forward.
  • Set the status of the bug to "Patch doesn't apply" and ask the author of the patch for a rebase.

Checking for regressions

Next, you should run some tests. For example, if the patch modifies several templates, you should at least run: 

perl xt/tt_valid.t
perl xt/author/valid-templates.t

Do some testing, and keep in mind you should be confident in the QA team.

Updating the DB Revision

Move the database update and set the version

If the patch set makes changes to Koha's database data or database schema, the RM will need to make a DBrev commit.

  • Increment the Koha version number in Koha.pm. Example: $VERSION = "24.12.00.000"; changed to: $VERSION = "24.12.00.001";
  • The database is usually a file in the atomicupdates. Git move the file to the db_revs directory using the version number from Koha.pm: git mv installer/data/mysql/atomicupdate/bug_xxxx.pl installer/data/mysql/db_revs/241200001.pl
    • 24 is the first part of the major version number.
    • 12 is the second part of version number, while .05 and .11 denote stable versions, .06 and .12 denote development versions.
    • 00 is the minor release version. This won't ever change for main.
    • 001 represents the database version.
  • Test the database update with updatedatabase
  • Commit the new db_rev file and the change to Koha.pm.

Update the schema files

Warning: The RM should ensure the db schema of their test system matches the community schema exactly. Failure to do so will introduce improper Schema files into the community code base.

If the database update changes the database schema, the RM must update Koha's Schema files using the script misc/devel/update_dbix_class_files.pl.

In koha-testing-docker: 

$ dbic

This deletes/creates an empty database called dbic in which it loads the kohastructure.sql file. It then uses it to regenerate the schema files.

Manual way: 

$ mysql -uroot -e 'DROP DATABASE IF EXISTS dbic; CREATE DATABASE dbic'
$ cd /home/vagrant/kohaclone
$ mysql -uroot dbic < installer/data/mysql/kohastructure.sql
$ misc/devel/update_dbix_class_files.pl --db_name dbic --db_user root

Commit the changes

Commit the changes you have made, and give the commit a description such as:

Bug XXXX: DBRev 24.12.00.001

Sign the patches

Once everything is solved, and testing proves everything is fine with the applied patches, you should sign all the new commits.

This git alias will allow you to just do git s <number of commits> 

s = "!f() { c=`expr $1 - 1`; git filter-repo --message-callback 'return message + b\"\\nSigned-off-by: Full Name <email>\"' --refs HEAD~$c^..; }; f"

You may need to install git filter-repo first.

Pushing to main

Once everything looks ready to go, it's time to push the new commits to main 

git push gitmain@git.koha-community.org:koha.git HEAD:main

Create a remote for pushing

It's recommended to set 2 remotes, one for pushing to main and a separate one for pushing to main-security: 

git remote add upstream git@git.koha-community.org:Koha-community/Koha.git
git remote add security git@git.koha-community.org:Koha-community/security.git

And then push using this shorter version:

git push upstream
git push security <local-security-branch>:main-security

Update Bugzilla

This is very important: let people know you pushed the patches to main.

Use the release-tools to automate these steps:

  • Add a comment on the relevant bugs like this "This patch has been pushed to main, thanks $author"
  • Change the status to Pushed to main
  • Add the version to the Version(s) released in list
# Add the text you want to etc/user.yaml 
release-tools --shell
koha-push updatebz --range HASH..HEAD
koha-push updatebz --range HEAD~12..HEAD # last 12 commits

Dependency changes

Perl

It is not uncommon that a patch set adds new dependencies.

This is the workflow:

  • Patch writer
    • Explains why the new dependency is required and provides information how it was picked on the bug report.
    • Updates the entries in the ./cpanfile file.
    • Adds dependency keyword to Bugzilla.
    • Adds Packaging manager/team in CC in Bugzilla
  • Packaging manager/team
    • Is required to sign-off on the bug before it moves further.
    • Will take care of packaging if required and provide the packages via the koha-staging repository.
  • Release manager
    • Tests patches as usual.
    • Makes sure all required dependencies are installed.
    • Updates the debian/control file and verifies changes before commiting:
      ktd --shell
sudo apt-get install apt-file
apt-file update
debian/update-control
    • When this happens, the needs to be updated to reflect this situation. The debian/control file is created out of the debian/control.in file, by the debian/update-control script. This script relies on the output of the apt-file
  • Automatism..
    • Once the patches have been pushed to a branch, a rebuild of KTD will be automatically triggered. [FIXME]
    • Once the patches have been pushed to a branch, a job to send bugzilla a debian/control patch will be automatically triggered. [FIXME]

Yarn

If yarn.lock changes as result of changes to dependencies:

  • RM needs to generate and commit the yarn.lock file with yarn install.
  • ktd images need to be rebuilt. See: Pipeline on gitlab Reach out to @joubu or @tcohen when in doubt about this step.

Releasing

Dependencies

Before each feature release, it would be nice to update any outdated libraries we include.

  • Bug 17178 - Mottie Keyboard releases - Download the release, unpack it, replace the content of koha-tmpl/intranet-tmpl/lib/keyboard/ with the content of the 'dist' of the release and finally squash the files within /languages and /layouts (cat *.js > all.min.js)

Schedule

See Release maintenance - Releasing as the steps for Release Maintainers and Release Managers are very similar where it concerns the release process!

Additional tools

During 17.11 and 18.05 release cycle, Jonathan Druart added few tricks to simplify and enhance RM workflow.

git pre-push hook

There is a pre-push hook documented in the 'Tips & Tricks' page.

Mark bug report as pushed to main

You will need to install and setup pybugz: https://wiki.koha-community.org/wiki/Pybugz_configuration

I have modified bugz to support --pushed, see the commit on my github


The following script will update the status of the bug reports that have just been pushed to main.

My workflow is: 

 1git remote -v
 2# RM      ssh://gitmainr@git.koha-community.org/koha.git (fetch)
 3# RM      ssh://gitmain@git.koha-community.org/koha.git (push)
 4# origin  git://git.koha-community.org/koha.git (fetch)
 5# origin  git://git.koha-community.org/koha.git (push)
 6
 7git remote update # origin and RM are in sync
 8git checkout -B main RM/main
 9git bz apply XXXXX # Apply patches
10# Review and test patches
11git so X # Sign off patches
12git push RM main # Will be rejected if the git hook catches something wrong
13perl ../rm-tools/mark_as_pushed.pl

Here is the content of mark_as_pushed.pl: 

 1use Modern::Perl;
 2use IO::Prompt;
 3
 4my $comment = q|Pushed to main for 18.05, thanks to everybody involved!|; # FIXME Adapt this
 5my @bug_numbers = `git log origin/main..HEAD --oneline | perl -lape 's/^\\S*\\sBug\\s(\\d+).*/\$1/' | sort | uniq`;
 6chomp for @bug_numbers;
 7
 8my @existing_authors = `git log origin/main--format="%aN" --reverse|sort|uniq`;
 9chomp for @existing_authors;
10my @current_authors = `git log origin/main..HEAD --format="%aN" --reverse|sort|uniq`;
11chomp for @current_authors;
12
13my @new_authors = array_minus( \@current_authors, \@existing_authors );
14
15if ( @new_authors ) {
16    say "\n\n========= NEW AUTHORS ===========";
17    say "\t $_" for @new_authors;
18    say "=================================\n\n";
19}
20
21say "Going to mark the following bug as 'Pushed to main'";
22for my $bn ( @bug_numbers ) {
23    unless ( $bn =~ m|^\d+$| ) {
24        say "WARNING - Bug '$bn' is not a valid bug number, skipping";
25        next;
26    }
27    say "- $bn";
28}
29my $mark_as_pushed = prompt('OK?: ');
30exit 0 unless $mark_as_pushed eq 'yes';
31print "\n";
32
33for my $bn ( @bug_numbers ) {
34    unless ( $bn =~ m|^\d+$| ) {
35        say "WARNING - Bug '$bn' is not a valid bug number, skipping";
36        next;
37    }
38
39    my $bugz_output = `bugz --connection Koha --skip-auth get -n $bn`;
40    my ($status)   = ($bugz_output =~ /\nStatus *: (.*)/);
41    if ( $status ne 'Passed QA' ) {
42        say "========= ERROR ================";
43        say "   Bug $bn is not Passed QA";
44        say "================================";
45        next;
46    }
47    say "Marking bug $bn as pushed to main";
48    # --pushed is a modified version of bugz
49    # Cannot use bugz from debian repo
50    # Use 0.14 from git, with python3
51    my $c = `bugz --connection Koha modify $bn --pushed -c "$comment"`;
52    say $c; # Search for 'Error:'
53}
54
55sub array_minus { # From Array::Utils
56    my %e = map{ $_ => undef } @{$_[1]};
57    return grep( ! exists( $e{$_} ), @{$_[0]} );
58}

Generate the pre-release notes

I have defined a zsh function release_notes 

 1release_notes () {
 2        if test $# -ne 2
 3        then
 4                echo "Usage: release_notes commit_from commit_to (`release_notes v17.11.00 HEAD`)"
 5                return
 6        fi
 7        from=$1
 8        to=$2
 9        export koha_release_notes_dir=../koha-tools/koha-release-notes
10        ../koha-tools/release-tools/bin/koha-release $from..$to notes
11        koha_version=`perl -MKoha -e "print Koha::version();"`
12        commit_id=`git rev-parse --short $to`
13        commit_message="$koha_version $commit_id"
14        mv "misc/release_notes/`ls -tr misc/release_notes/ | tail -n 1`" $koha_release_notes_dir
15        git -C $koha_release_notes_dir commit -a -m"$commit_message"
16}

You will certainly want to edit some paths.

You may need to install the following perl modules:

sudo apt install libtemplate-perl libwww-mechanize-perl libyaml-syck-perl libjson-perl libyaml-perl librest-client-perl libtext-multimarkdown-perl libhtml-tableextract-perl

When something is pushed to main, use it as

release_notes v17.11.00 HEAD

It will generate the release notes from 17.11.00 to HEAD (HEAD should be your current main branch).

A new commit will be added to the koha-release-notes repository For the first commit each release cycle you will need to add the file (it will be untracked) and commit before pushing. After that the commit should be automatic (you still need to push)

To make it works, you will need to clone the release-tools repo, edit etc/config.yaml with the new release team, and fill the kohadocs entry with a local path to the koha-manual repo Note: While running the rolling release notes the version for the team needs to be set to dev version - i.e. 18.06 for 18.11

Release day actions/checks

https://tree.taiga.io/project/koha-ils/task/106

Retrieved from "https://wiki.koha-community.org/w/index.php?title=Release_management&oldid=35138"