Circulation Rules Interface and Backend Revamp RFC

Rationale

The current circulation rules system in Koha has a few major issues:

The variety of limits and policies that can be set for each situation means that the rules editor scrolls horizontally on the widest of screens.
Often, a library wants to set a general policy for most situations and only override certain values (e.g., holds policy is the same for all item types, fines policy varies by branch).
The specificity rules (by specific/generic itemtype, branch and categorycode) are difficult to remember, and are not clearly shown in the interface. This has led to frequent confusion; see 8137 or 8366.

Proposed Solution

We propose solving these issues with a new, more flexible backend and a more intuitive interface on top.

Backend

To allow for more flexible specification of policy, a new table structure will be created. Instead of a single row per itemtype/branchcode/categorycode with columns for each policy value, we’ll use the following:

branchcode	categorycode	itemtype	rule_name	rule_value
Branch/itemtype/category, or `NULL` for all.			`loanlength`, `fine`, etc.	`14`, `0.05`, etc.

This means that the set of rules for a given situation may be spread across many different rows with different specificity. For instance, the rules for MPL/ADULT/BOOK may be spread across MPL/*/*, */ADULT/BOOK, etc. To deal with this, we will add a new API:

Right now, Koha selects groups of rules by the combination of branchcode, categorycode and itemtype. So, given these three criteria, we currently return all rules (loan length, fines, etc). The subroutine that does this is named GetIssuingRule.

The first iteration of the new solution will recreate this style of rule selection by fetching each rule separately, then combining them together and returning the result such that the code that requests the rules for a given combination will not know the underlying schema has changed at all.

The second iteration will identify all code calling the subroutine that fetches circulation rules, and instead of calling GetIssuingRule will fetch only the circulation rules it needs. Once this is complete, the subroutine GetIssuingRule will be removed altogether.

To migrate data to this new system, we will create a row for each column of each row of the current schema. That is, each row in issuingrules will become ~20 rows in the new table (for loanlength, fine, etc.). This is less than ideal, but we propose mitigation mechanisms below.

Frontend

Basic prototype:

This interface is intended to clearly show the specificity order and show only the rules that have been defined and are relevant.

Notes:

While the rules are specified in a table, much like the current interface, only the columns that have defined rules are actually shown. Others can be added using the "Choose rule..." dropdown. (This may be unneeded for the per-category and per-itemtype rules.)
Any cell whose value is inherited from a set of defaults is shown in italics. The user can specify a value by clicking on the cell.

Planned enhancements:

A dropdown to jump to a specified library.
A link next to each inherited value that takes you to the original definition.
A button next to each set value allowing the user to reset it to the inherited value.
Floating headers (like the system preferences editor), so the user knows which library they're currently editing rules for.
Help text, similar to the current UI.
A "Save" button.

Also, note the "Cleanup rules..." button. This would bring up an interface that allowed a librarian to deduplicate rules (if several itemtypes had the same loan length, for example, it would suggest copying that loan length to a default rule). As each existing row in the issuingrules table will create a large number of rules, this will be critical for making the new interface usable.

Overview

To give librarians a very quick way to ascertain what rule will be used for a given library / patron / item combination, a rules overview page will be added which will display a table of combinations, the resultant rule values, what rule the value came from, and a link to quickly get to that rule in the editor.

In this mockup we have only 3 libraries, 2 patron types, and 2 item types defined: Mockup image

The simplify this a bit more, we could retain the library selector and show the rules per library rather than one encompassing table.

Excerpted Notes from Koha General IRC Meeting 13 January 2016

   15:35 < cait> I think i am slightly in favour of a more table like thing ot be honest
   15:36 < cait> but that's me
   15:36 < khall> cait: we already have a table like thing ; ) The primary goal is to move to a more flexible approach that allows for 
                  simpler rule sets
   15:36 < cait> I'd like something where I can easily compare rules for different itemtypes or different categories
   15:36 < khall> this approach doesn't jive all that well with a table based approach
   15:36 < Joubu> I think we need oleonard :)
   15:36 < khall> cait: that's already on my list!
   15:37 < cait> i am not sure how that would work with the mock up
   15:37 < khall> I think a second screen that gives a matrix of types categories and libraries with what the rule will be ( and where 
                  that rule is coming from ) would be a good companion
   15:37 < andreashm> I have read the RFC... I'm a bit concerned that having a lot of issuing rules will make this new appoach hard to 
                      overview
   15:38 < andreashm> khall: that seems like a good approach
   15:38 < khall> andreashm: that is what the second screen I propose will take care of
   15:38 < khall> thanks!
   15:38 < barton> cait -- how would you feel about some sort of explicit 'compare circ rules' functionality?
   15:38 < cait> khall: could youmaybe mock up the second screen too?
   15:39 < cait> to get a better idea?
   15:39 < hbraum> my consortia is sponsoring this development, although i'm very much letting pianohacker and khall take the reins on 
                   this. we've currently got 1200 issuingrule lines, so they have a good test case for lots of rules already.
   15:39  * andreashm would like that too
   15:39 < khall> cait: yes, I can do that
   15:39 < andreashm> hbraum: ouch
   15:39 < hbraum> and looking at the current mockup, a second screen overview would be helpful, I agree.
   15:39 < cait> #action khall asked nicely to add another mock-up for an overview kind of page
   15:39 <@bag> yeah 1200 lines is crazy 
   15:40 < khall> The first step is to change the internals without changing the externals. If anyone has an issue with the proposed
                  schema please let us know asap
   15:40 < cait> it will be a bit harder to get a nice excel sheet out :)
   15:40 < cait> that's what we do now - because you can make it sortable
   15:40 < cait> if there is no longer a need for it... that might be not too bad
   15:40 < khall> this would be a table too, so very sortable
   15:41 <@rangi> first thing id do is create a script where you can pass a itemtype, categorycode and branch, and get back what the rule 
                  applied would be
   15:41 <@rangi> then you can change the internals
   15:41 <@rangi> run that again
   15:41 <@rangi> and prove it hasn't suddenly changed
   15:41 < khall> rangi: agreed, unit tests ++
   15:42 < barton> khall: I've got most of that written.
   15:42 <@rangi> that script could then have a web frontend put on it later, so librarians could choose from a drop down, and see what 
                  rule would be applied .. rather than having to read everything
   15:42 <@rangi> why was this book issued for 4 weeks .. oh it used this rule
   15:42 < khall> rangi: you're reading my mind ; ) We've discussed that idea
   15:42 < cait> #idea have a script that gives you the circ conditions for branch, itemtype and patron category entered
   15:42 <@rangi> 70% of support queuries gone :)
   15:42 < khall> I think that should definitely be built in
   15:42 <@bag> more time to signoff!!!
   15:43 < Joubu> bag: heh, we have found what you wanted
   15:43 < hbraum> rangi ha! no kidding.
   15:43 < barton> In design discussions, we talked about making the lookup available on the circ page too, just for reference.
   15:43 < cait> hbraum: nekls++ btw for sponsoring - much needed
   15:44 < cait> for libraries with only 150 and less rules too :)
   15:44 < cait> ok, any more ideas or can we move on?
   15:44 < cait> full agenda tonight, but i don't want to rush youtoo much :)
   15:44 < khall> going once
   15:44 < hbraum> yw! the overall development scope of the project will also help branch specific circ-related rules get applied through
                   this, right khall?
   15:45 < khall> hbraum: yes!
   15:45 < khall> it will hopefully be a grand unification theory for Koha circ rules