Notifications and OPAC messaging

Editing Notifications and OPAC messaging

Koha has mechanism to send messages to patrons and staff user that includes the ability to use message templates and send emails. LibLime proposes to enhance messaging by adding the following:

• creating a messaging preferences tab in the OPAC so that patrons can control what types of messages they receive.
• enhance the letters template mechanism to allow header, footer, and repeating section templates.
• add SMS as a delivery mechanism and unify it with other delivery options.
• add support to send messages via an interface with a telephony system.
• add the option to retrieve messages via RSS feeds
• add a central queue for messages so that processes to generate and send messages can be separated.
• add a mechanism to inject messages from another source (such as a library class scheduling system).
• unifying all of the code that can send emails and other types of messages into a single set of modules.

In addition, several circulation message types would be added:

• event or class notices
• notices sent when an item just becomes overdue
• notices sent in advance of an item's due date.

Details for enhancements to notifications and OPAC messaging

Initial author: Daniel Sweeney (daniel.sweeney at liblime dot com)

Overview

Libraries want to send a variety of notices and alerts to patrons from Koha. Patrons want to be able to control how they receive these notifications and in some cases, whether or not they receive them at all.

This specification discusses the various general requirements for enhancements to Koha's notification system.

Pre-Existing Functionality

Koha provides the following notifications by default:

1. Overdue notices.
2. Acquisitions Claim.
3. Serial Routing List
4. New patron registration

Each of these is installed at system startup. Administrators can add more notices in the tools area. Notices are defined using a templating system that can interpolate single values into the text of the notice. The single values are drawn directly from the database—in general the templates substitution tokens have the form table.column and draw directly from the database. There is no way to include a sub-template or to repeat a section of the template. The templates also include a name and a message subject. These are stored in the letter table.

Each notice has a Koha module—one of Catalogue, Serials (Routing List), Claim Acquisition, Claim Serial Issue, or Members. This is stored in the module field in the letter table. This column does not seem to reference anything in the database.

Libraries may in some cases substitute or add new notice types to existing system functionality; for instance in the Notice/Status Triggers area it is possible to substitute or add a new Circulation notice to the overdue notice processing system. The Koha Module associated with the notice indicates whether to include a notice in a particular functional area.

Koha patron records contain one set of addresses per record. These addresses include a primary postal address, primary email, telephone and cell-phone information, an alternate postal address and alternate contact information. There is no way to deactivate addresses or have them be valid for a specified period of time.

As part of recent and planned work, the following are being added to Koha:

1. Item borrowed notice.
2. Item returned notice.
3. Courtesy notice.
4. Class about-to-begin notice. This will be an interface with a separate library class scheduling system; the interface will be generalized to allow messages from an arbitrary third-party system to be injected into Koha's messaging system.
5. Just-become-overdue notice. (This is distinct from the regular overdue notice.)

Many of the new notices are intended to be transmitted electronically, using email or SMS, rather then necessarily through postal mail. In particular, the new overdue notice is intended to be an immediate reminder that an item has gone overdue, while the regular overdue notice may go via postal mail and serves as a more ‘official’ notice.

Requirements / Description

Delivery mechanisms

1. Multiple delivery mechanisms may be defined and configured for notifications. At a minimum the following will be supported:
   1. Postal Mail.
   2. E-mail.
   3. SMS text messaging, either through an email gateway or an dedicated SMS gateway.
   4. Syndication feeds, using RSS or Atom.
   5. A telephone circulation system. This system may read prepared information out of the Koha database.
2. Delivery mechanisms will have different configuration points. Most of them will be associated with a specific address from the patron record. For instance, the SMS delivery mechanism will use the Mobile Phone element in the Patron record, and if this column is empty for a given patron, the patron will not be able to use this delivery mechanism.
3. The Postal Mail delivery mechanism is somewhat special—it is the default if other addresses are not available.
4. Most delivery mechanisms will use a specific address in the patron record, and will not be available for a specific patron if that patron does not have the appropriate address. (Syndication feeds are an exception to this rule—they are not associated with a specific address.)
5. Delivery mechanisms will be configured per branch in a multi-branch Koha installation.

Notification Configuration

1. Each notification will have its own configuration options. For instance administrators will be able to configure the Item Borrowed and Item Returned notifications so that they are sent for particular patron category/item type combinations.
2. As in previous releases, the module column in the letter table will determine whether particular notifications can be configured for particular purposes. (This implies that the module field does not reference just a list of Koha modules—it’s essentially a generic notice type.)
3. For a given notification, administrators will be able to choose a different template for each delivery mechanism. This implies that multiple templates will exist in the letters table, one for each delivery mechanism. (It’s possible that some delivery mechanisms use entirely different templates that the letters in the regular Koha system—for instance if they create non-textual output somewhere. The regular Koha system is sufficient for many purposes, though.)
4. The set of notifications in the system will be stored in the authorized_values table. This set of authorized values will establish the range of values in the letter.module column. (Alternately, a new table for notification types could be added, but Koha seems to use the authorized_values table for this kind of thing.)

Configuring patron access to notifications

1. Patrons will be able to configure some notifications in the public catalog; some notifications will only be configured by administrators. Administrators will decide this for each notification and for each patron category.
2. Patrons will be able to turn off some notifications entirely. Administrators will decide this for each notification and for each patron category.
3. A new tab would be added to the default template for patron empowerment to house the alert setup information
   1. When selected the user would be presented with the options to define the following:
      1. for each defined message type, whether the patron wants to receive it via SMS, email, or RSS, or any combination thereof.
      2. for certain message types, whether the patron wishes to receive a digest version of the message.
      3. mobile phone number for SMS messages
4. By default, email and rss messages would come in a digested format. SMS messages specified to be delivered in a digested format will only indicate the number of items due in ‘x’ days and does not include the titles of the items. Patrons must be allowed to enter/define a cell phone number that they wish to have the SMS messages sent to. This number can be distinct from any number in their patron record.

Notification configuration for a specific delivery mechanism.

1. For a given notification and delivery mechanism, administrators may decide whether it is possible to send the notification in digested mode.
2. Digest mode may not make sense for some combinations of notification and delivery mechanism—delivery mechanisms may specify that they do not support digest mode.

Patron Notification Configuration

1. If a specific notification can be configured by patrons, patrons may be allowed to:
   1. Select what delivery mechanisms they wish to use for the notification (they may select more than one.) Patrons must have the appropriate address for the notification to select it.
   2. Opt-out of the notification, if opting out is available for the specific notification type.
   3. Choose whether to receive the notification in “Digest” mode. In “Digest” mode multiple notifications will be batched together into one single communication.
2. Circulation staff will be able to set the delivery mechanisms for patrons just as patrons can. This will be part of a new new "Messaging tab".

Notification templates

1. Note that the below is a simple suggestion to provide the minimum functionality. If a better template solution already exists in the Perl community, then we should use it. The primary requirement is that the templates are able to handle repeated data for digested messages, for instance for overdue notices to be able to populate a static header and footer, with a repeated entry for each item that is overdue.
2. Each notification template will have access to a variety of fields from the database to populate the template form. These fields will be different for each notification.
3. Each notification template may have three subtemplates—a header, a body and a footer.
4. The header subtemplate will always be at the top of the message.
5. The body template will be placed after the header.
6. If a notification template is supporting digested output, then the body template will be repeated for each entry in the digest. For instance for overdue notices, the body will be repeated for each overdue item.
7. The footer template will be placed after the body.
8. The header and footer subtemplates may be reused for more than one notice, even in non-digest mode.

Syndication feeds

1. If syndication feeds are available as a delivery mechanism, and notifications are configured to support them, then the syndication feeds will be produced by notification templates like the ones that produce textual notices for email. The templates will produce items or entries in the feed in XML format.
2. A separate process will take each generated item or entry and place it in a syndication feed.
3. The feeds will use RSS 2.0.
4. For Syndication feeds, digesting does not make sense—the mechanism that builds the feed will take care of grouping the individual items or entries together into a feed. There are some dependencies within an RSS feed between different elements in the XML stream.
5. Items will be placed in the feed using the date that they are generated by the system—we will not re-sort them in the feed according to some internal date like the item due date. The data in each entry will indicate the due dates. (This is consistent with Atom feeds from Google Calendar, which seem to make some sense.)
6. Items will be removed from the feed after a specific length of time. This length of time will be set as part of the configuration of the delivery mechanism. The default will be 30 days.
7. Syndication feeds for all patron-specific notifications will require authorization using http authentication. The user name and password will be the same user name and password the patron uses to log in to the OPAC. (This is consistent with the Gmail Atom feeds, which most feed-readers can now use.)
8. It must be possible to configure Koha so that the feeds are only available on https (SSL-protected) connections. Some libraries may not be overly concerned about this, or may not have SSL certificates, so this will not be required.
9. Each notification will have its own syndication feed.

Logging outgoing messages

1. Some customers require that all outgoing messages be logged.
2. LibLime proposes to create a message queue. This queue would be used to:
   1. stage messages that have been prepared (e.g., by the overdue notices job)
   2. store messages for the staff log
   3. store messages for use by RSS feeds
   4. allow a generic job to be written that processes the message queue and sends the messages.
3. If an explicit message queue is used, staff should be able to see the contents of the message queue, but not edit it.
4. When patrons transactions are anonymized, corresponding entries in the message queue should be anonymized as well.

Repeatable email, telephone and mobile phone fields

1. The five patron ‘contact’ fields, Phone (home), Phone (work), Phone (cell), E-mail (home) E-mail (work) and fax, will become repeatable.
2. Staff may add new values to these fields, as well as deleting old values.
3. Patrons may add new values to these fields, but may not delete old values.
4. Patrons may not add duplicate addresses of the same type (although they may add duplicates of different types, for instance the same number for Phone (home) and Fax.)
5. When patrons have added new values for contact fields, the values the patron adds will be protected from being overwritten by the patron loader.
6. The patron loader will be able to add multiple fields for the 5 contact types by including more than one comma-separated value in the appropriate field in the CSV input file. (The comma-separated values must be appropriately escaped in the file.)
7. For delivery mechanisms that use these address types, patrons will be able to choose any active address of the given type as the desired contact for that delivery mechanism.
8. Administrators may decide for each patron category whether to allow patrons to add new addresses in the public catalog. This will be part of the pre-existing patron category setup.

Patron address and contact deactivation

1. It will be possible for staff to deactivate and reactivate any of the patron addresses, except the main postal address.
2. When an address is deactivated, it will not be used to send messages, and any messages that would be sent using that address will not be sent at all.
3. For repeatable addresses as above, individual addresses may be deactivated without deactivating all the addresses of that type.
4. If a particular address is deactivated, and the patron loader replaces it, it will still be deactivated (in other words the patron loader will not change the activation status of an address.)
5. Patrons will see that a particular address is deactivated in the public catalog on the patron page. They will not be able to reactivate the address.
6. If a patron tries to add an address to their patron record that is deactivated, Koha will prevent them. (This should be taken care of by the prevention of loading duplicate addresses of the same type above.)
7. In the “Tools” area, administrators will be able to upload a file of addresses of a given type to activate or deactivate. The file will be a simple text file with one address per line.
   1. For each line, all matching addresses of the given type will be activated or deactivated.
   2. If an address appears in the file but not in the database, a warning will be logged for the administrator to see after the job is complete.
   3. If an address appears in the file and matches more than one patron, a warning will be logged for the administrator to see after the job is complete. The information about all matching patrons will appear. The matching addresses will still be activated or deactivated.
8. Some delivery mechanisms can immediately tell if an address is invalid. Delivery mechanisms will be able to deactivate addresses on their own.
9. Other delivery mechanisms can’t always tell right away that an address is invalid, so administrators will have to gather the invalid addresses from wherever the delivery stack puts them and process them down to a text file to deactivate the addresses.

Security and Privileges

1. The “Delivery Mechanisms” area in Administration will be controlled using a single yes/no privilege—if administrators lack this privilege they will be unable to enter this area. That being said, since Delivery Mechanisms are branch-specific, administrator’s branch affiliations will also allow or prevent them from editing Delivery Mechanisms for specific branches.
2. Circulation staff may edit patron’s delivery mechanisms if they have sufficient privileges to edit the patron itself—there is no specific permission controlling the alert delivery mechanisms.
3. Circulation staff may edit patron addresses using the existing privileges.
4. Administrators will be able to edit whether a patron category can edit addresses in the public catalog based on existing security allowing access to patron categories.

Open Questions and notes

1. The above doesn’t really discuss non-Patron notices like the acquisitions claims.
2. Telephone gateways appear may become another delivery mechanism.
3. The syndication feeds have some interesting caching issues—it’s not clear whether Koha should hold on to particular entries in the feed or for how long. The feed itself does not know whether or not the patron has actually used it, or whether the items have been returned. We provide a configurable default of 30 days in the delivery mechanism.
4. Atom support for the feeds may be desirable, but we have experience with RSS feeds now so it may be faster to just go that way and keep Atom in mind.
5. It may make sense to add fallback functionality, where if, for instance, an SMS message fails, then we try to send using the email delivery method. This presents some problems—it’s hard to know consistently whether a delivery has succeeded or failed, and administrators will have to maintain the delivery mechanism hierarchy.