MRenvoize/zebra

Making additional MARC Fields searchable

Note: This should be done in kohaclone and added to git to ensure the customisations carry across upgrades!

Background

The zebra configuration is made up four variables populated in different formats across of a series of interconnected files serving different purposes.

The variables are:

1. Marc field.
2. Zebra index name
3. Zebra index type
4. Z39.50 Attributes (See LOC or LOC BIB1 definitions)

The configuration files are:

1. ~/kohaclone/etc/zebradb/marc_defs/marc21/biblios/record.abs

   Links marc fields to zebra index names

2. ~/kohaclone/etc/zebradb/biblios/etc/bib1.att

   Links Z39.50 attributes to zebra index names (Documentation for bib1 is available at LOC, a long list of BIB1 definitions can be found here)

3. ~/kohaclone/etc/zebradb/ccl.properties

   Koha searchable indexes (to be searchable by Koha, the zebra index must appear in here, (Currently, QueryParser falls back to this too))

4. ~/kohaclone/etc/zebradb/pqf.properties

   SRU and Koha with QueryParser searchable indexes (to be searchable by SRU or Koha with QueryParser enabled, the zebra index must appear in here)

Supplementary Note: QueryParser uses pqf.properties for indexes, so enabling it requires all indexes be in that file as well as ccl.properties (QP Drops back to ccl on occasions)

Additional files for DOM...

1. ~/kohaclone/etc/zebradb/marc_defs/marc21/biblios/biblio-koha-indexdefs.xml

   Source for biblio-zebra-indexdefs.xsl

2. ~/kohaclone/etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl

   Plays the same role in DOM indexing as record.abs does for GRS1.

Search in Koha

For a marc field to be searchable in koha four things must be true.

1. The marc field must appear in ~/kohaclone/etc/zebradb/marc_defs/marc21/biblios/record.abs (Editing record.abs)
2. The associated zebra index must appear in ~/kohaclone/etc/zebradb/biblios/etc/bib1.att
3. The zebra index and associated z39.50 attribute and any shortnames for the search must appear in ~/kohaclone/etc/zebradb/ccl.properties
4. The zebra index must appear in the indexes array in ~/kohaclone/C4/Search.pm

Search in SRU

For a marc field to be searchable in SRU three things must be true.

1. The marc field must appear in ~/kohaclone/etc/zebradb/marc_defs/marc21/biblios/record.abs
2. The associated zebra index must appear in ~/kohaclone/etc/zebradb/biblios/etc/bib1.att
3. The zebra index and associated z39.50 attribute and any shortnames for the search must appear in ~/kohaclone/etc/zebradb/pqf.properties

DOM indexing

Applicable to either Koha or SRU search.

DOM usess ~/kohaclone/etc/zebradb/marc_defs/marc21/biblios/biblio-koha-indexdefs.xml and ~/kohaclone/etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl in place of GRS1's ~/kohaclone/etc/zebradb/marc_defs/marc21/biblios/record.abs.

Arguments have been made against using the make_zebra_dom_cfg_from_record_abs script to update an existing biblio-koha-indexdefs.xml. Because the script overwites biblio-koha-indexdefs.xml, some care must be taken to ensure that information is not lost from biblio-koha-indexdefs.xml. Careless use of this script will cause regressions. Depending on circumstance, it may be better to edit ~/kohaclone/etc/zebradb/marc_defs/marc21/biblios/biblio-koha-indexdefs.xml directly. Use your judgement about the relative benefits of keeping record.abs in sync vs. editing biblio-koha-indexdefs.xml. Be consistent with the way that indexes have been added before, and make sure that whichever approach that you take is adequately documented in the git logs so that those who come after you do not have to make guesses as to which file is authoritative.

Additional steps for DOM indexing...

1. Any time you edit ~/kohaclone/etc/zebradb/marc_defs/marc21/biblios/record.abs you must also update ~/kohaclone/etc/zebradb/marc_defs/marc21/biblios/biblio-koha-indexdefs.xml and ~/kohaclone/etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl
2. REGENERATE THE DOM CONFIGS.
3. Make a backup of the xml and xsl files above.
4. Generate new version of biblio-koha-indexdefs.xml:

   ~/kohaclone/misc/maintenance/make_zebra_dom_cfg_from_record_abs --input ~/kohaclone/etc/zebradb/marc_defs/marc21/biblios/record.abs --output ~/kohaclone/etc/zebradb/marc_defs/marc21/biblios/biblio-koha-indexdefs.xml
   

5. Unfortunately ~/kohaclone/etc/zebradb/marc_defs/marc21/biblios/record.abs and ~/kohaclone/etc/zebradb/marc_defs/marc21/biblios/biblio-koha-indexdefs.xml aren't kept in perfect sync in koha. Changes in biblio-koha-indexdefs.xml will be overwritten by make_zebra_dom_cfg_from_record_abs, and must be merged back in. Use your favorite diff/merge program to merge your backup of ~/kohaclone/etc/zebradb/marc_defs/marc21/biblios/biblio-koha-indexdefs.xml.
6. After the changes to the XML file have been merged, generate biblio-zebra-indexdefs.xsl

   /usr/bin/xsltproc ~/kohaclone/etc/zebradb/xsl/koha-indexdefs-to-zebra.xsl ~/kohaclone/etc/zebradb/marc_defs/marc21/biblios/biblio-koha-indexdefs.xml  >  ~/kohaclone/etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl
   

7. Run a diff between your backup copy of biblio-zebra-indexdefs.xsl and ~/kohaclone/etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl. There shouldn't be any changes aside from the indexes that you've added, but if there are, they will also need to be merged.

Finally: you should git add, git commit these changes to git and then run the koha update procedure. NOTE:Changing ANY of the above files will require a Zebra Server restart to take effect.

Further information can be found on the Koha Wiki here and here