User:Victor Grousset - tuxayo:Production

General, not Koha specific

Check for encoding issues

# Cherche dans tout les fichiers traduits fr ceux qui ne sont pas en UTF-8 (quand il n'y a pas d'accents, c'est us-ascii qui est affiché)
# Attention, finalement il y a un risque de faux négatifs: https://stackoverflow.com/questions/11018967/how-can-i-be-sure-of-the-file-encoding/11021413#11021413
find ~/src/koha-tmpl/intranet-tmpl/prog/fr-FR ~/src/koha-tmpl/opac-tmpl/bootstrap/fr-FR -type f -exec file --mime-encoding {} \; | grep -v -E 'utf-8|: us-ascii'

Restore a column from a database (MariaDB)

When something went wrong T_T

-- might need to be ran with the db admin user
CREATE DATABASE koha_restore_items;

It might be needed to give permissions to koha DB user

-- db admin user
GRANT ALL PRIVILEGES ON koha_restore_items.* TO 'kohaadmin'@'localhost';

# extract one table from a dump
zcat koha_mercredi.sql.gz | sed -n -e '/CREATE TABLE.*`items`/,/CREATE TABLE/p' > items-TODO-DATE-AND-TIME.sql
# remove the last lines that relate to another table
# and remove "FOREIGN KEY" constraints
# and remove "SET character_set_client" 
vim items-TODO-DATE-AND-TIME.sql

mysql koha_restore_items < items-TODO-DATE-AND-TIME.sql

Copy data from backup table to production table.

UPDATE koha.items broken, koha_restore_items.items backup
SET broken.my_column = backup.my_column
WHERE broken.itemnumber = backup.itemnumber;

Cleanup

show databases;
drop database koha_restore_items;

rm items-TODO-DATE-AND-TIME.sql

Koha itself

Filtering logs

In less, find the longest queries

less koha-access_log

# Duration >= 10 seconds
/time="[0-9]{2,20}

# Duration >= 1 second
/time="[1-9].

# Apache (logs in microseconds by default)
# Duration >= 10 seconds
/time=[0-9]{8,20}
# Duration >= 100 seconds
/time=[0-9]{9,20}

Greping rotated logs

Details might be specific to one's infrastructure.

zgrep -E 'REPLACE BY ERROR MSG' `ls -t koha-access_log koha-access_log.{1..4}*`

Filtering staff access logs (Nginx)

To remove queries to js, css, etc

grep -vE '(tmpl/prog|tmpl/lib|tmpl/js|/public)' koha-access_log | less

Filtering all logs (Apache)

tail -f koha-access_log plack-error.log koha-opac-access_log koha-opac-error_log koha-error_log | grep -vE "tmpl/prog|tmpl/lib|tmpl/js|/public"

Basic circulation stats to know when it was broken

Number of issues per 5 min slices to spot when a problem started and broke at least partly the circ. To know where to start looking into web server/plack/git logs.

SELECT issuedate, count(*)
FROM issues
WHERE issuedate > '2018-06-19 9:00:00'
AND issuedate < '2018-06-19 16:00:00'
GROUP BY 
UNIX_TIMESTAMP(issuedate) DIV 300;

SELECT returndate, count(*)
FROM old_issues
WHERE returndate > '2018-06-19 9:00:00'
AND returndate < '2018-06-19 16:00:00'
GROUP BY 
UNIX_TIMESTAMP(returndate) DIV 300;

Check all the error logs to see if one just broke something after applying a patch e.g.

tail -f ~/var/log/*err*log

Check database structure integrity

- mysqldump -d my_koha_prod_db - injection kohastructure.sql in a locale - mysqldump -d locale_db - use meld on both

Checklist for performance issues

e.g. high responses times

 * running processes
 * web server logs for long response times, otherwise the issue is likely the network between the browser and Koha
 * server monitoring
 * search for reports ran: "Run%20this%20report" in the staff web logs
 * syslog/journalctl for cron or other suspicious stuff
 * search for batch modifications
 * action logs in Staff => Tools => Logs
 * OPAC web server logs for too intense crawling

dump a table

time mysqldump koha TABLE_NAME | gzip -3 > ~/dumps/TABLE_NAME-$(date --iso-8601).sql.gz

Grepping a database dump

To find all the occurrences of a data piece.

-- the "INTO" allow to know which from table that was.
zgrep "MY_DATA" ~/sysop/koha_backup.sql.gz | sed -e 's/(/\n(/g' | grep -E "(INTO|MY_DATA)" --color=auto | less -R

alternative

zcat /home/koha/dumps/koha-sample.sql.gz | sed --expression='s/),(/),\n(/g' | grep "'Version'"

Getting records which have data in a given subfield

SELECT biblionumber
FROM   biblio_metadata
WHERE  ExtractValue(metadata,'//datafield[@tag="210"]/subfield[@code="s"]') != ''
LIMIT  3;

Zebra

Check indexation Zebra

Search sn=<biblionumber>

Reindex Zebra

Only bibliographic records.

time ~/tools/zebra/rebuild_full.sh -b -s 1k

Full

bash -c "time ~/tools/zebra/rebuild_full.sh -b -a -s 1k" 2>&1 | tee ~/var/log/zebra-reindex-full.log