Using multiple Inverted Files in ABCD

[Egbert de Smet]

Helpful

Up

0

Down

Not Helpful

::

With the use of full-text indexing, based on the indexing technique 8 (with some additional tools for HTML-stripping and extensive stopwords-lists), as done in the DUBCORE demo-database, we are running into problems with the lack of the incremental indexing option. Whereas a typical ‘metadata-only’ FST mostly finishes its process within seconds the full-text can take hours – we have experience with a collection of about 2000 e-books which takes almost 2 hours.

A solution to separate the indexing of the metadata (as incremental, immediate indexing) from the full-text, allowing therefore different work-flows, implies separate Inverted Files. This option exists already in CISIS as Bireme e.g. uses it in their iAH-OPAC. The [dbn].def files in their ‘index-definition’ section allow the definition of an alternate IF (in the subfield ^y) and its own prefix to be used replacing the ‘DATABASE’ placeholder for the default database. For the mx-tool a parameter invx= can be used, which requires a similar file to indicate names and prefixes. This parameter needs to be introduced into the indexing-commands of the Utilities-menu and somewhere (e.g. dr_path.def) the alternate names and prefixes need to be defined. To be seen is whether and how IsisScript (for the wxis-tool) can use this as the ABCD-indexes are built with it.

Topic