Using multiple Inverted Files in ABCD

Home Forums Suggestions for improvement Using multiple Inverted Files in ABCD

Give ideas, point out points of the system that can be improved. This forum is for creative minds.
  • Creator
  • #5016
    Egbert de Smet
    Not Helpful

    With the use of full-text indexing, based on the indexing technique 8 (with some additional tools for HTML-stripping and extensive stopwords-lists), as done in the DUBCORE demo-database, we are running into problems with the lack of the incremental indexing option. Whereas a typical ‘metadata-only’ FST mostly finishes its process within seconds the full-text can take hours – we have experience with a collection of about 2000 e-books which takes almost 2 hours.

    A solution to separate the indexing of the metadata (as incremental, immediate indexing) from the full-text, allowing therefore different work-flows, implies separate Inverted Files. This option exists already in CISIS as Bireme e.g. uses it in their iAH-OPAC. The [dbn].def files in their ‘index-definition’ section allow the definition of an alternate IF (in the subfield ^y) and its own prefix to be used replacing the ‘DATABASE’ placeholder for the default database. For the mx-tool a parameter invx= can be used, which requires a similar file to indicate names and prefixes. This parameter needs to be introduced into the indexing-commands of the Utilities-menu and somewhere (e.g. dr_path.def) the alternate names and prefixes need to be defined. To be seen is whether and how IsisScript (for the wxis-tool) can use this as the ABCD-indexes are built with it.

  • You must be logged in to reply to this topic.