EPrints Technical Mailing List Archive

Message: #08475

< Previous (by date) | Next (by date) > | < Previous (in thread) | Next (in thread) > | Messages - Most Recent First | Threads - Most Recent First

[EP-tech] Antwort: Re: Help indexing phrases

CAUTION: This e-mail originated outside the University of Southampton.
Hi Phil,

in the final end, reverse indexes of standard search engines are single term based. This is a basic principle.

Xapian is pretty basic in this matter - more advanced search engines such as ElasticSearch offer field types such as "keyword" that allow to store multi-term _expression_ - in the end however, the Lucene backend also will store single terms in its reverse indexes.

Still, there is the difficulty how to identify a multi-term _expression_ within a bulk of text - this is usually the field of Natural Language Processing, and special tools and thesauri are needed.

Kind regards,

Martin


An: <eprints-tech@ecs.soton.ac.uk>, "Phil Stacey" <phil@buildvoc.co.uk>
Von: "David R Newman via Eprints-tech"
Gesendet von:
Datum: 25.01.2021 10:39
Betreff: Re: [EP-tech] Help indexing phrases

Hi Phil,

Unfortunately, I don't think this is possible.  I think you would need to create a new field that is an id multiple field and use this.  You could probably write a script to map from the uncontrolled keywords field into this new multiple id field.  However, even with this new field I am not sure how well Xapian would index these as individual multi-word terms.  Advanced search for this field should work as you require.  In 3.4.2 I introduced the Idci MetaField that is basically the same as the Id MetaField but that matches case-insensitively, this is useful for mathcing things like email addresses and usernames, where case does not usually make a functional difference.

I have been thinking how best to implement a keywords fields that is more effective across simple, advanced and faceted search, particularly for multi-word terms.  I have yet to conclude on a solution, as I need to better understand how Xapian indexing works to see if it can be setup to allow EPrints to effectively index multiple-word terms.

Regards

David Newman

On 25/01/2021 07:06, Phil Stacey via Eprints-tech wrote:
CAUTION: This e-mail originated outside the University of Southampton.
Using uncontrolled keywords field which has phrases separated by commas, like to index the whole phrase.

For example :-
evacuation lift, part b - fire safety, b5 access and facilities for the fire
service, fire risk assessment, residual risk, building safety, b4 external
fire spread, means of escape, principal works, health & safety strategy
https://eprints.buildvoc.co.uk/id/eprint/865/

Question how do I configure xapian or indexing.pl to index the whole phrase instead of the individual terms for example fire, safety, or building



Virus-free. www.avg.com