EPrints Technical Mailing List Archive

See the EPrints wiki for instructions on how to join this mailing list and related information.

Message: #05384

< Previous (by date) | Next (by date) > | < Previous (in thread) | Next (in thread) > | Messages - Most Recent First | Threads - Most Recent First

[EP-tech] Re: Antwort: indexer stalls

Il 03/02/2016 11:13, martin.braendle@id.uzh.ch ha scritto:


Hi Yuri,

- EPrints 3.1.2 or 3.1.12?



EPrints 3.1.2.1 (Chocolate-coated Coffee Bean)
- I don't think the large row counts of the two tables are a problem - such counts are quite common. - Our experience is that the indexer rather is slowed down by too many entries in the event table, especially if there are some which are ancient (several weeks or months old). See Manage records > Tasks .
I don't have such "Manage records > Tasks" (events has been introduced in 3.2?), but i think I can just empty the index_queue. The problem is that high load it produces. 


Removing old tasks will help a lot.
Why the number of entries in the event table should slow the indexer? :-/ it processes one by one. 



Best regards,

Martin

--
Dr. Martin Brändle
Zentrale Informatik
Universität Zürich
Stampfenbachstr. 73
CH-8006 Zürich

mail: martin.braendle@id.uzh.ch
phone: +41 44 63 56705
fax: +41 44 63 54505
http://www.zi.uzh.ch
Inactive hide details for Yuri ---03/02/2016 09:58:47---Hi all! I've a big problem with the eprints indexer (Eprint 3.1.2). IYuri ---03/02/2016 09:58:47---Hi all! I've a big problem with the eprints indexer (Eprint 3.1.2). I was 

Von: Yuri <yurj@alfa.it>
An: "EPrints.org Technical List" <eprints-tech@ecs.soton.ac.uk>
Datum: 03/02/2016 09:58
Betreff: [EP-tech]  indexer stalls
Gesendet von: eprints-tech-bounces@ecs.soton.ac.uk

------------------------------------------------------------------------



Hi all!

 I've a big problem with the eprints indexer (Eprint 3.1.2). I was
looking for a high load problem in our server and discovered that, even
with no http requests or other process running, the load was above 1.

The indexer was also at Ss state, so sleeping and log (with an increased
log level) showed it was not doing any operation in the past days. Then
I shut down the indexer and the high load disappeared.

I think the problem could be my eprint__rindex which has 33 millions
rows and eprint__index has 3 million rows. So I think what happens is
that mysql issue some kernel calls, because I can see that, when the log
stops to be updated with new operations), the wa (i/o waiting) is above
50% for a lot of time.

Then the indexer stop doing indexing and load is high (above 2 or 3)
with an high wa (I/O).

Now I've stopped the indexer and running it as the webserver user
(www-data) with --once and --notdeamon and stopping it when stalling.
Then I run it again and so on (the index_queue has still 4000 rows to
process).

Any idea on how to solve this problem?

Question: which part of Eprints now is responsible to add an entry in
the index_queue table?
*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
*** Archive: http://www.eprints.org/tech.php/
*** EPrints community wiki: http://wiki.eprints.org/
*** EPrints developers Forum: http://forum.eprints.org/



*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
*** Archive: http://www.eprints.org/tech.php/
*** EPrints community wiki: http://wiki.eprints.org/
*** EPrints developers Forum: http://forum.eprints.org/