EPrints Technical Mailing List Archive

Message: #03990

< Previous (by date) | Next (by date) > | < Previous (in thread) | Next (in thread) > | Messages - Most Recent First | Threads - Most Recent First

[EP-tech] Re: DB slow down.

Hi there,

we're now running the indexer which is gradually clearing the events_queue table in the assumption that once that is down to size we'll get rid of the performance drain we were seeing and that we can then run the indexer on a regular basis to keep it down. That seems like a workable solution.

We can see the table reducing in size at a rate of about 30 records per minute. We're testing it on our development server which has a much smaller data set than our live service but at current speed it's going to take almost 6 days to complete. Assuming the same speed on our live service, it will take about 97 days to run.

What would happen if we just deleted the events_queue table?

You can see why I'm asking ....

Cheers,

Ray.


________________________________________
From: eprints-tech-bounces@ecs.soton.ac.uk <eprints-tech-bounces@ecs.soton.ac.uk> on behalf of Adam Field <af05v@ecs.soton.ac.uk>
Sent: 26 February 2015 10:00
To: eprints-tech@ecs.soton.ac.uk
Subject: [EP-tech] Re: DB slow down.

In theory, the indexer has morphed over the years into a general task queue, and so some of the things in it may not be text indexing.  I believe that the generation of thumbnails is handled this way.  Bazaar packages can use this to schedule the triggering of event plugins.

--
Adam


On 26 Feb 2015, at 09:16, Alan.Stiles <alan.stiles@open.ac.uk> wrote:

> Hmm - are all the actions (action field in the table) related to indexing?  I'm certainly no expert on the intricacies of it all, but I do know that, for instance, the IRUS plugin uses the event queue to record when it was unable to contact the main IRUS servers to report a download, so it can attempt to do so at a later date.
> I'll leave it to Adam or someone with a more in-depth knowledge of eprints events to advise as to whether you could simply clear the event queue, or whether that would cause you subsequent problems...
>
> Alan
>
> -----Original Message-----
> From: Ian Stuart [mailto:Ian.Stuart@ed.ac.uk]
> Sent: 26 February 2015 08:27
> To: eprints-tech@ecs.soton.ac.uk
> Subject: [EP-tech] Re: DB slow down.
>
> On 25/02/15 17:25, Alan.Stiles wrote:
>> My first thought is "is your indexer running?" as our event_queue table currently has 4 items in it - 4 rows.  These get removed as the indexer processes the items.
>> What is the oldest date in your table?  And can you do a count of the statuses in the table?
>
> We don't.... but then, we don't need to index anything as we have no text-searching (and why waste clock-cycles creating full-text searching we won't use)
>
> --
>
> Ian Stuart.
> Developer: ORI, RJ-Broker, and OpenDepot.org Bibliographics and Multimedia Service Delivery team, EDINA, The University of Edinburgh.
>
> http://edina.ac.uk/
>
> This email was sent via the University of Edinburgh.
>
> The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.
>
> *** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
> *** Archive: http://www.eprints.org/tech.php/
> *** EPrints community wiki: http://wiki.eprints.org/
> *** EPrints developers Forum: http://forum.eprints.org/
> -- The Open University is incorporated by Royal Charter (RC 000391), an exempt charity in England & Wales and a charity registered in Scotland (SC 038302). The Open University is authorised and regulated by the Financial Conduct Authority.
>
> *** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
> *** Archive: http://www.eprints.org/tech.php/
> *** EPrints community wiki: http://wiki.eprints.org/
> *** EPrints developers Forum: http://forum.eprints.org/


*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
*** Archive: http://www.eprints.org/tech.php/
*** EPrints community wiki: http://wiki.eprints.org/
*** EPrints developers Forum: http://forum.eprints.org/

-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.