EPrints Technical Mailing List Archive

See the EPrints wiki for instructions on how to join this mailing list and related information.

Message: #03144

< Previous (by date) | Next (by date) > | < Previous (in thread) | Next (in thread) > | Messages - Most Recent First | Threads - Most Recent First

[EP-tech] Re: Browse pages

Hi Ian,

This is from an email around 13/10/2008, which I sent to the list, when we were having problems with the views taking a long time to process (this was back in eprint v2.x era) ...  we ended up getting the view generation back to sub 1 hour processing time... Basically, check your MySQL settings and tune it up, so you can eliminate MySQL as the bottleneck.

		We had a few simple views being created each, we added a new people view, which then subsorted by year, 
		this took the processing time up to almost 22hours (from 2hrs).

		Eventually the number of eprints in our system grew, and the script was taking 24+ hours, and cron fired 
		off a 2nd copy, before the first had finished.  We ended up getting load averages of 300+ !!! While both were 
		running concurrently.

		To fix this I fiddled the settings for mysql.  Increasing memory etc.

		The 128M values I used as 1/8 of system memory.. I have since increased it even more, but I recommend 
		keeping it below 50% of system memory.

		Inside MySQL as root/owner of the MySQL system Run 

		SET GLOBAL query_cache_size = 128M; SET GLOBAL key_buffer_size = 128M; SET GLOBAL table_cache = 128;


Hope this helps a bit.

Matt



-----Original Message-----
From: eprints-tech-bounces@ecs.soton.ac.uk [mailto:eprints-tech-bounces@ecs.soton.ac.uk] On Behalf Of Ian Stuart
Sent: Tuesday, 10 June 2014 10:57 PM
To: eprints-tech@ecs.soton.ac.uk
Subject: [EP-tech] Re: Browse pages

On 10/06/14 12:39, John Salter wrote:
> If you set up a cron job to regenerate the page twice a day (so it's 
> never older that a day), does that help things?
Unfortunately not..... because the generate_views takes several days to complete (160,000+ records, all multiple-authors - "several" is a BIG
number) then the view is out of date before its even finished!

:)

-- 

Ian Stuart.
Developer: ORI, RJ-Broker, and OpenDepot.org Bibliographics and Multimedia Service Delivery team, EDINA, The University of Edinburgh.

http://edina.ac.uk/

This email was sent via the University of Edinburgh.

The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
*** Archive: http://www.eprints.org/tech.php/
*** EPrints community wiki: http://wiki.eprints.org/
*** EPrints developers Forum: http://forum.eprints.org/


_____________________________________________________________
This email (including any attached files) is confidential and is for the intended recipient(s) only. If you received this email by mistake, please, as a courtesy, tell the sender, then delete this email.

The views and opinions are the originator's and do not necessarily reflect those of the University of Southern Queensland. Although all reasonable precautions were taken to ensure that this email contained no viruses at the time it was sent we accept no liability for any losses arising from its receipt.

The University of Southern Queensland is a registered provider of education with the Australian Government.
(CRICOS Institution Code QLD 00244B / NSW 02225M, TEQSA PRV12081 )