EPrints Technical Mailing List Archive

See the EPrints wiki for instructions on how to join this mailing list and related information.

Message: #03681

< Previous (by date) | Next (by date) > | < Previous (in thread) | Next (in thread) > | Messages - Most Recent First | Threads - Most Recent First

[EP-tech] Re: load throttling strategy

Thanks Ian.  
RG

-----Original Message-----
From: eprints-tech-bounces@ecs.soton.ac.uk [mailto:eprints-tech-bounces@ecs.soton.ac.uk] On Behalf Of Ian Stuart
Sent: 15 December 2014 15:43
To: eprints-tech@ecs.soton.ac.uk
Subject: [EP-tech] Re: load throttling strategy

I confess I just throttled the server: in httpd.conf add

StartServers    2
MinSpareServers 1
MaxSpareServers 1
MaxClients    128

(I put mine right next to where I defined the user & group for the server)

On 15/12/14 15:35, Ian Stuart wrote:
> I bet he's suffering the problem I had: Google loads a 'view' page, 
> which has hit it's "out of date" time so rebuilds.
> In the mean time, another httpd catches the next google page load, 
> which has also hit its "out of date" time, so spawns ANOTHER page rebuild.
>
> It very quickly ramps up, and WILL swamp the server... and WILL crash 
> the server.
>
> On 15/12/14 15:21, sf2 wrote:
>> Which issue are you trying to solve by introducing such a delay?
>>
>> Seb
>>
>> On 15.12.2014 15:17, Gooding R.W. wrote:
>>
>>> Hi,
>>>
>>> does anyone have an easy way to throttle eprints website crawlers ?
>>>
>>> I'm wondering is something as a short delay for each ePrint accessed 
>>> - the delay being proportional to the current load ?  That's perhaps 
>>> a little blunt but might it be very easy to implement ?
>
>


-- 

Ian Stuart.
Developer: ORI, RJ-Broker, and OpenDepot.org Bibliographics and Multimedia Service Delivery team, EDINA, The University of Edinburgh.

http://edina.ac.uk/

This email was sent via the University of Edinburgh.

The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.


*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
*** Archive: http://www.eprints.org/tech.php/
*** EPrints community wiki: http://wiki.eprints.org/
*** EPrints developers Forum: http://forum.eprints.org/