EPrints Technical Mailing List Archive

See the EPrints wiki for instructions on how to join this mailing list and related information.

Message: #07871

< Previous (by date) | Next (by date) > | < Previous (in thread) | Next (in thread) > | Messages - Most Recent First | Threads - Most Recent First

Re: [EP-tech] generate_views

We've put a lot of energy into this problem over the years. I've
variously: overridden the default Search generation for some
problematic views with custom SQL queries, and added a wrapper around
the generate_views script that runs four instances in parallel (one
for each core on our server).  It still takes a couple of hours every
week, but it's no longer 12+ hours.

I'm not at work right now to get my hands on the code.  I'll see if I
can get to it tomorrow.

Cheers

On Wed, 5 Jun 2019 at 18:10, James Kerwin via Eprints-tech
<eprints-tech@ecs.soton.ac.uk> wrote:
>
> Hi All,
>
> The generate_views script runs on our EPrints server once a week (early hours of Saturday morning).
>
> It started on Saturday and is still running now. I'm confident that it's still generating views, there are just a lot of records to go through. Has anybody experienced anything similar or have suggestions on how I could decrease how long it takes?
>
> I think my main option is to get rid of some views (not too keen on that) or speak to our computing services department and ask for the server to be given a bit of a boost (likely to be rejected unless our EPrints instance is running so slowly it goes backwards).
>
> I suppose I could run the script every two weeks etc. but I already feel as though a week is quite a long time between the views being updated. Plus a minimum of 4 days is a long time for this script to be running.
>
> Does anybody have experience with this and any advice on what I could do?
>
> Thanks,
> James
> *** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
> *** Archive: https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.eprints.org%2Ftech.php%2F&amp;data=01%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C7d94c15d285a4701016c08d6e9944a87%7C4a5378f929f44d3ebe89669d03ada9d8%7C0&amp;sdata=0gt1oj49AZMu0EcuGJ21QoXFerVIwoI8moEo%2Fr5bsS8%3D&amp;reserved=0
> *** EPrints community wiki: https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwiki.eprints.org%2F&amp;data=01%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C7d94c15d285a4701016c08d6e9944a87%7C4a5378f929f44d3ebe89669d03ada9d8%7C0&amp;sdata=aRspX%2F%2F%2B0tSOapgeRxNFdEBpQxukNjpJucP3i0rzW80%3D&amp;reserved=0
> *** EPrints developers Forum: https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fforum.eprints.org%2F&amp;data=01%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C7d94c15d285a4701016c08d6e9944a87%7C4a5378f929f44d3ebe89669d03ada9d8%7C0&amp;sdata=nLKCKDnaXh2q8leKYYBD8RavYel25CVSrkY0P3Pw4X8%3D&amp;reserved=0



-- 
  Matthew Kerwin
  https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmatthew.kerwin.net.au%2F&amp;data=01%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C7d94c15d285a4701016c08d6e9944a87%7C4a5378f929f44d3ebe89669d03ada9d8%7C0&amp;sdata=i3Eu%2F5LanOaHYksNV5KKo%2B8y4aq1SASltLrgqR8iuIE%3D&amp;reserved=0