EPrints Technical Mailing List Archive

See the EPrints wiki for instructions on how to join this mailing list and related information.

Message: #09691

< Previous (by date) | Next (by date) > | < Previous (in thread) | Next (in thread) > | Messages - Most Recent First | Threads - Most Recent First

Re: [EP-tech] Browse Views - People

Hi James,

The general assumption is you would not want to generate browse view pages on the fly each time, as they are computational expensive to generate.  So if you have multiple bots that crawl your browse views every day, then you will add load for every bot (or user) who requests these pages beyond the first.

There is no specific way of turning off caching for a particular browse view, at best you can turn down the time the pages are cached for but this will just ensure the page is regenerated more regularly rather than reduce the space these pages take on disk.  

When you run generate_views do you get this to just generate menu pages or do you also get it to generate listings?  Listings are what usually take up most of the space and with a large number of creators, trying to regenerate these through generate_views overnight may take too long and still be running into the work day.  So I normally advise only generating menus and allow listings to be generated on first request since the cache expired.  This reduces the overall load for the day, as sometimes the cache will have expired many days before the listing page is requested again.

I think my plan to help solve your issue would be to clear cached pages where the cache has already expired.  This is 24 hours by default.  If you run a find and delete cron job something like the following (substituting as applicable):

find EPRINTS_PATH/archives/ARCHIVE_ID/html/en/view/VIEW_ID/ -mindepth 1 -mtime +1 -delete

Then this should delete expired cache files and not increase the load on your server by adding to the number of pages that need to be generated on the fly.

Regards

David Newman

On 11/04/2024 11:44, James Kerwin wrote:
CAUTION: This e-mail originated outside the University of Southampton.
CAUTION: This e-mail originated outside the University of Southampton.
Hi All,

I was looking at our file storage usage yesterday and made a surprising discovery. The browse view for people/authors/creators (whichever you choose to call them) takes up a huge amount of storage space on our server.

Presumably because we have a lot of authors listed in EPrints. At any one time almost all of these pages are not in use. Short of disabling this view is there any sort of option to have these pages generated on-the-fly as users request them instead of all being premade? 

We're still on an old EPrints as we await a suitable time to switch over to the new server with a more recent version of EPrints.

Thanks,
James

*** Options: https://wiki.eprints.org/w/Eprints-tech_Mailing_List
*** Archive: https://www.eprints.org/tech.php/
*** EPrints community wiki: https://wiki.eprints.org/