EPrints Technical Mailing List Archive

See the EPrints wiki for instructions on how to join this mailing list and related information.

Message: #05631

< Previous (by date) | Next (by date) > | < Previous (in thread) | Next (in thread) > | Messages - Most Recent First | Threads - Most Recent First

Re: [EP-tech] Eprints-tech Digest, Vol 91, Issue 43

Hi Martin,
The deletion of older files is definitely a good idea, I'm looking at that now.
Not sure about the --generate menus option, but I'm going to try it out with our Eprints test repository. 

Thanks!

Verena


> Date: Wed, 27 Apr 2016 11:18:12 +0200
> From: martin.braendle@id.uzh.ch
> Subject: [EP-tech] Antwort: Re: Problems with view generation: EPrints
> 	System Error
> To: eprints-tech@ecs.soton.ac.uk
> Message-ID:
> 	<OF8739E530.67C3E499-ONC1257FA2.0032CB1D-C1257FA2.00331B1A@lotus.uzh.ch>
> 	
> Content-Type: text/plain; charset="iso-8859-1"
>
>
> Hi Verena,
>
> did you try out the --generate menus  option of generate_views? This
> reduces the number of files considerably.
>
> Also, in our nightly cron job, we delete all files that are older than 24
> hours.
>
> Best regards,
>
> Martin
>
> --
> Dr. Martin Br?ndle
> Zentrale Informatik
> Universit?t Z?rich
> Stampfenbachstr. 73
> CH-8006 Z?rich
>
> mail: martin.braendle@id.uzh.ch
> phone: +41 44 63 56705
> fax: +41 44 63 54505
> http://www.zi.uzh.ch
>
>
>
> Von:	Verena Mattes <verena.mattes@ub.uni-bayreuth.de>
> An:	eprints-tech@ecs.soton.ac.uk
> Datum:	27/04/2016 10:54
> Betreff:	Re: [EP-tech] Problems with view generation: EPrints System
>              Error
> Gesendet von:	eprints-tech-bounces@ecs.soton.ac.uk
>
>
>
> Hi Adam,
>
> For now, my colleagues in the IT service centre solved the problem by
> raising the number of files permitted in one directory on our Netapp,
> but that is only a temporary solution. Almost 340.000 files in one
> directory are just too many, so I'll have to find a way to split the
> person view into groups somehow.
>
> Your suggestion of checking the number of variations should be a first
> step in reducing the number of files. Here's our current view
> configuration, which I'm going to change by taking away the DEFAULT
> variation:
>
>>         {
>>                  id => "person",
>>                  hideempty => 0,
>>                  allow_null => 0,
>>                  menus => [
>>                          {
>>                                  fields => [ "person_view_name" ],
>>                                  mode => "sections",
>>                                  grouping_function =>
> "EPrints::Update::Views::group_by_first_character",
>>                                  group_range_function =>
> "EPrints::Update::Views::cluster_ranges_40",
>>                                  new_column_at => [40],
>>                                  open_first_section => 1,
>>                          },
>>                          ],
>>                  order => "-date;res=year/title/publication/book_title",
>>                  hideup => 0,
>>                  nocount => 0,
>>                  notimestamp => 0,
>>                  include => 1,
>>                  variations => [
>>                          "type",
>>                          "date;truncate=4,reverse",
>>                          "DEFAULT",
>>                  ],
>>                  citation => "view",            # Views mit
> Volltext-Hinweis!
>>          },
>
> Thanks for all your help!
>
> Verena
>
>>
>> Message: 1
>> Date: Tue, 26 Apr 2016 08:35:49 +0000
>> From: Adam Field <Adam.Field@jisc.ac.uk>
>> Subject: Re: [EP-tech] Problems with view generation: EPrints System
>> 		 Error
>> To: "eprints-tech@ecs.soton.ac.uk" <eprints-tech@ecs.soton.ac.uk>
>> Message-ID: <C2A1398C-702E-4C79-B4CD-2BAD5987789F@jisc.ac.uk>
>> Content-Type: text/plain; charset="utf-8"
>>
>> My previous suggestion wasn't right and won't work exactly as expected on 
> reflection.
>>
>> What happens when you run generate_views from the command line?
>> Just how many files do you have in the directory on the hard disk?
>>
>> Can you paste in the browse view configuration?  It may be that turning
> of variations will solve this.
>>
>> [Jisc]<http://www.jisc.ac.uk/>
>>
>> Adam Field
>> SHERPA services analyst developer
>>
>>
>> From: <eprints-tech-bounces@ecs.soton.ac.uk<
> mailto:eprints-tech-bounces@ecs.soton.ac.uk>> on behalf of Verena Mattes
> <verena.mattes@ub.uni-bayreuth.de<mailto:verena.mattes@ub.uni-bayreuth.de>> 
>> Reply-To: "eprints-tech@ecs.soton.ac.uk<
> mailto:eprints-tech@ecs.soton.ac.uk>" <eprints-tech@ecs.soton.ac.uk<
> mailto:eprints-tech@ecs.soton.ac.uk>>
>> Date: Tuesday, 26 April 2016 07:48
>> To: "eprints-tech@ecs.soton.ac.uk<mailto:eprints-tech@ecs.soton.ac.uk>"
> <eprints-tech@ecs.soton.ac.uk<mailto:eprints-tech@ecs.soton.ac.uk>>
>> Subject: Re: [EP-tech] Problems with view generation: EPrints System
> Error
>>
>> Hi Alan and Adam,
>>
>> Thanks for your suggestions. We are running EPrints 3.3.11 and the free
>> space on our hard disk is not a problem. I checked with a colleague and
>> he confirmed my guess that the problem is caused by the number of files
>> in the directory, obviously the maximum number was reached last week and
>> no further files can be created.
>> Does anybody have a quick idea on how to divide files for one view, the
>> person view, to different directories? Would it be possible to define
>> different views for groups of letters, e.g. A-D or something like that?
>>
>> Thanks!
>>
>> Verena
>>
>>
>>
>> Date: Mon, 25 Apr 2016 10:58:47 +0000
>> From: Adam Field <Adam.Field@jisc.ac.uk<mailto:Adam.Field@jisc.ac.uk>>
>> Subject: Re: [EP-tech] Problems with view generation: EPrints System
>> Error
>> To: "eprints-tech@ecs.soton.ac.uk<mailto:eprints-tech@ecs.soton.ac.uk>"
> <eprints-tech@ecs.soton.ac.uk<mailto:eprints-tech@ecs.soton.ac.uk>>
>> Message-ID: <3D933692-5E6D-4B33-9499-1DD23E2B4C62@jisc.ac.uk<
> 3D933692-5E6D-4B33-9499-1DD23E2B4C62@jisc.ac.uk">mailto:3D933692-5E6D-4B33-9499-1DD23E2B4C62@jisc.ac.uk>>
>> Content-Type: text/plain; charset="utf-8"
>>
>> What specific version of EPrints are you running?  What is line 1550
> of /usr/share/eprints3/perl_lib/EPrints/Update/Views.pm
>>
>> Also, silly question, but how much free space do you have on your hard
> disk?
>>
>> [Jisc]<http://www.jisc.ac.uk/>
>>
>> Adam Field
>> SHERPA services analyst developer
>>
>>
>> From: <eprints-tech-bounces@ecs.soton.ac.uk<
> mailto:eprints-tech-bounces@ecs.soton.ac.uk><
> mailto:eprints-tech-bounces@ecs.soton.ac.uk>> on behalf of Verena Mattes
> <verena.mattes@ub.uni-bayreuth.de<mailto:verena.mattes@ub.uni-bayreuth.de>< 
> mailto:verena.mattes@ub.uni-bayreuth.de>>
>> Reply-To: "eprints-tech@ecs.soton.ac.uk<
> mailto:eprints-tech@ecs.soton.ac.uk><mailto:eprints-tech@ecs.soton.ac.uk>" 
> <eprints-tech@ecs.soton.ac.uk<mailto:eprints-tech@ecs.soton.ac.uk><
> mailto:eprints-tech@ecs.soton.ac.uk>>
>> Date: Monday, 25 April 2016 09:51
>> To: "eprints-tech@ecs.soton.ac.uk<mailto:eprints-tech@ecs.soton.ac.uk><
> mailto:eprints-tech@ecs.soton.ac.uk>" <eprints-tech@ecs.soton.ac.uk<
> mailto:eprints-tech@ecs.soton.ac.uk><mailto:eprints-tech@ecs.soton.ac.uk>> 
>> Subject: [EP-tech] Problems with view generation: EPrints System Error
>>
>> Hello,
>>
>> since last week, we've had problems with the view generation for our
>> author/person view. These problems specifically concern the views for
>> new author names, which are not generated, while the views for already
>> existing author names are updated.
>>
>> For each author name concerned, there is an entry in the apache error
> log:
>>
>> Error writing
> to /usr/share/eprints3/archives/ubt_eref/html/de/view/person/Kaiser=3AMario=3A=3A.export: 
>   File too large
>> ------------------------------------------------------------------
>>       at /usr/share/eprints3/perl_lib/EPrints/Update/Views.pm line 1550
>>              EPrints::Update::Views::output_files
> ('EPrints::Repository=HASH(0x7fb0663688d0)',
> '/usr/share/eprints3/archives/ubt_eref/html/de/view/person/Kai...',
> 'XML::LibXML::DocumentFragment=SCALAR(0x7fb06a7e5938)',
> '/usr/share/eprints3/archives/ubt_eref/html/de/view/person/Kai...',
> 'XML::LibXML::Element=SCALAR(0x7fb06a45f160)',
> '/usr/share/eprints3/archives/ubt_eref/html/de/view/person/Kai...',
> 'XML::LibXML::DocumentFragment=SCALAR(0x7fb06a7e5938)',
> '/usr/share/eprints3/archives/ubt_eref/html/de/view/person/Kai...',
> 'XML::LibXML::DocumentFragment=SCALAR(0x7fb06a668aa8)', ...) called
> at /usr/share/eprints3/perl_lib/EPrints/Update/Views.pm line 935
>>              EPrints::Update::Views::update_view_list
> ('EPrints::Repository=HASH(0x7fb0663688d0)',
> '/usr/share/eprints3/archives/ubt_eref/html/de/view/person/Kai...', 'de',
> 'EPrints::Update::Views=HASH(0x7fb06a6863f0)', 'ARRAY(0x7fb064b78c70)')
> called at /usr/share/eprints3/perl_lib/EPrints/Update/Views.pm line 259
>>              EPrints::Update::Views::update_view_file
> ('EPrints::Repository=HASH(0x7fb0663688d0)', 'de',
> '/view/person/Kaiser=3AMario=3A=3A.html',
> '/view/person/Kaiser=3AMario=3A=3A.html') called
> at /usr/share/eprints3/perl_lib/EPrints/Apache/Rewrite.pm line 513
>> EPrints::Apache::Rewrite::handler('Apache2::RequestRec=SCALAR 
> (0x7fb06a471710)') called at -e line 0
>>              eval {...} called at -e line 0
>>
>> In this specific case, the author's name is linked to only 2 EPrints
>> entries, so it is unlikely that this file would be particularly large.
>>
>> Does anybody have an idea concerning this? I would appreciate any help.
>>
>> Thanks!
>>
>> Verena