EPrints Technical Mailing List Archive

Message: #05632

< Previous (by date) | Next (by date) > | < Previous (in thread) | Next (in thread) > | Messages - Most Recent First | Threads - Most Recent First

Re: [EP-tech] Eprints-tech Digest, Vol 91, Issue 43

Hi Verena,

Another thing to possibly look at:
You appear to have (accessible in the English version of the site)
http://eref.uni-bayreuth.de/view/person/Kaiser=3AMario.html,
http://eref.uni-bayreuth.de/view/person/Kaiser=3AMario=3A.html,
http://eref.uni-bayreuth.de/view/person/Kaiser=3AMario=3A=3A.html

and possibly more.  I don't know If your system is actually generating an HTML file for each of those variants, for each user, in both languages, but that may be part of the issue with the 300 000 + files?

Regards,
Alan

-----Original Message-----
From: eprints-tech-bounces@ecs.soton.ac.uk [mailto:eprints-tech-bounces@ecs.soton.ac.uk] On Behalf Of Verena Mattes
Sent: 27 April 2016 14:37
To: eprints-tech@ecs.soton.ac.uk
Subject: Re: [EP-tech] Eprints-tech Digest, Vol 91, Issue 43

Hi Martin,

The deletion of older files is definitely a good idea, I'm looking at that now.

Not sure about the --generate menus option, but I'm going to try it out with our Eprints test repository.

Thanks!

Verena


 > Date: Wed, 27 Apr 2016 11:18:12 +0200  > From: martin.braendle@id.uzh.ch  > Subject: [EP-tech] Antwort: Re: Problems with view generation: EPrints
 > System Error
 > To: eprints-tech@ecs.soton.ac.uk
 > Message-ID:
 > <OF8739E530.67C3E499-ONC1257FA2.0032CB1D-C1257FA2.00331B1A@lotus.uzh.ch>
 >
 > Content-Type: text/plain; charset="iso-8859-1"
 >
 >
 > Hi Verena,
 >
 > did you try out the --generate menus  option of generate_views? This  > reduces the number of files considerably.
 >
 > Also, in our nightly cron job, we delete all files that are older than 24  > hours.
 >
 > Best regards,
 >
 > Martin
 >
 > --
 > Dr. Martin Br?ndle
 > Zentrale Informatik
 > Universit?t Z?rich
 > Stampfenbachstr. 73
 > CH-8006 Z?rich
 >
 > mail: martin.braendle@id.uzh.ch
 > phone: +41 44 63 56705
 > fax: +41 44 63 54505
 > http://www.zi.uzh.ch
 >
 >
 >
 > Von:Verena Mattes <verena.mattes@ub.uni-bayreuth.de>
 > An:eprints-tech@ecs.soton.ac.uk
 > Datum:27/04/2016 10:54
 > Betreff:Re: [EP-tech] Problems with view generation: EPrints System
 >              Error
 > Gesendet von:eprints-tech-bounces@ecs.soton.ac.uk
 >
 >
 >
 > Hi Adam,
 >
 > For now, my colleagues in the IT service centre solved the problem by  > raising the number of files permitted in one directory on our Netapp,  > but that is only a temporary solution. Almost 340.000 files in one  > directory are just too many, so I'll have to find a way to split the  > person view into groups somehow.
 >
 > Your suggestion of checking the number of variations should be a first  > step in reducing the number of files. Here's our current view  > configuration, which I'm going to change by taking away the DEFAULT  > variation:
 >
 >>         {
 >>                  id => "person",
 >>                  hideempty => 0,
 >>                  allow_null => 0,
 >>                  menus => [
 >>                          {
 >>                                  fields => [ "person_view_name" ],
 >>                                  mode => "sections",
 >>                                  grouping_function =>
 > "EPrints::Update::Views::group_by_first_character",
 >>                                  group_range_function =>
 > "EPrints::Update::Views::cluster_ranges_40",
 >>                                  new_column_at => [40],
 >>                                  open_first_section => 1,
 >>                          },
 >>                          ],
 >>                  order => "-date;res=year/title/publication/book_title",
 >>                  hideup => 0,
 >>                  nocount => 0,
 >>                  notimestamp => 0,
 >>                  include => 1,
 >>                  variations => [
 >>                          "type",
 >>                          "date;truncate=4,reverse",
 >>                          "DEFAULT",
 >>                  ],
 >>                  citation => "view",            # Views mit
 > Volltext-Hinweis!
 >>          },
 >
 > Thanks for all your help!
 >
 > Verena
 >
 >>
 >> Message: 1
 >> Date: Tue, 26 Apr 2016 08:35:49 +0000  >> From: Adam Field <Adam.Field@jisc.ac.uk>  >> Subject: Re: [EP-tech] Problems with view generation: EPrints System
 >>  Error
 >> To: "eprints-tech@ecs.soton.ac.uk" <eprints-tech@ecs.soton.ac.uk>  >> Message-ID: <C2A1398C-702E-4C79-B4CD-2BAD5987789F@jisc.ac.uk>
 >> Content-Type: text/plain; charset="utf-8"
 >>
 >> My previous suggestion wasn't right and won't work exactly as expected on  > reflection.
 >>
 >> What happens when you run generate_views from the command line?
 >> Just how many files do you have in the directory on the hard disk?
 >>
 >> Can you paste in the browse view configuration?  It may be that turning  > of variations will solve this.
 >>
 >> [Jisc]<http://www.jisc.ac.uk/>
 >>
 >> Adam Field
 >> SHERPA services analyst developer
 >>
 >>
 >> From: <eprints-tech-bounces@ecs.soton.ac.uk<
 > mailto:eprints-tech-bounces@ecs.soton.ac.uk>> on behalf of Verena Mattes  > <verena.mattes@ub.uni-bayreuth.de<mailto:verena.mattes@ub.uni-bayreuth.de>>
 >> Reply-To: "eprints-tech@ecs.soton.ac.uk<  > mailto:eprints-tech@ecs.soton.ac.uk>" <eprints-tech@ecs.soton.ac.uk<  > mailto:eprints-tech@ecs.soton.ac.uk>>
 >> Date: Tuesday, 26 April 2016 07:48
 >> To: "eprints-tech@ecs.soton.ac.uk<mailto:eprints-tech@ecs.soton.ac.uk>"
 > <eprints-tech@ecs.soton.ac.uk<mailto:eprints-tech@ecs.soton.ac.uk>>
 >> Subject: Re: [EP-tech] Problems with view generation: EPrints System  > Error  >>  >> Hi Alan and Adam,  >>  >> Thanks for your suggestions. We are running EPrints 3.3.11 and the free  >> space on our hard disk is not a problem. I checked with a colleague and  >> he confirmed my guess that the problem is caused by the number of files  >> in the directory, obviously the maximum number was reached last week and  >> no further files can be created.
 >> Does anybody have a quick idea on how to divide files for one view, the  >> person view, to different directories? Would it be possible to define  >> different views for groups of letters, e.g. A-D or something like that?
 >>
 >> Thanks!
 >>
 >> Verena
 >>
 >>
 >>
 >> Date: Mon, 25 Apr 2016 10:58:47 +0000  >> From: Adam Field <Adam.Field@jisc.ac.uk<mailto:Adam.Field@jisc.ac.uk>>
 >> Subject: Re: [EP-tech] Problems with view generation: EPrints System  >> Error  >> To: "eprints-tech@ecs.soton.ac.uk<mailto:eprints-tech@ecs.soton.ac.uk>"
 > <eprints-tech@ecs.soton.ac.uk<mailto:eprints-tech@ecs.soton.ac.uk>>
 >> Message-ID: <3D933692-5E6D-4B33-9499-1DD23E2B4C62@jisc.ac.uk<
 > 3D933692-5E6D-4B33-9499-1DD23E2B4C62@jisc.ac.uk">mailto:3D933692-5E6D-4B33-9499-1DD23E2B4C62@jisc.ac.uk>>
 >> Content-Type: text/plain; charset="utf-8"
 >>
 >> What specific version of EPrints are you running?  What is line 1550  > of /usr/share/eprints3/perl_lib/EPrints/Update/Views.pm
 >>
 >> Also, silly question, but how much free space do you have on your hard  > disk?
 >>
 >> [Jisc]<http://www.jisc.ac.uk/>
 >>
 >> Adam Field
 >> SHERPA services analyst developer
 >>
 >>
 >> From: <eprints-tech-bounces@ecs.soton.ac.uk<
 > mailto:eprints-tech-bounces@ecs.soton.ac.uk><
 > mailto:eprints-tech-bounces@ecs.soton.ac.uk>> on behalf of Verena Mattes  > <verena.mattes@ub.uni-bayreuth.de<mailto:verena.mattes@ub.uni-bayreuth.de><
 > mailto:verena.mattes@ub.uni-bayreuth.de>>
 >> Reply-To: "eprints-tech@ecs.soton.ac.uk<  > mailto:eprints-tech@ecs.soton.ac.uk><mailto:eprints-tech@ecs.soton.ac.uk>"
 > <eprints-tech@ecs.soton.ac.uk<mailto:eprints-tech@ecs.soton.ac.uk><
 > mailto:eprints-tech@ecs.soton.ac.uk>>
 >> Date: Monday, 25 April 2016 09:51
 >> To: "eprints-tech@ecs.soton.ac.uk<mailto:eprints-tech@ecs.soton.ac.uk><
 > mailto:eprints-tech@ecs.soton.ac.uk>" <eprints-tech@ecs.soton.ac.uk<  > mailto:eprints-tech@ecs.soton.ac.uk><mailto:eprints-tech@ecs.soton.ac.uk>>
 >> Subject: [EP-tech] Problems with view generation: EPrints System Error  >>  >> Hello,  >>  >> since last week, we've had problems with the view generation for our  >> author/person view. These problems specifically concern the views for  >> new author names, which are not generated, while the views for already  >> existing author names are updated.
 >>
 >> For each author name concerned, there is an entry in the apache error  > log:
 >>
 >> Error writing
 > to
/usr/share/eprints3/archives/ubt_eref/html/de/view/person/Kaiser=3AMario=3A=3A.export:
 >   File too large
 >> ------------------------------------------------------------------
 >>       at /usr/share/eprints3/perl_lib/EPrints/Update/Views.pm line 1550
 >>              EPrints::Update::Views::output_files
 > ('EPrints::Repository=HASH(0x7fb0663688d0)',
 > '/usr/share/eprints3/archives/ubt_eref/html/de/view/person/Kai...',
 > 'XML::LibXML::DocumentFragment=SCALAR(0x7fb06a7e5938)',
 > '/usr/share/eprints3/archives/ubt_eref/html/de/view/person/Kai...',
 > 'XML::LibXML::Element=SCALAR(0x7fb06a45f160)',
 > '/usr/share/eprints3/archives/ubt_eref/html/de/view/person/Kai...',
 > 'XML::LibXML::DocumentFragment=SCALAR(0x7fb06a7e5938)',
 > '/usr/share/eprints3/archives/ubt_eref/html/de/view/person/Kai...',
 > 'XML::LibXML::DocumentFragment=SCALAR(0x7fb06a668aa8)', ...) called  > at /usr/share/eprints3/perl_lib/EPrints/Update/Views.pm line 935
 >>              EPrints::Update::Views::update_view_list
 > ('EPrints::Repository=HASH(0x7fb0663688d0)',
 > '/usr/share/eprints3/archives/ubt_eref/html/de/view/person/Kai...', 'de',  > 'EPrints::Update::Views=HASH(0x7fb06a6863f0)', 'ARRAY(0x7fb064b78c70)')  > called at /usr/share/eprints3/perl_lib/EPrints/Update/Views.pm line 259
 >>              EPrints::Update::Views::update_view_file
 > ('EPrints::Repository=HASH(0x7fb0663688d0)', 'de',  > '/view/person/Kaiser=3AMario=3A=3A.html',
 > '/view/person/Kaiser=3AMario=3A=3A.html') called  > at /usr/share/eprints3/perl_lib/EPrints/Apache/Rewrite.pm line 513  >> EPrints::Apache::Rewrite::handler('Apache2::RequestRec=SCALAR
 > (0x7fb06a471710)') called at -e line 0
 >>              eval {...} called at -e line 0
 >>
 >> In this specific case, the author's name is linked to only 2 EPrints  >> entries, so it is unlikely that this file would be particularly large.
 >>
 >> Does anybody have an idea concerning this? I would appreciate any help.
 >>
 >> Thanks!
 >>
 >> Verena
*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
*** Archive: http://www.eprints.org/tech.php/
*** EPrints community wiki: http://wiki.eprints.org/
*** EPrints developers Forum: http://forum.eprints.org/
-- The Open University is incorporated by Royal Charter (RC 000391), an exempt charity in England & Wales and a charity registered in Scotland (SC 038302). The Open University is authorised and regulated by the Financial Conduct Authority.