EPrints Technical Mailing List Archive
See the EPrints wiki for instructions on how to join this mailing list and related information.
Message: #03402
< Previous (by date) | Next (by date) > | < Previous (in thread) | Next (in thread) > | Messages - Most Recent First | Threads - Most Recent First
[EP-tech] Re: Injecting gigabyte-scale files into EPrints archive - impossible?
- To: eprints-tech@ecs.soton.ac.uk
- Subject: [EP-tech] Re: Injecting gigabyte-scale files into EPrints archive - impossible?
- From: Florian Heß <hess@ub.uni-heidelberg.de>
- Date: Tue, 19 Aug 2014 10:35:32 +0200
Am 18.08.2014 um 16:17 schrieb Paolo Tealdi:
Il 04/08/2014 10:13, Yuri ha scritto:The only option seems to enlarge the tmp :-)or trying to convince the system to change the TMPDIR to a bigger filesystem.
Thank you both, Yuri and Paolo. :-)I am afraid, though, that this is not quite a sustainable, scalable approach. If for instance one used Recollect Plugin or whatever, anyway EPrints as a research data store, even bigger files would be to serve, say hundred gigabyte large database dumps. EPrints could not handle that giant files this way, as it would need absurd *several hundred* gigabyte of RAM and swap, let alone other involved bottle-necks in the program flow. If you don't mind my assuming that in regard to its easygoing use of join("", <STDIN>).
Concerning large files, we thought we could serve files up to a maximum of a DVD image. For the time being I will prefer the described copy & hack database method (but as said, I'll try use the API next time).
Look, as a public university library it is not overwhelming hardware resources which we have got for the given purpose, so we must use wise and economically what is available.
Kind regards Florian
Best regards, Paolo Tealdi
-- UB Heidelberg (Altstadt) Plöck 107-109, 69117 HD Abt. Informationstechnik http://www.ub.uni-heidelberg.de/
- References:
- [EP-tech] Injecting gigabyte-scale files into EPrints archive - impossible?
- From: Florian Heß <hess@ub.uni-heidelberg.de>
- [EP-tech] Re: Injecting gigabyte-scale files into EPrints archive - impossible?
- From: Yuri <yurj@alfa.it>
- [EP-tech] Re: Injecting gigabyte-scale files into EPrints archive - impossible?
- From: Florian Heß <hess@ub.uni-heidelberg.de>
- [EP-tech] Re: Injecting gigabyte-scale files into EPrints archive - impossible?
- From: Yuri <yurj@alfa.it>
- [EP-tech] Re: Injecting gigabyte-scale files into EPrints archive - impossible?
- From: Paolo Tealdi <paolo.tealdi@polito.it>
- [EP-tech] Injecting gigabyte-scale files into EPrints archive - impossible?
- Prev by Date: [EP-tech] Interesting issue with EP 3.2 & Perl 5.20... noticed with SWORD
- Next by Date: [EP-tech] wiki.eprints & files.eprints are down...
- Previous by thread: [EP-tech] Re: Injecting gigabyte-scale files into EPrints archive - impossible?
- Next by thread: [EP-tech] File Upload customisation
- Index(es):