EPrints Technical Mailing List Archive

See the EPrints wiki for instructions on how to join this mailing list and related information.

Message: #10423

< Previous (by date) | Next (by date) > | < Previous (in thread) | Next (in thread) > | Messages - Most Recent First | Threads - Most Recent First

Re: [EP-tech] Bulk upload of data

CAUTION: This e-mail originated outside the University of Southampton.
Hi Yuri,
I have a 'helper' script that does something like what you need (EPrints v3.4, but I think it worked under 3.3 too):
https://gist.github.com/jesusbagpuss/25ebff597050df77316011991914ec2d

It wasn't really written to be shared - but it will hopefully be useful/understandable.

I get the files onto the server via SCP, and then run the script:
~/archives/ARCHIVEID/bin/upload_from_filesystem ARCHIVEID EPrintID file_dir [do-it [no-confirm]]

If you specify a file, it will add that to the EPrint.
If you specify a directory, it will add each file in there as a separate document to the EPrint.

If you don't pass the 'do it' flag (as 'yes'), it will just tell you what it would have done.
It also asks what security you want to apply to the document. You could extend that part to ask for other document-level metadata.

Let me know if you have any questions about it!

Cheers,
John


From: eprints-tech-request@ecs.soton.ac.uk <eprints-tech-request@ecs.soton.ac.uk> on behalf of Yuri <yurj@alfa.it>
Sent: Monday, September 21, 2026 14:44
To: EPrints.org Technical List <eprints-tech@ecs.soton.ac.uk>
Subject: [EP-tech] Bulk upload of data

CAUTION: External Message. Use caution opening links and attachments.

CAUTION: This e-mail originated outside the University of Southampton.

CAUTION: This e-mail originated outside the University of Southampton.

Hi!

  we manage a research data repository made with Eprints 3.3.15. I need
to upload some large dataset with many (hundreds of) files, is there a
way to upload them with a script or a similar tool? The records are
relative few (about 20). Some of these files are quite big (20-30 GB).
 From your experience, what is the best strategy here?