EPrints Technical Mailing List Archive

Message: #06087

< Previous (by date) | Next (by date) > | < Previous (in thread) | Next (in thread) > | Messages - Most Recent First | Threads - Most Recent First

Re: [EP-tech] get_mime_type - Video support in EPrints

Thanks to everyone who helped me understand how mime_types and video/audio support is configured in EPrints!
There are still some aspects video/audio handling by EPrints that I am confused by, such as what event in the workflow triggers the transcoding/thumbnail generating, but I really appreciate just how much  I was able to learn with your help. 

I updated our eprint_render.pl to integrate video-js player for audio and video documents.
These changes are now on our live repository, so as promised, I am forwarding some examples:

Audio examples:
http://spectrum.library.concordia.ca/6655/ 
http://spectrum.library.concordia.ca/977050/ 
Video example (psychology publication with video dataset): http://spectrum.library.concordia.ca/977815/   
Video example (Graduate project, with video):  http://spectrum.library.concordia.ca/981138/ 
Video example (PhD thesis with video): http://spectrum.library.concordia.ca/981907/ 

As I mentioned before, I tried to document what I learned about video support in EPrints here:
https://wiki.eprints.org/w/Video_documents 
Getting the video-js player to display was accomplished by customizing eprint_render.pl, building on the basic integration code I posted in the wiki.

Tomasz




-----Original Message-----
From: eprints-tech-bounces@ecs.soton.ac.uk [mailto:eprints-tech-bounces@ecs.soton.ac.uk] On Behalf Of Tomasz Neugebauer
Sent: October-14-16 4:49 PM
To: eprints-tech@ecs.soton.ac.uk
Subject: Re: [EP-tech] get_mime_type - Video support in EPrints

Hi John,

Thanks, that is very useful information.  So when precisely does EPrints generate the thumbnails during normal workflow?  Right after document upload?  This is important to know, because in case of video, the "thumbnails" could mean a long transcoding job with FFMPEG.

I don't know if/where my repository is using the citations/eprint/summary_page.xml, I think that I am actually rendering the abstract page without it, inside eprint_render.pl

I found the answers to some of my questions in the context of eprint_render.pl, and I was able to integrate Video.JS player on the abstract page as a result.
I updated the Wiki Page for Video Documents with how to do this:
https://wiki.eprints.org/w/Video_documents#Video.js_Integration 
For now, I only have it working on the development server, but I should more over all recent changes related to video to the live repository soon, and provide a link to an example.  

To get this done, I had to trial and error for the right API calls.  In an early version of it, I was using $doc->search_related() to find the URLs to the derivative video files and thumbnail image, but then I realized that it can be done with these calls:

my $video_preview_image = $doc->thumbnail_url("preview");
my $video_preview_1 = $doc->thumbnail_url("video_mp4");				
my $video_preview_2 = $doc->thumbnail_url("video_ogg");

So that simplified the code... One caveat with that is that I leave the width and height of the video player unset, because the only way I know to get those values is to use the $doc->search_related loop.  However, video.js doesn't seem to have a problem rendering the video player of appropriate dimension without those variables set.

To answer this: does EPrints, by default, attempt to serve the derivative files to the public?  It looks like if your configuration uses the summary_page.xml, it should be providing an HTTP link to a "preview" h264 encoded video.  I don't think it outputs an embedded video player though, or links to the ogg file.  

Best wishes,

Tomasz



-----Original Message-----
From: eprints-tech-bounces@ecs.soton.ac.uk [mailto:eprints-tech-bounces@ecs.soton.ac.uk] On Behalf Of John Salter
Sent: October-14-16 4:21 AM
To: eprints-tech@ecs.soton.ac.uk
Subject: Re: [EP-tech] get_mime_type?

...where do I start this explanation...

In the default citations/eprint/summary_page.xml, there is this:
<epc:when test="$doc.thumbnail_url('video_mp4').is_set()">
    <epc:print expr="$doc.preview_link($doc.citation_link('default'),'docs','video_mp4')" /> </epc:when>

The xml functions are in EPrints::Script::Compiled. Searching for 'thumbnail_url' in this file will lead to this method:
    run_thumbnail_url
which includes a call to $doc->thumbnail_url.
For the example above, if there is a 'video_mp4' thumbnail version, you'll get a URL back.

The 'video_mp4' is the same as a size for an image (and in the code it's normally referred to as 'size' which might be conifusing).

It's also worth having a look at:
EPrints::Script::Compiled::run_preview_link
EPrints::Script::Compiled::run_icon
EPrints::Script::Compiled::run_related_objects

EPrints::DataOBj::Document::thumbnail_url
EPrints::DataOBj::Document::search_related

And possibly even this:
EPrints::Dataobj::related_dataobjs
(but the 'search_related' above does a similar job for documents).

Hope that gets you a bit further on!
Cheers,
John




-----Original Message-----
From: eprints-tech-bounces@ecs.soton.ac.uk [mailto:eprints-tech-bounces@ecs.soton.ac.uk] On Behalf Of Tomasz Neugebauer
Sent: 13 October 2016 18:44
To: eprints-tech@ecs.soton.ac.uk
Subject: Re: [EP-tech] get_mime_type?

For video documents, let's say a user uploads a 1.4 GB video, a QuickTime MOV file.
By default, EPrints will create 2 derivative video files, and place them on like the thumbnails, for example, here:

~/archives/ARCHIVEID/documents/disk0/00/00/79/00/22/video.mp4
~/archives/ARCHIVEID/documents/disk0/00/00/79/00/23/video.ogg

Are there any API calls to find out if these derivative files exist for an item/document?
Are there any API calls to get the URL of these derivative video files?
By default, does EPrints attempt to serve these to the public? 

Tomasz


-----Original Message-----
From: eprints-tech-bounces@ecs.soton.ac.uk [mailto:eprints-tech-bounces@ecs.soton.ac.uk] On Behalf Of John Salter
Sent: October-13-16 12:05 PM
To: eprints-tech@ecs.soton.ac.uk
Subject: Re: [EP-tech] get_mime_type?

Hi Tomasz,
I don't think I've ever run 'redo_thumbnails' on video files, but for images, it will remove the derivative versions, and re-create them.
The thumbnails are stored as document objects, with a 'relation' defined.

Normally, the new files are stored in the same folder as the original - but in a new 'pos' directory e.g. for eprint 1234:
~/archives/ARCHIVEID/documents/disk0/00/00/00/12/34/01/This is the original.tiff ~/archives/ARCHIVEID/documents/disk0/00/00/00/12/34/02/large.jpg
~/archives/ARCHIVEID/documents/disk0/00/00/00/12/34/03/medium.jpg
~/archives/ARCHIVEID/documents/disk0/00/00/00/12/34/04/small.jpg

...run the redo thumbnails

~/archives/ARCHIVEID/documents/disk0/00/00/00/12/34/01/This is the original.tiff ~/archives/ARCHIVEID/documents/disk0/00/00/00/12/34/05/large.jpg
~/archives/ARCHIVEID/documents/disk0/00/00/00/12/34/06/medium.jpg
~/archives/ARCHIVEID/documents/disk0/00/00/00/12/34/07/small.jpg

If you change the settings for the video thumbnails (e.g. bitrate, or size), redo_thumbnails will get rid of the old ones and make new ones.

You can also configure storage plugins to e.g. save the originals to one pot of storage, and the thumbnails to another (which might be useful if you want to use different backup/archiving strategies for the two types, or want to put the thumbnails on a CDN).

BE WARNED... 
The thumbnailing jobs are added to the indexer queue. If you change settings, but don't restart the indexer, the old configuration will be used! This can be very confusing!  I know, I've been there ;o)

Cheers,
John

-----Original Message-----
From: eprints-tech-bounces@ecs.soton.ac.uk [mailto:eprints-tech-bounces@ecs.soton.ac.uk] On Behalf Of Tomasz Neugebauer
Sent: 13 October 2016 16:25
To: eprints-tech@ecs.soton.ac.uk
Subject: Re: [EP-tech] get_mime_type?


Thanks, Thomas, $doc->get_value( "mime_type") works! 
Also, thanks, John, I'm familiar with the redo_mime_type command; I think I understand this one:
The one that is more mysterious is redo_thumbnails.
For video documents, if you have FFMPEG and the H264 codec on the server, this transcodes to video to MP4 and OGG.
What does EPrints do with these transcoded files?  Do they get served to the public?  
Where does EPrints put them?  Is there an API call to get these derivative files for the document?

Best wishes,
Tomasz


-----Original Message-----
From: eprints-tech-bounces@ecs.soton.ac.uk [mailto:eprints-tech-bounces@ecs.soton.ac.uk] On Behalf Of John Salter
Sent: October-13-16 5:35 AM
To: eprints-tech@ecs.soton.ac.uk
Cc: Tomasz Neugebauer <Tomasz.Neugebauer@concordia.ca>
Subject: Re: [EP-tech] get_mime_type?

Might be worth mentioning that there is a function in bin/epadmin to redo mime types.

If you upgraded from 3.3.1 to 3.3.2, a suggestion to run this was made, but you can run it if it looks like your mime types are not quite right (and as file type detection improves).

>From the perldoc:
epadmin redo_mime_type repository_id dataset [ objectid, ... ]
    Re-run the file format identification. Dataset may be one of 'document' or 'file'. If 'document' only re-does the identification of the main files in documents.


Cheers,
John

-----Original Message-----
From: eprints-tech-bounces@ecs.soton.ac.uk [mailto:eprints-tech-bounces@ecs.soton.ac.uk] On Behalf Of Thomas Lauke
Sent: 13 October 2016 09:09
To: eprints-tech@ecs.soton.ac.uk
Cc: tomasz.neugebauer@concordia.ca
Subject: Re: [EP-tech] get_mime_type?

Hi Tomasz,

> How would I get the mime_type of a document in EPrints?

refering to ~/Eprints/perl_lib/EPrints/DataObj/Document.pm imho $doc->get_value( "mime_type" ); does the job!?

Hth
Thomas
*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
*** Archive: http://www.eprints.org/tech.php/
*** EPrints community wiki: http://wiki.eprints.org/
*** EPrints developers Forum: http://forum.eprints.org/

*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
*** Archive: http://www.eprints.org/tech.php/
*** EPrints community wiki: http://wiki.eprints.org/
*** EPrints developers Forum: http://forum.eprints.org/

*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
*** Archive: http://www.eprints.org/tech.php/
*** EPrints community wiki: http://wiki.eprints.org/
*** EPrints developers Forum: http://forum.eprints.org/

*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
*** Archive: http://www.eprints.org/tech.php/
*** EPrints community wiki: http://wiki.eprints.org/
*** EPrints developers Forum: http://forum.eprints.org/

*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
*** Archive: http://www.eprints.org/tech.php/
*** EPrints community wiki: http://wiki.eprints.org/
*** EPrints developers Forum: http://forum.eprints.org/

*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
*** Archive: http://www.eprints.org/tech.php/
*** EPrints community wiki: http://wiki.eprints.org/
*** EPrints developers Forum: http://forum.eprints.org/

*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
*** Archive: http://www.eprints.org/tech.php/
*** EPrints community wiki: http://wiki.eprints.org/
*** EPrints developers Forum: http://forum.eprints.org/