EPrints Technical Mailing List Archive

See the EPrints wiki for instructions on how to join this mailing list and related information.

Message: #10419

< Previous (by date) | Next (by date) > | < Previous (in thread) | Next (in thread) > | Messages - Most Recent First | Threads - Most Recent First

Re: [EP-tech] Links in PDFs that don't contain a protocol, resulting in lots of 'hits'

Hi John,

I have not seen this exact behaviour but I have certainly seen cases where a good URL has been suffix with extra text and then requested.  This extra text makes no sense but usually I have found it is either testing well known pages (e.g. wp-login.php) or is trying to inject something for MySQL/_javascript_ type attacks.

I have written some fixes to more efficiently handle requests like this:

https://github.com/eprints/eprints3.4/blob/5efb3db2050938cd6097cb93153c603a138cc3f7/perl_lib/EPrints/Apache/Rewrite.pm#L545-L548

This is intended to stop trying to process the request if there is not a file with the specified filename under that document.  This saves quite a bit of extra processing, which is unnecessary if the file with a particular filename does not exist.  I would be interested in knowing whether this causes any problems when you have files that are under sub-directories.  I appreciate in this case the "files under sub-directories" don't actually exist.  So as a side-request, if you do have documents that contain files under sub-directories it would be useful if you could test the above code change.  Files under sub-directories typically only happen in two main scenarios AFAIAA:

1. Uploading a URL which if it is a webpage will need to upload the resources (images, CSS, _javascript_, etc.) for the page.

2. Uploading an archive (zip) file that is then unpacked.

Regards

David Newman

On 07/09/2026 10:54, John Salter wrote:
CAUTION: This e-mail originated outside the University of Southampton.
CAUTION: This e-mail originated outside the University of Southampton.
Hi All,
I've been looking at some 'intersting' traffic patterns in our repository and traced some of them back to PDFs that have been crawled, but also have poorly formatted links in them.
From an intitial PDF downloaded from 
    /id/eprint/[eprintid]/[pos]/Document.pdf
the 'bad' links results in the crawler trying to access links within the downloaded PDF relatively to the PDF URL. In the logs these appear like:
    GET /id/eprint/[eprintid]/[pos]/www.example.com/some_page/

EPrints responds to the above request with the PDF again (based on eprintid and pos). The crawler then uses that as a base, so the next 'round' from the crawler requests:
    GET /id/eprint/[eprintid]/1/www.example.com/www.example2.com/some_other_page/

This real example from my logs shows 8-iterations of this behaviour (I've changed the initial domain, so their site doesn't get crawled with this URL - that would be unfair):
www.housinglin.org.uk.example.com/_assets/Resources/Housing/OtherOrganisation/www.ageingwellinwales.com/Libraries/Documents/www.alz.co.uk/research/www.ageingwellinwales.com/Libraries/Documents/www.ssehsactive.org.uk/userfiles/Documents/www.gov.uk/government/uploads/system/uploads/attachment_data/file/215101/www.ageingwellinwales.com/Libraries/Documents/www.mrc.ac.uk/documents/pdf/mrc-phsrn-process-evaluation-guidance-final

Have others noted similar behaviour? Is there a way to limit this behaviour?
The PDFs aren't generally in our control - they're author accepted manuscripts, so correcting the PDF somehow isn't a realistic option.

Cheers,
John

John Salter

https://orcid.org/0000-0002-8611-8266

 

White Rose Libraries Technical Officer
Library and Research Management team, IT
University of Leeds


*** Options: https://wiki.eprints.org/w/Eprints-tech_Mailing_List
*** Archive: https://www.eprints.org/tech.php/
*** EPrints community wiki: https://wiki.eprints.org/