EPrints Technical Mailing List Archive

Message: #07830

< Previous (by date) | Next (by date) > | < Previous (in thread) | Next (in thread) > | Messages - Most Recent First | Threads - Most Recent First

Re: [EP-tech] EPrints/Elements Merge Problems

Hi James,

I've put some notes/code here:

https://gist.github.com/jesusbagpuss/ee27acd24a5d0e3fa3d29ef0075d921b

let me know if it doesn't make sense.

 

From my comments in the code (might be useful knowledge for others):

EPrints' default behaviour is to remove the 'pos' during a document clone *only* when the doc is being cloned to the same parent.

 

Cheers,

John

 

From: eprints-tech-bounces@ecs.soton.ac.uk [mailto:eprints-tech-bounces@ecs.soton.ac.uk] On Behalf Of James Kerwin via Eprints-tech
Sent: 16 May 2019 12:22
To: John Salter <J.Salter@leeds.ac.uk>
Cc: eprints-tech@ecs.soton.ac.uk
Subject: Re: [EP-tech] EPrints/Elements Merge Problems

 

He David and John,

 

I've taken a look at the symplectic_merge and symplectic_pids table, and using by budding skills of divination I couldn't find anything out of sorts in there - so far as I can tell. Although in trying to fix this with multiple re-deposits this one record has a load of different EPrints IDs.

 

It appears to be as John said. I've managed to clean the record up by fiddling with the document and file tables and moving some files around. Everything appears to be working...

 

John, thank you for the piece of SQL. It appears this is isn't a huge problem, but there are other instances that I'm now at least aware of. I'll put a ticket in with Symplectic. If you do have a solution it would be brilliant, but out of principle I think Symplectic should provide the solution to their customers. Capitalists shouldn't be depending on an open source group to fix their code. Anyway, I don't want to get too political...

 

Thank you both for your help.

 

Thanks,

James

 

On Thu, May 16, 2019 at 9:49 AM John Salter <J.Salter@leeds.ac.uk> wrote:

Hi James,
Yes - and I've submitted a bug report to Symplectic - with a fix for their connector - which they've never rolled out *sigh*.

When two items are merged, and initially they both have documents in folder '01', all these then get put into the 'surviving EPrint's '01' directory.
Worse still, if both items have a file of the same name e.g. Doc1.pdf (but they are different files), one of them will overwrite the other - and you have data-loss :o|

You can see how many things are affected by this with the following query:
SELECT   eprintid, pos, count(*) as c FROM   document GROUP BY   eprintid, pos HAVING c > 1;

The issue is in Symplectic/RepoProcess/MergeManager.pm - and a call to:
        my $new_doc = $doc->clone($target);
This clone doesn't reset the 'pos' - so you get the results reported.

I'll put a gist together with the changes needed to resolve this.

Feel free to log it as a ticket with Symplectic...

Cheers,
John

-----Original Message-----
From: eprints-tech-bounces@ecs.soton.ac.uk [mailto:eprints-tech-bounces@ecs.soton.ac.uk] On Behalf Of Newman D.R. via Eprints-tech
Sent: 16 May 2019 09:28
To: eprints-tech@ecs.soton.ac.uk; James Kerwin <jkerwin2101@gmail.com>
Subject: Re: [EP-tech] EPrints/Elements Merge Problems

Hi James,

Based on some experience with this it can be due to the symplectic_pids
table in EPrints getting out of sync.  I have in the past had to make
manual corrections to fix this.  It has never been clear to me what
caused the issue, as I only have access to the EPrints side.

I would take a look in this EPrints database table.  It has three IDs
two from Symplectic and one from EPrints.  A record with no outstanding
merge issues should have the same two Symplectic IDs and the associated
EPrint ID.  Sometimes I have not even been able to find the record I
need in this table based on a lookup against either the EPrint or
Symplectic ID.  Tell me what you find and I maybe able to advise
further or confirm whether this is or is not the issue you are
experiencing.

Regards

David Newman

On Thu, 2019-05-16 at 09:18 +0100, James Kerwin via Eprints-tech wrote:
> Hi All,
>
> This may be a question for the Symplectic list, but on the off-chance
> anybody has experienced similar problems... Has anybody had trouble
> with merging records in Elements and the result in EPrints being a
> complete mess?
>
> A record was merged recently and the results in EPrints are two
> documents in the same folder on the server (where the file download
> link points two).
>
> For example, there is usually one file per bottom level directory:
>
> Dir 01 = file1.pdf
> Dir 02 = file2.pdf
>
> and so on.
>
> I'm getting:
>
> Dir 01 = file1.pdf, file2.pdf
> Dir02 = file3.pdf
>
> Also the files showing Elements seem to be duplicating themselves and
> keeps changing to "deposit incomplete" Always one more of the
> duplicated file in Elements than is appearing in EPrints.
>
> I know I can tidy this up in EPrints, but I'd like to stop it
> happening altogether.
>
> Thanks,
> James
> *** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-
> tech
> *** Archive: https://eur03.safelinks.protection.outlook.com/?url="">
> *** EPrints community wiki: https://eur03.safelinks.protection.outlook.com/?url="">
> *** EPrints developers Forum: https://eur03.safelinks.protection.outlook.com/?url="">

*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
*** Archive: http://www.eprints.org/tech.php/
*** EPrints community wiki: http://wiki.eprints.org/
*** EPrints developers Forum: http://forum.eprints.org/