EPrints Technical Mailing List Archive

See the EPrints wiki for instructions on how to join this mailing list and related information.

Message: #09993

< Previous (by date) | Next (by date) > | < Previous (in thread) | Next (in thread) > | Messages - Most Recent First | Threads - Most Recent First

Re: [EP-tech] Indexer Errors: Counters table update error

Hi James,

Adding to what John has already said.  If it is not that the number is too high then you may just be able to manually update or set the tripleid in counter in the counters table to the current MAX tripleid in the triple table  (update if present, insert if not):

UPDATE counters SET counter = (SELECT MAX(tripleid) from triple) WHERE countername = 'tripleid';
INSERT INTO counters VALUES('tripleid', (SELECT MAX(tripleid) from triple));

This does feel a bit hacky but I don't think there is any way beyond a MySQL query command to fix this.  As "epadmin rebuild_triples" does not reset the counter it just pushes tasks to the indexer for all items recalculate its triples.

I guess it is possible you can hit the out of bounds limit but I have checked some very large and old repositories and they have only reached about a third (700 million) for their tripleid counter.  This suggests you may have something that is causing excessive updates to items in your repository that keeps replacing triples and incrementing their counter. 

One issue I have noted for the repository that has a max triple ID over 700 million is that they have used up over 50% of the inodes for the filesystem because of excessive revision files being generated in the past.  I think more modern operating systems have increased the inodes per gigabyte ratio but this is another issue you might encounter, if you have an integer overflow in your tripleid counter.

Overall, these may be symptoms of an issue with how updates are pushed to items in your repository, if that is done by a third-party system.  I have seen cases where these have created null updates where a new revision file is generated but all that has changed in the lastmod and rev_number fields.

Regards

David Newman

On 21/02/2025 9:37 am, John Salter wrote:
CAUTION: This e-mail originated outside the University of Southampton.
CAUTION: This e-mail originated outside the University of Southampton.

What does:
SELECT * FROM counters WHERE countername = ‘tripleid’;

Give you?

 

Is it somewhere near 2,147,483,648?

 

In my DB the counters column is an INT(11) – which gives 2^31 – but positive and negative.

Resolving this might just be a quick change to the table – maybe to an unsigned Int.

I’m not sure how this would impact the DOI minting – other than the indexer will be doing lots of failing – as that’s what updates the triples.

 

Cheers,

John

 

From: eprints-tech-request@ecs.soton.ac.uk <eprints-tech-request@ecs.soton.ac.uk> On Behalf Of James Kerwin
Sent: 21 February 2025 09:18
To: eprints-tech@ecs.soton.ac.uk
Subject: [EP-tech] Indexer Errors: Counters table update error

 

CAUTION: External Message. Use caution opening links and attachments.


CAUTION: This e-mail originated outside the University of Southampton.

CAUTION: This e-mail originated outside the University of Southampton.

Hi All,

 

I was taking a look at the indexer logs today to see why a DOI wouldn't mint and noticed the indexer logs are filled with a recurring series of errors that appear to be related.

 

I don't know if this is safe to ignore, but I'm a bit stuck as to how to investigate it further. I can't tell which error comes first, but they include:

 

DBD::mysql::db do failed: Out of range value for column 'counter' at row 1 at /opt/eprints3/bin/../perl_lib/EPrints/Database.pm line 839.
SQL ERROR (do): UPDATE `counters` SET counter=LAST_INSERT_ID(counter+1) WHERE `countername` = 'tripleid'
SQL ERROR (do): Out of range value for column 'counter' at row 1 (#1264)
Use of uninitialized value $rows_affected in numeric eq (==) at /opt/eprints3/perl_lib/EPrints/Database/mysql.pm line 394.
Column 'tripleid' cannot be null at /opt/eprints3/bin/../perl_lib/EPrints/DataObj.pm line 316.

 

The log only goes back five days, but they're all filled with this. I'm on EPrints 3.4.5. I suspect if it was a huge problem I would have noticed something by now, but I also can't ignore it now I know that it is there.

 

Thanks,

James


*** Options: https://wiki.eprints.org/w/Eprints-tech_Mailing_List
*** Archive: https://www.eprints.org/tech.php/
*** EPrints community wiki: https://wiki.eprints.org/