CAUTION: This e-mail originated outside the University of Southampton.

Hi David!

 A good starting point is to create an example that can create such 2^31-1 triples. Is there a way in a running EPrints to calculate how many triples should be in the database? A sort of reindex without writing in the database but just counting them. I think it might be useful to check whether there are any software bugs in the case brought by James Kerwin. If so, I think a recalculation of the triples could solve the problem in that case.

Hi Yuri,

I don't disagree we have negated adding changes we have made to the schema, so they alter existing archive's database schema, when they run "epadmin upgrade".  I have created a GitHub issue for this:

https://github.com/eprints/eprints3.4/issues/430

On the specific point about the counters table; I don't think we ever contemplated that more than 2.1 billion triples would have been generated in the lifetime of a repository.  However, there is no good reason not to make the counters table bigints 2^63-1.  Unfortunately, if tables that are liable to exceed 2^31 limit need to be modified, I am concerned how long and how much memory an alter table command might take.  In an ideal world, we should probably also make it an unsigned bigint.  However, It is probably easiest to inherit from the existing Bigint field [1] for this purpose, which I am pretty sure is signed.  I have logged another GitHub issue for this:

https://github.com/eprints/eprints3.4/issues/431

Regards

David Newman

[1] https://wiki.eprints.org/w/Bigint_field

On 21/02/2025 11:52, Yuri wrote:

CAUTION: This e-mail originated outside the University of Southampton.

CAUTION: This e-mail originated outside the University of Southampton.

It seems to me that upgrading from 3.3 to 3.4 does not upgrade the counter INT column to use long int. Or, maybe, this database has been upgraded over time and this step is missing.

Il 21/02/25 10:37, John Salter ha scritto:

CAUTION: This e-mail originated outside the University of Southampton.

CAUTION: This e-mail originated outside the University of Southampton.

What does:
SELECT * FROM counters WHERE countername = ‘tripleid’;

Give you?

 

Is it somewhere near 2147483648?

 

In my DB the counters column is an INT(11) – which gives 2^31 – but positive and negative.

Resolving this might just be a quick change to the table – maybe to an unsigned Int.

I’m not sure how this would impact the DOI minting – other than the indexer will be doing lots of failing – as that’s what updates the triples.

 

Cheers,

John

 

From: eprints-tech-request@ecs.soton.ac.uk <eprints-tech-request@ecs.soton.ac.uk> On Behalf Of James Kerwin
Sent: 21 February 2025 09:18
To: eprints-tech@ecs.soton.ac.uk
Subject: [EP-tech] Indexer Errors: Counters table update error

 

CAUTION: External Message. Use caution opening links and attachments.

CAUTION: This e-mail originated outside the University of Southampton.

CAUTION: This e-mail originated outside the University of Southampton.

Hi All,

 

I was taking a look at the indexer logs today to see why a DOI wouldn't mint and noticed the indexer logs are filled with a recurring series of errors that appear to be related.

 

I don't know if this is safe to ignore, but I'm a bit stuck as to how to investigate it further. I can't tell which error comes first, but they include:

 

DBD::mysql::db do failed: Out of range value for column 'counter' at row 1 at /opt/eprints3/bin/../perl_lib/EPrints/Database.pm line 839.
SQL ERROR (do): UPDATE `counters` SET counter=LAST_INSERT_ID(counter+1) WHERE `countername` = 'tripleid'
SQL ERROR (do): Out of range value for column 'counter' at row 1 (#1264)
Use of uninitialized value $rows_affected in numeric eq (==) at /opt/eprints3/perl_lib/EPrints/Database/mysql.pm line 394.
Column 'tripleid' cannot be null at /opt/eprints3/bin/../perl_lib/EPrints/DataObj.pm line 316.

 

The log only goes back five days, but they're all filled with this. I'm on EPrints 3.4.5. I suspect if it was a huge problem I would have noticed something by now, but I also can't ignore it now I know that it is there.

 

Thanks,

James

*** Options: https://wiki.eprints.org/w/Eprints-tech_Mailing_List

*** Archive: https://www.eprints.org/tech.php/

*** EPrints community wiki: https://wiki.eprints.org/

*** Options: https://wiki.eprints.org/w/Eprints-tech_Mailing_List

*** Archive: https://www.eprints.org/tech.php/

*** EPrints community wiki: https://wiki.eprints.org/