Hi Agung,

The cache tables are for storing search results to make it quicker to move between pages of results.  This is somewhat of a legacy feature.

To remove old cache tables that failed to be removed when they were not longer needed you can run the command:

EPRINTS_PATH/bin/epadmin cleanup_cachemaps ARCHIVE_ID

I have also noted that when dumping out a database may fail if you do this with the webserver (i.e. Apache) still running EPrints.  This is because cache tables can be automatically be deleted by EPrints whilst the dump is happening and then mysqldump will complain it cannot dump a table that no longer exists, as it determines with tables it will be dumping out before it starts.  This is why I advise using the following mysqldump call to avoid dumping any cache tables (you will need to substitute <USERNAME>, <PASSWORD> and <DB_NAME> as appropriate:

mysqldump --single-transaction -u <USERNAME> -p<PASSWORD> <DB_NAME> `mysql -u <USERNAME> -p<PASSWORD> <DB_NAME> -e 'SHOW TABLES;' | grep -v cache[0-9] | grep -v Tables_in` > <DB_NAME>.sql

My suspicion is that the issue with 1 and 2 is that you have not run:

EPRINTS_PATH/epadmin update <ARCHIVE_ID>

As in particular a user login will not work if the user tables in the database does not match the user data object for EPrints.  I am exactly sure what your mean by "Latest Collecttion".  If you mean https://HOSTNAME/cgi/latest then it may be that this displays no results as no new publication has been moved to the live archive in the last 7 days.  Otherwise, I would tails the Apache error logs to see if you see and errors when your try to load "Latest Collection".

3.  I would double-check that documents are in the right place.  Look at some eprint records' dir fields (e.g. SELECT eprintid, dir FROM eprint WHERE eprints_status = 'archive' LIMIT 10;) and make sure that the documents directory for those eprints is under EPRINTS_PATH/archives/<ARCHIVE_ID>/documents/<DIR_VALUE>.  In my experience some older repositories don't quite put the documents where you expect.  (e.g. if you EPRINTS_PATH is /opt/eprints3 and your ARCHIVE_ID is my_archive then eprint 1234's document directory would be the following on a modern EPrints repository: /opt/eprints3/archives/my_archive/documents/disk0/00/00/12/34/. 

An alternative issue for 3 could be permissions.  What user and group do you have set for Apache (this can be found by running "httpd -S" on RHEL-based Liinux or "source /etc/apache2/envvars && apache2 -S" on Debian-based Linux).  If  the user for Apache does not match the user who owns the files under documents you may not be able to access these files through EPrints.  It would be useful to see the results from the command previous mentioned and and "ls -larZ EPRINTS_PATH/archives/<ARCHIVE_ID>/documents/<DIR_VALUE>" on one of the eprints found in the earlier SQL query.

Finally, 3 could be an SELinux issue.  What does the command "getenforce" say?

Regards

David Newman

On 30/10/2024 3:04 am, Agung Prasetyo W. wrote:

CAUTION: This e-mail originated outside the University of Southampton.

Hi David and John,

Still related to the database problem. I see the error database, there are lots of cachexx.ibd and cachexxx.frm. Can you explain, what is this cache file for? Then if it is deleted, is it okay for eprints?

I don't know if this is related to the command: epadmin update ArchiveID, but currently our repository is experiencing problems, including:
1. We can't see the Latest Collection.
2. Ee also can't log in, even though the user and password we entered are correct
3. All pdf documents attached to the item cannot be opened. Whereas if we check, the pdf document is there and its position is in accordance with the item.

Thanks.

Regards,

Agung PW

On Sat, 26 Oct 2024 at 15:41, David R Newman <drn@ecs.soton.ac.uk> wrote:

Hi John and Agung,

This field was added in EPrints 3.4.3 [1].  This was because a number of third party applications for CRIS [2] tried to keep their data in sync with EPrints but did not realise that the date_embargo field would be unset when lift_embargos ran on the day the embargo expired.  Therefore, this field was added so they could sync their embargo date with the date_embargo_retained field instead [3].

Subsequently, in EPrints 3.4.4 [4], extra functionality was added to the lift_embargos script to allow you to retain the embargo_date field by setting a --retain flag when calling the script [5].  This was quite a complex change, as other places in the code would be unhappy if the date_embargo was in the past or if date_embargo was set but the security set to public.  The date_embargo_retained was keep after this change, as it may have caused issues for those who has adopted it for the purpose described above, requiring them having to reconfigure and re-integrate with this third-party application.

Anyway, date_embargo_retained was added to the core fields in the Document data object [6].  As already observed, it should just be a case of run "epadmin update ARCHIVE_ID" to add this field to your repository's database.  If you don't make use of this field it does not matter, as it should not impact on how any other aspect of how your EPrints repository behaves.

Regards

David Newman

[1] https://wiki.eprints.org/w/EPrints_3.4.3#:~:text=Adds%20date_embargo_retained%20field%20to%20documents%20to%20retain%20the%20embargo%20date%20after%20the%20embargo%20is%20lifted.
[2] https://en.wikipedia.org/wiki/Current_research_information_system
[3] https://github.com/eprints/eprints3.4/issues/100
[4] https://wiki.eprints.org/w/EPrints_3.4.4#:~:text=Allows%20lift_embargos%20to%20retain%20date_embargo%20if%20--retain%20flag%20is%20set.
[5] https://github.com/eprints/eprints3.4/issues/221
[6] https://github.com/eprints/eprints3.4/blob/master/perl_lib/EPrints/DataObj/Document.pm#L315-L316

On 26/10/2024 7:52 am, John Salter wrote:

CAUTION: This e-mail originated outside the University of Southampton.

CAUTION: This e-mail originated outside the University of Southampton.

Hi Agung,

Something, somewhere has added the definition of that field to your repository.

Search through the code to find any mention of 'date_embargo_retained'.

I'd use something like this, run from the main EPrints directory:

grep -r 'date_embargo_retained' archives/*/cfg perl_lib lib site_lib flavours ingredients 

(Some of those directories might not exist, depending on which version of EPrints you are on).

If you've added or updated a plugin/ingredient/flavour, they could have added that field to the document dataset.

Cheers,

John

---

From: eprints-tech-request@ecs.soton.ac.uk <eprints-tech-request@ecs.soton.ac.uk> on behalf of Agung Prasetyo W. <prazetyo@gmail.com>
Sent: 25 October 2024 23:32
To: eprints-tech@ecs.soton.ac.uk <eprints-tech@ecs.soton.ac.uk>
Subject: Re: [EP-tech] Help : Ask about mysql error

 

CAUTION: External Message. Use caution opening links and attachments.

CAUTION: This e-mail originated outside the University of Southampton.

CAUTION: This e-mail originated outside the University of Southampton.

Hi John,

I wonder why the error can appear, because I did not upgrade my eprints version. Why can this happen? Are there any conditions why this can happen?

Thank you

Regards,

Agung PW

On Fri, 25 Oct 2024 at 21:06, John Salter <J.Salter@leeds.ac.uk> wrote:

CAUTION: This e-mail originated outside the University of Southampton.

CAUTION: This e-mail originated outside the University of Southampton.

Hi Agung,
The 'date_embargo_retained' field was added to 3.4: https://github.com/eprints/eprints3.4/commit/aa38b060053f05f09801d752a5c0f3332756bc73

 

I guess your database needs to be updated, so you should try running:

~/bin/epadmin update_dry_run ARCHIVEID

 

This should show if there are other fields missing from the database.

If the changes look OK, run:

~/bin/epadmin update ARCHIVEID

 

Cheers,

John

 

From: eprints-tech-request@ecs.soton.ac.uk <eprints-tech-request@ecs.soton.ac.uk> On Behalf Of Agung Prasetyo W.
Sent: Friday, October 25, 2024 1:08 PM
To: eprints-tech@ecs.soton.ac.uk
Subject: [EP-tech] Help : Ask about mysql error

 

CAUTION: External Message. Use caution opening links and attachments.

CAUTION: This e-mail originated outside the University of Southampton.

CAUTION: This e-mail originated outside the University of Southampton.

Hi,

I'd like to ask about mysql errors. In apache error.log, the error show a lot and display like below:

SQL ERROR (execute): Unknown column 'date_embargo_retained_year' in 'field list'
DBD::mysql::st fetchrow_array failed: fetch() without execute() at /usr/share/eprints/perl_lib/EPrints/Database.pm line 2538.
DBD::mysql::st execute failed: Unknown column 'date_embargo_retained_year' in 'field list' at /usr/share/eprints/perl_lib/EPrints/Database.pm line 1003.
SQL ERROR (execute): SELECT `docid`,`rev_number`,`eprintid`,`pos`,`placement`,`mime_type`,`format`,`formatdesc`,`language`,`security`,`license`,`main`,`date_embargo_year`,`date_embargo_month`,`date_embargo_day`,`date_embargo_retained_year`,`date_embargo_retained_month`,`date_embargo_retained_day`,`embargo_reason`,`content`,`upload_url`,`media_duration`,`media_audio_codec`,`media_video_codec`,`media_width`,`media_height`,`media_aspect_ratio`,`media_sample_start`,`media_sample_stop` FROM `document` WHERE `docid` IN (29398,29399,29400,29401,29402,29403,29404,29411,29412,29413,29414,29415,29416,29417)
SQL ERROR (execute): Unknown column 'date_embargo_retained_year' in 'field list'
DBD::mysql::st fetchrow_array failed: fetch() without execute() at /usr/share/eprints/perl_lib/EPrints/Database.pm line 2538.
DBD::mysql::st execute failed: Unknown column 'date_embargo_retained_year' in 'field list' at /usr/share/eprints/perl_lib/EPrints/Database.pm line 1003.
SQL ERROR (execute): SELECT `docid`,`rev_number`,`eprintid`,`pos`,`placement`,`mime_type`,`format`,`formatdesc`,`language`,`security`,`license`,`main`,`date_embargo_year`,`date_embargo_month`,`date_embargo_day`,`date_embargo_retained_year`,`date_embargo_retained_month`,`date_embargo_retained_day`,`embargo_reason`,`content`,`upload_url`,`media_duration`,`media_audio_codec`,`media_video_codec`,`media_width`,`media_height`,`media_aspect_ratio`,`media_sample_start`,`media_sample_stop` FROM `document` WHERE `docid` IN (26906)

 

Please, need help about this error.

 

Thank you.

 

Regards,

Agung PW

*** Options: https://wiki.eprints.org/w/Eprints-tech_Mailing_List
*** Archive: https://www.eprints.org/tech.php/
*** EPrints community wiki: https://wiki.eprints.org/

*** Options: https://wiki.eprints.org/w/Eprints-tech_Mailing_List
*** Archive: https://www.eprints.org/tech.php/
*** EPrints community wiki: https://wiki.eprints.org/