EPrints Technical Mailing List Archive

Message: #08494

< Previous (by date) | Next (by date) > | < Previous (in thread) | Next (in thread) > | Messages - Most Recent First | Threads - Most Recent First

Re: [EP-tech] [ext] Re: [3.4.2] Wrong status of indexer on the admin site

Hi,

So there are three separate ways of starting/stopping the indexer:

1. Through the web interface

2. Through the command line using /usr/share/eprints/bin/indexer

3. Through systemd using "systemctl start epindexer"
I have found in the past that methods 1 and 3 can have issues with each other but that if you have 3 setup and enabled (i.e. systemctl enable epindexer) then what the browser says should correspond to what systemctl status epindexer says.  However, stopping the indexer in the web browser will kill the indexer process but systemctl will likely start a new one up in its place.  This is where you can get in a bit of a mess and sometimes have two indexers running at once (4 indexer processes, 2 parents and 2 children) or an indexer running but the status page on in your browser saying it is not.
I did update the epindexer systemd module to start the indexer using a shell script that could to a bit of extra tidying: 

-----------------------

[Unit]
Description=EPrints Indexer
After=crond.service
Requires=mariadb.service

[Service]
Type=forking
PIDFile=/usr/share/eprints/var/indexer.pid
TimeoutStartSec=0
User=eprints
ExecStart=/usr/share/eprints/site_lib/bin/epindexer_systemd_start
ExecStop=/usr/share/eprints/bin/indexer stop
Restart=on-failure
RestartSec=60

[Install]
WantedBy=multi-user.target

-------------------------

#!/bin/bash
cd `dirname $0`
cd ../../
if [ -e var/indexer.pid ]; then
        pid=`ps aux | awk 'BEGIN{FS="[\t ]+"}{ print $11 }' | grep "indexer" | head -n 1` 
        if [ "$pid" != "" ]; then
                bin/indexer stop
                sleep 5
        fi
        rm -f var/indexer.pid var/indexer.tick
fi
bin/indexer start

---------------------------
This is by no means a perfect script but has helped reduce the amount of issues I have had with the indexer dying or running multiple instances.  Getting seamless integration between the systemd module and the web interface is near impossible.  You would not want to integrate starting and stopping using systemctl into EPrints, as it may not be applicable in all operating systems.  On the other hand, reliably trying to get the aspects of EPrints indexer management to recognise those of systemd and vice-versa is a very difficult task. 

Regards

David Newman

On 27/01/2021 10:46, systems via Eprints-tech wrote:

CAUTION: This e-mail originated outside the University of Southampton.

Hi David,
I only see the button "start indexer".
The page says that the indexer is stopped, but the indexer process is alive.

ps axu | grep $(cat /usr/share/eprints/var/indexer.pid)
eprints     6134  0.0  2.7 210776 50992 ?        Ss   11:06   0:00
indexer

The indexer.tick exits and will contain:

cat /usr/share/eprints/var/indexer.tick
# This file is by the indexer to indicate
# that the indexer is still is_running.

When I stop the idexer via
systemctl stop epindexer

The pid and tick file is gone and no inder process is running now.
And the log if it says:
[Wed Jan 27 11:25:18 2021] 6134 *** Indexer sub-process stopped
[Wed Jan 27 11:25:18 2021] 6134 ** Indexer process stopping

It looks like, when the indexer is starte via the command line, then the
gui don't detect it and vice versa.

After starting it from the gui, the gui say the it will runs, but not
the console.

systemctl status epindexer
● epindexer.service - The eprints indexer
     Loaded: loaded (/usr/lib/systemd/system/epindexer.service; static;
vendor preset: disabled)
     Active: inactive (dead)



Am 27.01.21 um 10:42 schrieb David R Newman:

Hi,

This is an interesting issue.  When you go to the main Admin menu page
under the System Tools tab what buttons are available for starting and
stopping the indexer?  What drives whether the Admin status page stays
Running, Stopped, etc. is the presence of the file:

/usr/share/eprints/var/indexer.pid

and whether there is an indexer process running with the process ID
stored in this file.  Also the presence of the file
/usr/share/eprints/var/indexer.tick may affect the status you see.  At
best guess  the indexer is running but under a different process ID to
that in indexer.pid.  It should be noted that if your run something like:

ps aux | grep indexer

You would get two processes back (three if you include the "grep
indexer" process in the command above).  The first is the parent process
and this should have the process ID that is in indexer.pid and the
second is a child process.  The latter is vulnerable to dying if the
indexer task it is undertaking fails in certain ways.  The parent
process should spawn a new child process if this happens.

Normally what I would do to clear things up is stop the indexer from the
command line.  Then if any indexer processes are still running use the
Unix kill command then kill these.  Then finally make sure that the
/usr/share/eprints/var/indexer.pid and
/usr/share/eprints/var/indexer.tick files are no longer present and
delete if they are.  Then I would go back to the web interface to check
the indexer's status (to ensure it is stopped) and hopefully then use
the Admin page's "Start Indexer" button to restart the indexer.  This
should get things back in a consistent state and hopefully the=is
inconsistency will not re-occur.

Hope this helps.  Please let me know if this does not solve your problem.

Regards

David Newman

Subject:  [3.4.2] Wrong status of indexer on the admin site
From:  systems <systems@mdc-berlin.de>
Date: 27/01/2021, 07:28
To: <eprints-tech@ecs.soton.ac.uk>

Hi list,
I try to get eprints working on CentOS8.
As far it looks working, but the on the admin status page, the indexer
is marked as stopped. But it runs and the task queue is empty.

systemctl tells, that it is running and
sudo -u eprints /usr/share/eprints/bin/indexer status
also:
Indexer is running with PID 10960. Next index in 27 seconds

How will the status page check the state of the indexer?


Thanks
for any hints.


*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
*** Archive: https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.eprints.org%2Ftech.php%2F&amp;data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C21b0c3c123b545a6191408d8c2bb1f8f%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637473456228424773%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=cjHX7UYUOXLtx0cP0I%2BkwkYZplAcb8yYTczXJMPEuw0%3D&amp;reserved=0
*** EPrints community wiki: https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwiki.eprints.org%2F&amp;data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C21b0c3c123b545a6191408d8c2bb1f8f%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637473456228424773%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=QZZDx8QsNqzpsEyvHsIkxosg%2FvG%2FyZXWwiUdgQvc7lk%3D&amp;reserved=0


*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
*** Archive: https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.eprints.org%2Ftech.php%2F&amp;data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C21b0c3c123b545a6191408d8c2bb1f8f%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637473456228429763%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=YOjZZnCSz2gufRkHoZaOlI4aPx0Qgy6iv97QUVC6Fs8%3D&amp;reserved=0
*** EPrints community wiki: https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwiki.eprints.org%2F&amp;data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C21b0c3c123b545a6191408d8c2bb1f8f%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637473456228429763%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=VL4vNRTUwCDSzz9cVlj%2FlEsIb3WU%2FOqUm5Ya7fSEgIw%3D&amp;reserved=0


--
This email has been checked for viruses by AVG.
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.avg.com%2F&amp;data=04%7C01%7Ceprints-tech%40ecs.soton.ac.uk%7C21b0c3c123b545a6191408d8c2bb1f8f%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637473456228429763%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=aKjyxYNpiuHOIY4yDXfOMpOoehhVd11Ct7GjKfuBcmA%3D&amp;reserved=0