EPrints Technical Mailing List Archive

See the EPrints wiki for instructions on how to join this mailing list and related information.

Message: #07701

< Previous (by date) | Next (by date) > | < Previous (in thread) | Next (in thread) > | Messages - Most Recent First | Threads - Most Recent First

Re: [EP-tech] Error when running irstats

Hi,

I think it means that there are around 5 million records that it considered to be duplicates, bots, crawlers and other non-human access.  It kept the details of 3.5 million downloads and abstract page accesses.

 

Alan

 

From: eprints-tech-bounces@ecs.soton.ac.uk [mailto:eprints-tech-bounces@ecs.soton.ac.uk] On Behalf Of Ajunk Pracetio via Eprints-tech
Sent: 15 February 2019 01:20
To: eprints-tech@ecs.soton.ac.uk; John Salter <J.Salter@leeds.ac.uk>
Subject: Re: [EP-tech] Error when running irstats

 

Hi,

 

Previously, thank you for your help. The process of the process_stats command is complete.

 

I also read your e-mail, and I want to ask, does this mean we have to make changes from utf8 to utfmb4? I read the process of moving from utf8 to utf8mb4 is quite complicated, and the fear has an impact on the database.

When the process_stats command is complete, a description appears as follows:

 

Access: it took 152588 secs to parse 8741665 records (average = 57.29 records / sec)
Access: 3533033 records kept out of 8741665 (ratio = 40.42%)

 

image.png

 

In the line above, does that mean there are around 3533033 records that are not entered?

Thank you,

 

Best regards,

Agung Prasetyo W.

 

On Thu, Feb 14, 2019 at 10:58 PM John Salter via Eprints-tech <eprints-tech@ecs.soton.ac.uk> wrote:

Hi Agung,
I think this might be a problem with how MySQL handles UTF-8 data.

What MySQL refers to as UTF-8 cannot handle 4-byte UTF-8 characters.
Yes - this is very stupid

This explains it well: https://emea01.safelinks.protection.outlook.com/?url="">, and has some solutions.

This might also be useful:
https://emea01.safelinks.protection.outlook.com/?url="">

Does that help at all?

Cheers,
John

-----Original Message-----
From: eprints-tech-bounces@ecs.soton.ac.uk [mailto:eprints-tech-bounces@ecs.soton.ac.uk] On Behalf Of Yuri Carrer via Eprints-tech
Sent: 13 February 2019 08:04
To: eprints-tech@ecs.soton.ac.uk
Subject: Re: [EP-tech] Error when running irstats

in /usr/share/eprints3/lib/plugins/EPrints/Stats/Handler.pm , at the
beginning.

Il 13/02/19 08:36, Ajunk Pracetio ha scritto:
> Hi,
>
> Where I can activate this debug?
>
> Thank you
>
> Best regards,
> Agung Prasetyo W.
>
> On Wed, Feb 13, 2019 at 2:16 PM Yuri via Eprints-tech
> <eprints-tech@ecs.soton.ac.uk <mailto:eprints-tech@ecs.soton.ac.uk>>
> wrote:
>
>     https://emea01.safelinks.protection.outlook.com/?url="">
>
>     maybe this?
>
>     \xF76 should be a four byte utf-8. But Eprints itself don't have this
>     problem, so it is something in Irstats connection or irstat tables
>     with
>     different column encoding. Anyway, you can activate
>
>     my $DEBUG_SQL = 0; (switch to 1) to debug the sql.
>
>     Il 13/02/19 01:58, Ajunk Pracetio via Eprints-tech ha scritto:
>     > Hi,
>     >
>     > Previously, I migrated eprints 3.3.12 to 3.4 on Ubuntu 12.04 LTS.
>     > Currently I have managed to move eprints from an old server to a
>     > server using Ubuntu 16.04 LTS.
>     >
>     > Only, when doing the irstats process, the *process_stats REPOID
>     > --setup --verbose* command appears an error like the following:
>     >
>     > *DBD::mysql::st execute failed: Incorrect string value: '\xF76' for
>     > column 'value' at row 1 at
>     > /usr/share/eprints3/lib/plugins/EPrints/Stats/Handler.pm line 653.
>     > ....
>     > DBD::mysql::st execute failed: Data too long for column 'value'
>     at row
>     > 1 at /usr/share/eprints3/lib/plugins/EPrints/Stats/Handler.pm
>     line 653.*
>     >
>     > Did this error occur because our previous database actually came
>     from
>     > the version of mysql 5.5.54 (eprints 3.3.12 and Ubuntu 12.04) and
>     > changed to mysql 5.7.25 (eprints 3.4 and Ubuntu 16.04).
>     >
>     > I have searched Google and haven't found a solution. Can anyone
>     help
>     > us with this error? Then will this error affect the amount of data
>     > displayed on IRStats?
>     >
>     > thank you.
>     >
>     > Best regards.
>     > Agung Prasetyo W.
>     >
>     > --
>     > Life isn't valuable if you don't have someone to care
>     >
>     > Blog :
>     https://emea01.safelinks.protection.outlook.com/?url="">
>
>     >
>     <https://emea01.safelinks.protection.outlook.com/?url="">>
>     > IG & Twitter : prazetyo
>     >
>     > *** Options:
>     http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
>     > *** Archive:
>     https://emea01.safelinks.protection.outlook.com/?url="">
>     > *** EPrints community wiki:
>     https://emea01.safelinks.protection.outlook.com/?url="">
>     > *** EPrints developers Forum:
>     https://emea01.safelinks.protection.outlook.com/?url="">
>     *** Options:
>     http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
>     *** Archive: https://emea01.safelinks.protection.outlook.com/?url="">
>     *** EPrints community wiki: https://emea01.safelinks.protection.outlook.com/?url="">
>     *** EPrints developers Forum: https://emea01.safelinks.protection.outlook.com/?url="">
>
>
>
> --
> Life isn't valuable if you don't have someone to care
>
> Blog : https://emea01.safelinks.protection.outlook.com/?url="">
> IG & Twitter : prazetyo

--
Yuri Carrer

  CAB - Centro di Ateneo per le Biblioteche, Università di Padova
  Tel: 049/827 3615 - Via Anghinoni, 3 - Padova


*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
*** Archive: https://emea01.safelinks.protection.outlook.com/?url="">
*** EPrints community wiki: https://emea01.safelinks.protection.outlook.com/?url="">
*** EPrints developers Forum: https://emea01.safelinks.protection.outlook.com/?url="">

*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
*** Archive: http://www.eprints.org/tech.php/
*** EPrints community wiki: http://wiki.eprints.org/
*** EPrints developers Forum: http://forum.eprints.org/



--

Life isn't valuable if you don't have someone to care

 

IG & Twitter : prazetyo

-- The Open University is incorporated by Royal Charter (RC 000391), an exempt charity in England & Wales and a charity registered in Scotland (SC 038302). The Open University is authorised and regulated by the Financial Conduct Authority in relation to its secondary activity of credit broking.