EPrints Technical Mailing List Archive

See the EPrints wiki for instructions on how to join this mailing list and related information.

Message: #06132

< Previous (by date) | Next (by date) > | < Previous (in thread) | Next (in thread) > | Messages - Most Recent First | Threads - Most Recent First

[EP-tech] Antwort: IRStats2: Innodb tables; forks

Hi John,

we migrated our test system to InnoDB about a year ago, did some tuning using the MySQL performance tuning primer script, and carried out some performance tests with the Apache Benchmark tool on both the test and production system.

Result: For sequential accesses, response times with InnoDB were 930-960 ms, with MyISAM they were 700ms. For concurrent accesses, response times dropped to 250ms on average with InnoDB. But you need many concurrent accesses to achieve this number. Also, when Web browsing, the InnoDB test system felt sluggish compared to the MyISAM production system.
IRStats2 weekly update runs take about 10 hours on the MyISAM system, and 17 hours on the InnoDB system, although there are more updates to process on the production system.  (I see that you have added transactions to IRStats2 on GitHub, we will check whether this helps).

To bring InnoDB to fly, you would need a huge innodb_buffer_pool_size, around the total size of your databases (which in our case is around 30-40 GB), requiring a hefty DB server.

Since we were not sure, in as much the current EPrints version supports InnoDB transactions, we decided to stay with MyISAM on the production system for the moment.

Cheers,

Martin


Inactive hide details for John Salter ---13/12/2016 12:59:06---Hi, Our IRStats2 processing runs slowly, and I suspect there migJohn Salter ---13/12/2016 12:59:06---Hi, Our IRStats2 processing runs slowly, and I suspect there might be some improvements to be made -

Von: John Salter <J.Salter@leeds.ac.uk>
An: "'eprints-tech@ecs.soton.ac.uk'" <eprints-tech@ecs.soton.ac.uk>
Datum: 13/12/2016 12:59
Betreff: [EP-tech] IRStats2: Innodb tables; forks
Gesendet von: eprints-tech-bounces@ecs.soton.ac.uk




Hi,
Our IRStats2 processing runs slowly, and I suspect there might be some improvements to be made - especially as we're using InnoDB tables.
Before I start trying to work out how to solve this, has anyone done any work in this area?
 
Also, there are two forks of IRStats2 that might need to be merged:
https://github.com/eprints/irstats2
https://github.com/eprintsug/irstats2
 
(Most other forks come from the head of eprints/irstats2).
 
Comparing these two forks:
https://github.com/eprints/irstats2/compare/master...eprintsug:master
it looks like there has been some re-arrangement of files (these seem sensible to me), and a couple of fixes.
 
Does anyone have any thoughts on whether these should be merged, so we have one main master again? Will doing this cause problems for people who have installed the package using the previous file-layout?
 
Cheers,
John*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
*** Archive: http://www.eprints.org/tech.php/
*** EPrints community wiki: http://wiki.eprints.org/
*** EPrints developers Forum: http://forum.eprints.org/