EPrints Technical Mailing List Archive

Message: #06388

< Previous (by date) | Next (by date) > | < Previous (in thread) | Next (in thread) > | Messages - Most Recent First | Threads - Most Recent First

Re: [EP-tech] Apache log getting a lot of errors and Mysql Going away

Hi John!

Thanks a lot for pointing this out!  I checked the access log and found the culprit.  Someone from another university here probably has a script or something that harasses our server every minute or so.  It was trying to load an Atom and a RSS export on our thesis types.

I have contacted the said university so they can look it up and I have blocked the ip with iptables.

Thanks all for your help!  This techlist rocks!

Have a nice day all!

Mathieu Perron
Chargé de projet informatique et technologique

Service des technologies de l'information 
Université du Québec à Chicoutimi
555, boul. de l'Université
Chicoutimi (Québec) G7H 2B1

-----Message d'origine-----
De : eprints-tech-bounces@ecs.soton.ac.uk [mailto:eprints-tech-bounces@ecs.soton.ac.uk] De la part de John Salter
Envoyé : 28 mars 2017 05:51
À : eprints-tech@ecs.soton.ac.uk
Objet : Re: [EP-tech] Apache log getting a lot of errors and Mysql Going away

Yep - something that regular looks very much like a robot!

I would investigate the access logs - and try to match the entries in the error log with requests. The timestamps might not match up exactly - but you should be able to see the same patterns.
In the error log you just get /eprints3/cgi/exportview - but in the access log you should get e.g. /cgi/exportview/[VIEWID]/[VALUE]/[Export Format]/[filename]

You should be able to see what is being requested. You can see how quickly the request responds for you (and also if there are things you can do to speed it up).

Try (note spaces around the 500!):
> grep 'exportview' [APACHE ACCESS LOG] | grep ' 500 ' 

The XML generated from an 'export' of a view can be quite large, and also take quite a while to process.

Cheers,
John


-----Original Message-----
From: eprints-tech-bounces@ecs.soton.ac.uk [mailto:eprints-tech-bounces@ecs.soton.ac.uk] On Behalf Of Yuri
Sent: 28 March 2017 09:37
To: eprints-tech@ecs.soton.ac.uk
Subject: Re: [EP-tech] Apache log getting a lot of errors and Mysql Going away

Seems like search engines, tryng to export something huge.

The 

Il 27/03/2017 20:21, Mathieu Perron ha scritto:
>
> Hi all,
>
> Thank you for your advice.  I have noticed something in my error log.  
> The "connection abort" errors always happen at the same intervals.
>
> Here's a look at a part of my log:
>
> [Mon Mar 27 13:38:11 2017] [error] Software caused connection abort at 
> /eprints3/cgi/exportview line 108.\n
>
> [Mon Mar 27 13:39:55 2017] [error] Software caused connection abort at 
> /eprints3/cgi/exportview line 108.\n
>
> [Mon Mar 27 13:40:10 2017] [error] Software caused connection abort at 
> /eprints3/cgi/exportview line 108.\n
>
> [Mon Mar 27 13:41:55 2017] [error] Software caused connection abort at 
> /eprints3/cgi/exportview line 108.\n
>
> [Mon Mar 27 13:42:11 2017] [error] Software caused connection abort at 
> /eprints3/cgi/exportview line 108.\n
>
> [Mon Mar 27 13:43:54 2017] [error] Software caused connection abort at 
> /eprints3/cgi/exportview line 108.\n
>
> [Mon Mar 27 13:44:10 2017] [error] Software caused connection abort at 
> /eprints3/cgi/exportview line 108.\n
>
> [Mon Mar 27 13:45:53 2017] [error] Software caused connection abort at 
> /eprints3/cgi/exportview line 108.\n
>
> [Mon Mar 27 13:46:10 2017] [error] Software caused connection abort at 
> /eprints3/cgi/exportview line 108.\n
>
> There's 2 recurring time gaps (1 min 45 secs then 15 secs then it 
> starts over.)
>
> I'm a little lost where I should look to pinpoint what is causing this.
>
> I looked at my crons jobs and nothing there.
>
> All help is appreciated!
>
> Thank you again!
>
> *Mathieu Perron*
> Chargé de projet informatique et technologique
>
> *De :*eprints-tech-bounces@ecs.soton.ac.uk
> [mailto:eprints-tech-bounces@ecs.soton.ac.uk] *De la part de* Matthew 
> Kerwin *Envoyé :* 27 mars 2017 07:06 *À :* 
> eprints-tech@ecs.soton.ac.uk *Objet :* Re: [EP-tech] Apache log 
> getting a lot of errors and Mysql Going away
>
> Yeah, for what it's worth we've had a history of these sorts of 
> errors. Eventually I resolved a lot of it by reducing the number of 
> Apache processes and recycling them a little more aggressively.  In 
> our case, it was a choice between: a few failed connects, or a lot of 
> successful connects that don't go anywhere.
>
> Your mileage may vary, of course.
>
> Cheers
>
> --
>
> *Matthew**Kerwin*  |Senior Web Developer|Applications & 
> Development<https://wiki.qut.edu.au/display/lib/Digital+Repository+Tea
> m>
>
> ----------------------------------------------------------------------
> --
>
> *From:*eprints-tech-bounces@ecs.soton.ac.uk
> <mailto:eprints-tech-bounces@ecs.soton.ac.uk>
> <eprints-tech-bounces@ecs.soton.ac.uk
> <mailto:eprints-tech-bounces@ecs.soton.ac.uk>> on behalf of Yuri 
> <yurj@alfa.it <mailto:yurj@alfa.it>>
> *Sent:* 27 March 2017 17:03
> *To:* eprints-tech@ecs.soton.ac.uk 
> <mailto:eprints-tech@ecs.soton.ac.uk>
> *Subject:* Re: [EP-tech] Apache log getting a lot of errors and Mysql 
> Going away
>
> I think it happens when you've a lot of connections on mysql. Rise up 
> the number of connection in mysql and use some script in a cronjob to 
> check if mysql has gone away and notify you.
>
> The problem can go even worse when you've a lot of Apache threads and 
> it eats al the RAM available, you've to rise or lower the apache 
> threads according to the load.
>
>
> Il 24/03/2017 19:26, Mathieu Perron ha scritto:
> >
> > Hi,
> >
> > We are using ePrints v3.3.13 and lately we have been experiencing 
> > some crashes due to MySQL going away.  I looked in the Apache log 
> > and I see a lot of the following errors:
> >
> > "[error] Software caused connection abort at 
> > /eprints3/cgi/exportview line 108.\n"
> >
> > I looked at the cgi script and searched in the files and this script 
> > has something to do with the Views.pm module.  I can't seem to make 
> > the error happen on demand and I'm wondering if anyone could help me 
> > with this.  I don't know if the MySQL server going away is related 
> > to these errors or not.
> >
> > Thank you for your time!
> >
> > *Mathieu Perron*
> > Chargé de projet informatique et technologique
> >
> >
> > Service des technologies de l'information Université du Québec à 
> > Chicoutimi 555, boul. de l'Université Chicoutimi (Québec) G7H 2B1
> >
> > *Mathieu_Perron@uqac.ca* <mailto:*Mathieu_Perron@uqac.ca*>
> <mailto:Mathieu_Perron@uqac.ca>
> >
> >
> >
> > *logo uqac* <http://www.uqac.ca/>
> >
> > Le présent document (y compris les pièces qui y sont annexées, le 
> > cas
> > échéant) s'adresse au destinataire indiqué et peut contenir des 
> > renseignements de caractère privé ou confidentiel. Si vous n'êtes 
> > pas le destinataire de ce document, nous vous signalons qu'il est 
> > strictement interdit de le diffuser, de le distribuer ou de le 
> > reproduire. Si ce message vous a été transmis par erreur, veuillez 
> > en informer l'expéditeur et le supprimer immédiatement. Avant 
> > d'imprimer, pensez à l'environnement!
> >
> >
> >
> > *** Options: 
> http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
> > *** Archive: http://www.eprints.org/tech.php/
> > *** EPrints community wiki: http://wiki.eprints.org/
> > *** EPrints developers Forum: http://forum.eprints.org/
>
>
> *** Options: 
> http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
> *** Archive: http://www.eprints.org/tech.php/
> *** EPrints community wiki: http://wiki.eprints.org/
> *** EPrints developers Forum: http://forum.eprints.org/
>
>
>
> *** Options: 
> http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
> *** Archive: http://www.eprints.org/tech.php/
> *** EPrints community wiki: http://wiki.eprints.org/
> *** EPrints developers Forum: http://forum.eprints.org/


*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
*** Archive: http://www.eprints.org/tech.php/
*** EPrints community wiki: http://wiki.eprints.org/
*** EPrints developers Forum: http://forum.eprints.org/

*** Options: http://mailman.ecs.soton.ac.uk/mailman/listinfo/eprints-tech
*** Archive: http://www.eprints.org/tech.php/
*** EPrints community wiki: http://wiki.eprints.org/
*** EPrints developers Forum: http://forum.eprints.org/