Server

     Found that the entire web server was not down, only mariadb (what used to be mysql).   MariaDB is necessary to our site and any site which uses WordPress or other content management engines.

     I’ve added a script to crontab now that checks once a minute and restarts it if it dies.  But there is no indication in the logs of what caused it to die.  The only errors logged where an attempt to access a non-existent database, but doing that does not cause MariaDB to die so I still do not know what did.

 

 

Web Server Crash

     Our web server crashed some time this morning.  Only one call so I was unaware that it was down.  I’m going to try to put together some means of automatic monitoring.

     I do not yet know the cause of the crash.  We have been hit with denial of service attacks for the last few days but I do not know if this was related.

     It is back in service now but there are some database errors I am still investigating.

ns3.eskimo.com

     One of our name servers, ns3.eskimo.com, is currently down.

     This was due to a combination of DNSSEC root-key change and my attempts to being lazy in resolving it.  I attempted to purge / re-install the bind package thinking that would get me fresh conf files including the DNS key file however, Redhat had removed the package from the CentOS repository so I could not re-install.

      I am in the process of moving this off of the CentOS6 server onto a Ubuntu based server.  This may cause slow name resolution at times if ns3 is tried first until this is completed.

Zorin Upgrade

     Zorin is presently down for an operating system upgrade.  Unfortunately, Zorin provides no online upgrade path, it is necessary to do a full re-install so it will be down for a few hours.

Web Server Upgrade

     I upgraded our web server operating system from Ubuntu 17.10 to Ubuntu 18.04 tonight.

     I replaced our previous database, MySql 5.7 with MariaDB 10.3.  This was not entirely smooth owing to the existence of mysql in the NIS database conflicting with the local user that the install script wanted to create and NOTHING in the error messages gave a clue as to why ti was failing.

     When all was said and done, this shaved about 150ms off of the time to first byte and about half a second off our total page load time so it was worthwhile.

Router Replaced

     Router arrived last night.  Configured and brought it up to current software release and installed tonight.  All work is done, everything is back up and running normally.