As you may or may not have noticed, I’ve been working on the website.  If you have any input with respect to issues or things you’d like to see on it, please let me know.

E-mail support@eskimo.com

I will be a bit late into the office today as I have to make a trip to the co-location facility to resurrect a broken Radius server.  Radius infrastructure is duplicated so this should not affect authentication except to delay it slightly if it is the first server tried.

Starting around 9pm, I will be taking various servers down for reboots (kernel updates) and imaging.  The main file server won’t be done until after midnight, and same for mail and web servers, but other shell servers will be done earlier starting with the least busy.

This work will take until about 2AM and might possibly be continued on Saturday evening if everything isn’t finished Friday evening / Saturday morning.  All the Redhat machines have kernel updates and a huge number of updates (over 200 each) to install, the Debian based systems mostly just need to be imaged.

Debian is broken.  I installed X2Go and FreeNX and then I needed to reboot but couldn’t because the shutdown program was not installed.  I installed that, rebooted, but somehow the virtual machine was corrupted beyond restoration.  I’m recovering from a backup but it will take some more work to get everything back to current so this machine may be down overnight.  Please use Ubuntu or Mint if you need a Debian-based machine in the meantime.

There is a new virus going around that clamav does not have a signature for yet and thus is not catching.  Our spam filtering software does recognize it as spam and is putting it in your spam folder.  If you find e-mail in your spam folder with an attachment, I strongly recommend against opening it as it most likely contains a virus.

The volume of this virus is so large that it has overloaded our incoming mail servers and caused mail to be backed up in queue.  I was able to make some adjustments that improved the mail servers ability to process mail and it is catching up, but as of 2AM it has worked through about half the queue, so at this rate it will probably be caught up by around 5AM.

Hopefully the clamav folks will kick out a database that includes a signature for this thing and put an end to it, but to prevent your machine from being infected, do not open any attachments you are not expecting and keep your anti-virus database up to date.

Debian is back up and functioning.  I’m still applying updates that were rolled back but the corruption is resolved and it is operational.

Our shell server debian.eskimo.com is down for repairs.  The ‘apt’ database has somehow gotten corrupted and I’m unable to figure out how to correct it so I’m reverting the machine to a image made prior to it’s corruption and then will re-apply updates.  This should require about 45 minutes.

After the power glitch this morning when everything rebooted, it left some of the MySQL tables corrupted.  I didn’t notice until I tried to go into the Forums just now and I received no complaints about broken blogs, etc.

I’ve run myisamchk -r on all the tables with the server down to repair any inconsistencies and brought the server back online.  Forums are working now.

If you notice any unusual behaviour pertaining to MySQL, please notify support@eskimo.com.

All of our machines simultaneously rebooted at 10:17AM this morning.  Isomedia claims there was no power hit at the facility but given that these machines have been otherwise rock steady I am sceptical.  I have opened a trouble ticket on the issue.

OpenSuse is down for a total rebuild.  An update today left it in pretty much an unusable state and I didn’t have a good backup since I was still installing after the build.

Because I have to deal with power issues at the co-lo facility, this probably will not happen tonight.