Tonight I am going to perform hardware surgery on the machine that hosts home directories.  As a result, ALL services except virtual private servers will be down for a number of hours.

     The server which hosts the /home directory partition has an ill drive in the RAID array for this partition.  It has about seven bad sectors which if they were HARD failed would not be a big deal, the drive would re-map them and life would go on, but they aren’t.  Instead if you write them and read immediately they will pass but sometime in a week or so following reads will fail again.

     If the mate to this drive in the RAID array were to fail, this would result in data corruption so I’m going to replace this drive tonight.

     The other issue, when I tried to get the kernel upgraded on this machine last night, the drivers for the Fusion I/O flash drive would not compile under the 5.16 kernel.  Earlier kernels have a bug that can result in either data corruption or privilege escalation, either of which are undesirable.  The Fusion I/O folks tell me it may be a while before drivers are fixed as there were extensive changes to the kernel.

     So I am going to replace the fusion I/O drive with a Western Digital Black 1TB drive which is natively supported by the Linux kernel.  This drive is much larger so I am going to put both the root file system and boot block and database on it.  It will take some time to copy all this data and change the boot block to this drive.  The database copy should go fast as it is flash-to-flash but the rest will take several hours.  This drive also does not have a conflict with the Broadcom NIC card, so when this is completed and I can remove this drive, I can restore the Broadcom NIC which handles hardware offloading properly.

     This will affect ALL services EXCEPT for virtual private servers which do not depend upon the site wide /home directories.  It will affect all web services including virtual domains, hosting packages, and virtual domains.  I had hoped to put the new server in place and just transfer these services over to it and then fix the old but the security flaw being publicized in the Linux kernel no longer affords me this luxury.

     This will also affect https://friendica.eskimo.com/, https://hubzilla.eskimo.com/, https://nextcloud.eskimo.com/, and our main website https://www.eskimo.com/.