Things are mostly back up now.  I will be checking for proper NFS/NIS mounts over the next couple of hours but things should be basically functional.

     The machine which hosts the mail spool, client mail server, and some other hosts is currently down.  It failed to reboot because of missing software prohibiting it from accessing the Internet.  Since it can not access the Internet I can not restore the missing software that way.  I attempted to restore from backups but do not know all of the files involved in the package.  I am burning a DVD and will be heading back down to the co-lo to complete the restoration once I have that in hand.

     Rebooting all servers this morning (Sunday July 21st) to make upgrades effective.

     Borg Backup is installed on all of our shell servers at the request of one of our customers.

     We make backups automatically of all user home directories and e-mail weekly.  And both of these things are also on RAID10 partitions so data will not be lost in the event of a drive failure.

     But if you need things backup more frequently than that you can use this to backup data to your machine or your machine to your file space here.

     Mail is now operational on Debian however, elm will not be available.  I can not get elm-2.5.8, the last official version, to compile.  C syntactical requirements have just changed too much for even clang to properly grok the code anymore and that’s always been my go-to compiler for antique code.

     If there are other applications you would like that are not there, please e-mail support@eskimo.com or generate a ticket requesting their installation.

     Debian is back up.  Mail is not yet configured.  Many applications still being installed.

     All the desktops are presently installed.  Mail is not yet configured.  You will not yet be able to send or receive e-mail from Debian.

     Applications are still installing.  There are lots.  This will take some time.

     Fedora was shut down unintentionally, it is back online.

     One of our name servers, ns3.eskimo.com, was down earlier.  I have restored it to service.  There were three others that were operational so this should have only caused a temporary delay in resolution if it happened to be the first chosen.

     A short while after I fixed Debian it ate itself again.  This time leaving itself in an unbootable state.  Rather than restoring from a stretch backup, I am doing a fresh install.  So much changed between stretch and buster that it is not surprising the upgrade wasn’t entirely smooth.

     It may take several days to get Debian back into a fully usable state.  In the meantime, other machines which provide a similar modern Debian based environment include Ubuntu, Mint, JuLinux, Zorin, and MxLinux.

     Around 7:15 PM this evening Debian networking went away.

     I logged into the console and everything at first appeared normal but I could not ping or connect to anything outside.

     I tried restarting networking, it was uneventful but did not restore connectivity.  I think restarted NetworkManager but it failed to start with the error stating it had been restarted too frequently.

     At this point I rebooted the machine.  After it came back up there was still no networking but the NetworkManager was working again.  I went in and found the Ethernet device was just gone.  I recreated it and now everything is working again.

     This must be a new NetPlan feature.