Linux Security Exploit (Iglulik was attacked)

Warning!!!! This security exploit has not been widely published but it IS actively being exploited. Someone caused my server that houses our customers /home directories to spontaneously reboot trying to exploit it. Fortunately the kernel logged their attempts. See: https://www.kernel.org/doc/html/latest/admin-guide/l1tf.html In our cause I performed measurements on system load, web page loading times and latency with and without this CPU feature turned off and in our case it made no measurable difference so I turned it off with: echo ‘off’ > /sys/devices/system/cpu/smt/control. I put this in /etc/rc.local which is enabled on our machines for this and some other adjustments.

Iglulik Crash

     Iglulik, our system which houses /home directories and some virtual machines spontaneously rebooted.

     Examination of the logs suggest someone tried to take advantage of a CPU exploit similar to Meltdown.  I am looking into several possible mitigation strategies, all of them have performance impacts.

Debian

     Debian’s key database has gotten badly corrupted.  The machine is down at present and is being restored from backups.  It should be available again in about an hour.

     In the meantime, please use one of the other debian derived machines such as julinux.yellow-snow.net, mint.eskimo.com, mxlinux.eskimo.com, ubuntu.eskimo.com, or zorin.eskimo.com.

Centos 7

     CentOS 7 stopped accepting connections today.

     I found ypbind was not bound so it could not access the NIS database to get user authentication information.

     However, ypbind would not rebind on restart.  Usually the cause of this is portmapper (rpcbind) not running.  I checked, it was but I restarted it anyway, still ypbind would not bind.

     At this point I attempted to reboot the machine, systemd got stuck during shutdown and would not complete the shutdown process so I was forced to hard boot it.

     There were two newer kernels than the one it was running on.  It is now running on the newest and seems to be operating normally again.

Maintenance Sunday Morning Midnight – 0400 AM

     I will be rebooting all servers early Sunday morning to make a kernel upgrade effective, and then taking select servers down for a period to perform imaging, a form of backups that basically captures the entire state of a machine so it can be quickly restored in the case of severe corruption.

     Brought to you by: Run On Sentences R Us.

Zorin

     I was never able to get Gnome to work on Zorin, Gnome is usually one of the easier desktops to get functioning.  I am going to re-install Zorin on the assumption that something probably went wrong.  It will be down for a couple of days.  We will also be going from the free Zorin Core to the paid Zorin Ultimate.  It is reasonably priced at about $21 US ($19 EU) and includes a Gnome 2 and Mac OS Desktop layout as well as 1.5GB of other software not included in core so I think it will be a good upgrade.

 

MxLinux

      The shell server mxlinux is now back online.  It is also now available from the the web and audio works.  Not all of the application software has been re-installed yet, this will take some time.  If there is something you need, please let me know and I will prioritize it.

CentOS 6 – Sublime-Text

     The application sublime-text has been removed from CentOS 6 because the authors have changed it to required GTK-3 and CentOS 6 has Gnome-2 and thus no GTK-3.

     If you need this application use ANY other shell server EXCEPT Centos 6 and Scientific Linux both of which have Gnome 2.