I am with out power and Phones at my home office but if you call and leave a message it will be automatically transcribed and I can pick it up on my tablet.  Alternatel, you can create a ticket.

     It would be the understatement of the year or at least the week to say that the work did not go as anticipated.  The machine I was going to use as a basis for comparison, it also had an old BIOS and given it was the same as the other machine that kept losing it’s fan profile, I decided it was best to upgrade it as well.

     When I upgraded the BIOS it returned to the default settings which meant that I had to determine good settings from scratch.  While I was doing this another machine crashed.

     I don’t really know what caused the crashes but I found a number of problems.  For one thing VTd instructions were not enabled but should have been since we have kvm/qemu virtual machines on these boxes.  Not having it enabled isn’t a show stopper but it causes Linux to  more work to perform I/O in a virtual machine.

     Then I found an issue with an item that allows full 64 bit address decoding for I/O devices.  For some reason this breaks the built in graphics of the i7–6700k causing instability.

     I now have all the machines on current BIOS, ran some torture tests on the CPU and they ran clean.  So we’ll see how it goes.  I had to slow down the CPU’s slightly on two machines to get them to complete the torture tests without error.

     Virtual private servers will go down briefly so I can compare BIOS settings between a machine that is stable and the mail server which presently is not.  When I updated the BIOS to resolve a fan issue, it reset all settings to default.  I tried to set them back the best I could by memory but suspect didn’t do the best job so will be attempting to correct that tonight.

     The virtual private servers should only be down a few minutes, the mail server somewhat longer as I will be running memory diagnostics to eliminate the possibility of a bad DIMM.  It may take several hours of intermittent downtime to resolve these issues.

    The host machine which houses the mail spool, mail server virtual machine, and debian virtual machine, spontaneously booted today even though temperatures are correct.  So I am going to take it down around midnight tonight to run some diagnostics and make some adjustments.

     The BIOS had corrupted the Q-fan profiles and turned one chassis fan off entirely, put the others in a mode where they were just barely running so the machine was overheated.  A BIOS upgraded fixed the fan issues.

     While I was there and letting a torture test run for a while (8 copies of an AVX version of prime95 running small fft’s, about the most torturous thing you can throw at a modern CPU, I worked on some other issues and discovered the problem with postfix not starting on debian, missing postfix and postdrop groups and postfix user.  I’ve fixed this problem before but it turns out the post.install script of the postfix package distributed with stretch (and Ubuntu 16.04.3 LTS) is broken.  So it will likely break again the next time a postfix update is kicked out.

     I need to take the server that holds the mail spool down a few times briefly tonight to make some BIOS adjustments.  I believe I’ve got the CPU voltage just a touch too low and it is slightly unstable.  This may take several reboots and benchmarks to find the correct level.  It is presently unstable and has rebooted spontaneously twice in the last couple of days.  This is symptomatic of CPU voltage too low for the clock rate and I did adjust it down as low as I could get it to run, what I thought was stable, the last time I worked on it, so I’m going to crank it up slightly.

     Our shell server mint.eskimo.com has been upgraded to Sonya 18.2.  Initially I had some problems getting postfix working, chased it down to sendmail already being installed and not automatically deinstalled when postfix was installed, as it normally is on other debian derivatives.

     Also had issues with idmapd not working initially, this turned out to be an old issue with nsswitch.conf, compat does not work as advertised, it is necessary instead to specify nis files, which gives the behavior compat is supposed to but does not.

     The kernel upgrades are completed.  All host machines rebooted properly.  The longest took 3 minutes.  However, the web server did not properly NFS mount the /home directory after rebooting so some websites were unavailable for about 1/2 hour until I discovered and corrected this.

     I need to reboot the host machines which will cause a brief interruption in all services lasting 5-10 minutes if all goes well.  If all does not go well and I need to drive to the co-location facility, it could be as long as an hour but I have not had any failure to boots since the upgrade to Ubuntu 17.10 so I do not expect difficulties.

     I am taking the Linux Mint shell server, mint.eskimo.com, down for an upgrade from Serena 18.1 to Sonya 18.2.  Serena has been particularly problematic with NIS always failing to bind because the ordering of applications starting is not right in systemd, in particular systemd does not wait for portmap to start before trying to start ypbind.  Early versions of Ubuntu with systemd had similar issues, I am hoping 18.2 will resolve this.

     Because Mint online upgrades generally do not work well, particularly when third party repositories are concerned, and because there has been a change of hardware architecture from an i7-2600 to an i7-6850k which supports some additional instructions, I am doing a fresh install which will require recompiling some third party apps.  So it will likely be down most of this evening.

     Please consider using debian or ubuntu as alternatives, like Mint they are debian or debian derived but generally work better.  Hopefully Sonya will function better.