I’ve completed the maintenance I had planned for this tonight. Iglulik which is the server that hosts the mail spool and several virtual machines, including mail, has been sporadically running out of memory invoking the Linux OOM killer which kills the largest process and that is usually the mail server process however I don’t know that it is it that is consuming the memory as it appears normal sized in the kernel error messages. I suspect something else is but is very transient so by the time the killer walks the process table, the real culprit is gone.
I have upgraded this machine’s kernel from 4.4 to 4.8 kernel tonight, implemented some process limits, changed the kernel configuration so it should deny additional memory to what is eating memory instead of killing the virtual host. Also a new version of libvirt, the process that runs virtual machines, came out today. So if these things don’t fix the issue, I am hoping they will at least help me diagnose the cause.