I now have 8 of these gateway boxes out there. This morning as I was checking backups on one of them, I observed that it took quite a long time to respond. I ran a top on it and was horrified to see that its memory use was 100% and so was its swap. Holy @#$%!@ Batman!

Most of the memory was being used by the lxpanel. And (hangs head in embarrassment) there were actually two lxpanels running – one for the console and one in the vnc window I launch at startup.

It seems the lxpanels leak. I don’t know how badly, but it doesn’t matter. These boxes are meant to run forever so even a tiny leak is eventually fatal.

Well this was simple. I will seldom, if ever, need to get into a graphical environment remotely, and if I do I can always start vnc from the command line. So I took out the startvnc from the startup script. And I have even LESS need for a graphical console since there is not even a monitor on these things. So I set the default systemd target to multi-user.target.

Did this on all the gateways that are running on pi-zeros. Those few running on bigger ubuntu boxes I didn’t really have the problem anyway.

After rebooting them they come up with no lxpanels. I’ll watch the memory use, but I think this will fix the problem.