I have a dozen or so boxes, mostly little raspberry pis, out in people’s houses which let me do backups for them, and attach to their networks. I’ve documented this before in “Gateway pi”, “Memory on the Gateway Pi”, and “Timemachine on Gateway pi” for example.
Connection between these boxes and my house is with SSH, and I use openssh certificates as described in “Using openssh certificates” and “Re-signing Openssh Certificates”. However, there has always been a little nagging problem, which is that these boxes must (re-)establish their connection to me automatically upon reboot, without user intervention. This means that the private keys that accompany the certificates cannot be encrypted, for that would require human intervention.
So there is a risk. Those raspberry pis have upon them a certificate and a private key which would enable access to boxes in my house. Not completely unrestricted access, and not root access, but nevertheless.
Continue reading Protecting ssh
This is a follow up to the activities in IPv6 implementation, which was published on March 2nd and revised up through March 19th, as new challenges were addressed. Since March 19th a great deal of what I wrote has been revised, as I have learned a lot more.
The main issue was that there remained a number of problems with the implementation of IPv6 in my residence.
Continue reading IPv6 Re-implementation
- The biggest was the question how to handle the delegated prefix, particularly in renumbering. Over the course of the last several months I have to note that Comcast has never changed my prefix, except early on, when I forced it to do so by changing my DUID. And I don’t think it likely that my prefix would change unless some great catastrophe befalls which results in my being down for a very extended period – like 30 days; or more likely there is some change in my service (a change in ISP, or perhaps fiber arriving in my area).
- The first implementation required that I make patches to the code of my router. This meant that I would have to figure out how to carry those patches forward in the event of firmware updates from Ubiquiti, the maker of the Edgerouter-X that I am using.
- The implementation was pretty fragile, with a lot of unrelated bits in different places. In particular there was a lot of hand-waving in trying to assign and maintain a separate network for the virtual machines on one of the interior boxes.
I was revising some things in my startup scripts. I have a sort of generalized startup script in all the boxes in my constellation, which is capable of doing 8 or 10 different things that various of the boxes need to do at startup.
For example, the various gateway boxes need to open up (auto)ssh connections to my house with reverse tunnels so I can reach them. On some boxes I want them to open a vncserver so I can get a graphical environment up. On some others they may need to mount some filesystems, with smb or nfs. On some of them I want them to figure out where their router is, in case I want to open up their router in a browser. On some I need them to establish the keychain.
Continue reading Waiting for networks
Some people for whom I provide some kinds of support with gateway pis, use Macs. For the pc folk – at least for those on Windows 10, I’ve been seting up to do the filehistory thing, and putting the filehistory onto the /backup drive on the gateway pi. Then it gets sent here overnight. I wanted to do the same for the folks who have Macs, of which there are several.
Continue reading Timemachine on Gateway pi
I just built another little gateway pi, on Raspbian. It is a newer Raspbian than I used for the others. And newer is better, right? But when the little fellow came up, postfix claimed it couldn’t relay through tarragon, because it couldn’t accomplish login. Sasl authentication failed, no worthy mechs.
I remember good old “no worthy mechs” from way back when, always thought it a very cool error message. But why in the heck am I getting this? I’m not doing anything fancy.
I had a vague nagging feeling from some old Fedora problems, long ago. Could it be that I have to install some kind of sasl library, even to do plain authentication? Poked around a little. Eventually did an apt install libsasl2-modules and sasl2-bin. Sure enough, they actually installed.
And afterwords postfix came up and send the mail out of the queue. I’ll be dipped in … I am surprised. This was not something I had to do before. Is this an improvement in Raspbian. Don’t package any sasl mechs, make the poor sod figure out why sasl authentication won’t work.
This is to help me remember the next time this happens.
Seldom do I get to write a post where I am offering information which might not actually be out there in a lot of places. I could not find this information on the web, and had to figure it out myself, by reading the code, and doing experiments.
I talked in the last post about the need to re-issue all the openssh certificates, in order to update the hash algorithm used for the signatures. My way of maintaining the certificates, in my repository, would make it easy for the signing box to get all the existing certificates, but not (directly) the public keys that are inside those certificates.
Continue reading Re-signing Openssh Certificates
I’ve encountered a problem migrating from Fedora to Arch which ends up being about ssh and openssh certificates. I look back and discover that I never posted anything about my movement toward openssh certificates. Curious because I wrote a lengthy document about it (because of my leaky brain – not because I am any kind of authority on it).
I will probably go back and write a post about it, and back date it. But now a problem has arisen. Rather than explain, let the boys at openssh speak for themselves, in the release notes for openssh 8.2:
Continue reading SSH Certificate signing
I now have 8 of these gateway boxes out there. This morning as I was checking backups on one of them, I observed that it took quite a long time to respond. I ran a top on it and was horrified to see that its memory use was 100% and so was its swap. Holy @#$%!@ Batman!
Most of the memory was being used by the lxpanel. And (hangs head in embarrassment) there were actually two lxpanels running – one for the console and one in the vnc window I launch at startup.
It seems the lxpanels leak. I don’t know how badly, but it doesn’t matter. These boxes are meant to run forever so even a tiny leak is eventually fatal.
Well this was simple. I will seldom, if ever, need to get into a graphical environment remotely, and if I do I can always start vnc from the command line. So I took out the startvnc from the startup script. And I have even LESS need for a graphical console since there is not even a monitor on these things. So I set the default systemd target to multi-user.target.
Did this on all the gateways that are running on pi-zeros. Those few running on bigger ubuntu boxes I didn’t really have the problem anyway.
After rebooting them they come up with no lxpanels. I’ll watch the memory use, but I think this will fix the problem.
I’m writing this in February 2020, but I should have posted it last April. This is about using openssh certificates for authentication.
For many years I have used exclusively public-key authentication for ssh connections. None of the boxes I support ever allows password authentication for ssh. But as the number of boxes in the constellation has grown, the problem of keeping the public keys up to date has become nightmarish. If I really truly needed NxN capability it would be unmanageable. The truth is though that there are only 3 or 4 boxes from which I typically establish ssh connections to all the others.
Still it is – if not a full-blown nightmare, at least a bad dream. The folks at openssh implemented a mechanism to make it easier: certificates. These are not full blow x509 certificates, they are simpler, but the idea is the same. The problem: how do we arrange to get a public key on to some box with confidence that the public key belongs to the party it purports to represent? The certificate answer is to have the public-key signed by a party that the recipient trusts.
So how does it work. I designate a box as my “signer”. All the other boxes in my world trust the signer. Instead of putting public keys on every box, instead every box has its own public key signed by the “signer”. When it wishes to connect to another box (in my own constellation of boxes) it presents its certificate – i.e. its signed public key. Since the recipient trusts the signer, it will accept the signed public key as being the genuine public key, and it can then proceed to authenticate.
How it works mechanically is that when I initialize a new box, and create a new key pair for it, I have to send the new public key to the signer. Then, on the signer system I sign the key, and send it back to the new box. I’m actually using my repository for the moving of this stuff back and forth. I think this is ok, since the only thing that ever appears in the repository are public keys and public certificates. The public keys exist in the repo for mere moments. The certificates stay there. But the private keys of course never leave the box on which they originate.