Fedora Crash, again

Preparing to go off on my semi-annual visit east I was trying to ensure that the primary systems here that have encrypted root drives (oregano, cinnamon and rosemary) could each be rebooted from afar by attaching to a dropbear instance during the initramfs. See article on booting notes about that.

Somehow Oregano became unbootable. Again. It usually takes three or four hours of flailing around to figure out what little thing has caused it to point its casters to the sky. It takes only a little longer to just rebuild from scratch with the latest release.

Continue reading Fedora Crash, again

Using openssh certificates

I’m writing this in February 2020, but I should have posted it last April. This is about using openssh certificates for authentication.

For many years I have used exclusively public-key authentication for ssh connections. None of the boxes I support ever allows password authentication for ssh. But as the number of boxes in the constellation has grown, the problem of keeping the public keys up to date has become nightmarish. If I really truly needed NxN capability it would be unmanageable. The truth is though that there are only 3 or 4 boxes from which I typically establish ssh connections to all the others.

Still it is – if not a full-blown nightmare, at least a bad dream. The folks at openssh implemented a mechanism to make it easier: certificates. These are not full blow x509 certificates, they are simpler, but the idea is the same. The problem: how do we arrange to get a public key on to some box with confidence that the public key belongs to the party it purports to represent? The certificate answer is to have the public-key signed by a party that the recipient trusts.

So how does it work. I designate a box as my “signer”. All the other boxes in my world trust the signer. Instead of putting public keys on every box, instead every box has its own public key signed by the “signer”. When it wishes to connect to another box (in my own constellation of boxes) it presents its certificate – i.e. its signed public key. Since the recipient trusts the signer, it will accept the signed public key as being the genuine public key, and it can then proceed to authenticate.

How it works mechanically is that when I initialize a new box, and create a new key pair for it, I have to send the new public key to the signer. Then, on the signer system I sign the key, and send it back to the new box. I’m actually using my repository for the moving of this stuff back and forth. I think this is ok, since the only thing that ever appears in the repository are public keys and public certificates. The public keys exist in the repo for mere moments. The certificates stay there. But the private keys of course never leave the box on which they originate.

New Internal Network setup

Owing to the failure of oregano detailed in the last post, I have finally taken steps to clean up a long standing issue in my internal network, viz: that oregano, the primary development computer was offering essential network services which all the other boxes relied upon. When oregano was down, almost everything else suffered.

This problem dates back at least 20 years. In early days I began the practice of having my primary linux computer act as a firewall separating the rest of the network from the internet, and as the dhcp server. I won’t try to defend the practice – it was what I did; but it has made less and less sense over the years. Plus it had the very undesirable side effect that when that primary computer was down the other systems lost their dhcp server and their path to the net.

I had this generic Chinese openwrt router which I bought last year, for reasons passing understanding. I’d planned to replace the primary router with it, but that proved a bad plan. I decided to use this extra router to fill the role oregano had filled, of separating the internal and external networks.

So this router, named obelisk, performs dhcp and dns forwarding. Henceforth Oregano will be just another box on the internal network.

Automatic update of Fedora Fails

I tried doing an update of fedora the other day, with the dnf system upgrade business, to upgrade in place. I should have known better. The failure is almost certainly related to the ongoing frustration of the graphics card.

One is offered the choice of two nice poisons. One may elect to use the open source nouveau driver, in which case the graphics driver will spontaneously crash about once a week forcing a reboot. Or, alternatively, one may choose instead to install the proprietary nvidia driver, in which case every few months one will get a new kernel that invalidates the driver, and the machine will suddenly not boot at all, requiring that you get in and fool around with grub and intitramfs until you can get it up enough to download and rebuild a new nvidia driver. Continue reading Automatic update of Fedora Fails

Certificates Redux

An earlier post talked about switching my server tarragon (where this blog sits) to a wildcard certificate from letsencrypt. There were two reasons why I was using a wildcard certificate. One had to do with test versions of websites that run on this server, and the need that some of those sites have for wildcards, of the form: bob.websitename.com, sally.websitename.com, etc. The other reason was that I have a lot of hosts (oregano, cinnamon, paprika, lemongrass) in addition to tarragon that “need” to have a certificate, for https, for imap, and for smtp, and when I was having to pay for them, it was cheaper to get one wildcard for wmbuck.net. Continue reading Certificates Redux

Dynamic DNS

I have used dynamic dns for around 20 years, I think. But I have always used dyndns.com which these days seems to want to call themselves dyn.com. And some years back they were bought by Oracle, the kiss of death, and now they are impossible to deal with, arrogant, unsupportive, insular – all the things I expect of Oracle.

And why have I kept using them? Because that is what the routers supported. Dyndns was  there first, and the ubiquitous linksys and netgear routers usually have a feature to do automatic updates for dynamic dns, but (often) the router will only update dyndns: nobody else. And I’ve got routers installed in various people houses that are doing this.

But I recently added a new house that I support, and that person has a proprietary and ponderous comcast router, which will barely do anything useful, and has no facility to update dynamic dns.

Continue reading Dynamic DNS

Apache Quiet Failure on Certificate/Key mismatch

In an earlier post I related how I had moved to letsencrypt for tarragon. In the process of doing some cleanup of the /etc/letsencypt directory, and my repository, I managed to stupidly get one wrong private key file into the batch of all the https vhosts, such that the http config file for xyz.com specified an SSLCertificateFile and SSLCertificateKeyFile which did not match.

It took me hours and hours to figure this out, because Apache simply fails to start and gives no indication whatever what has pissed him off. I wasn’t too stupid to figure out that I had been messing with the certs yesterday, and the problem might lie there somehow. But I have about 15 vhosts, so it was tedious. In the end I resorted to strace, and saw the problem.

 

LetsEncrypt Wildcard Certificates

Comodo is after me to renew, offering a free year. The last time I attempted to install a wildcard certificate from Lets Encrypt, shortly after they introduced the feature, I wasn’t able to figure it out. Now, 9 months later, there is a lot more information about how to do it. Before spending the money for a commercial cert, I thought I would give it a try.

I used the following on tarragon:


certbot certonly \
--server https://acme-v02.api.letsencrypt.org/directory \
--manual
-d wmbuck.net -d *.wmbuck.net \
--preferred-challenges dns

It is important that the server url by v02, because v01 servers can’t issue wildcard certs. I had to put TXT records in the DNS for them to verify, and they created the cert into the /etc/letsencrypt/live directory where all the others are.

This was trivially easy. Goodbye Comodo.

GDM fails to start

Experimenting with VMs on Ubuntu, I had the system running on a relatively small root disk. Was troubleshooting problems in the disk array, doing a lot of booting, and crashing, when suddenly the boot wouldn’t finish. Fortunately the boot wasn’t quiet and I could watch as it tried 20-30 times to start the gdm service.

I change the default to multi-user.target and got it up, and what do you know, the root filesystem is at 100%. Cleaned up some logs and crash logs and it came right up.

Interesting that it doesn’t have a way to at least alert you that is what its problem is.