Category Archives: boot issues

boot issues, fedora, linux, networking, security

New Internal Network setup

Owing to the failure of oregano detailed in the last post, I have finally taken steps to clean up a long standing issue in my internal network, viz: that oregano, the primary development computer was offering essential network services which all the other boxes relied upon. When oregano was down, almost everything else suffered.

This problem dates back at least 20 years. In early days I began the practice of having my primary linux computer act as a firewall separating the rest of the network from the internet, and as the dhcp server. I won’t try to defend the practice – it was what I did; but it has made less and less sense over the years. Plus it had the very undesirable side effect that when that primary computer was down the other systems lost their dhcp server and their path to the net.

I had this generic Chinese openwrt router which I bought last year, for reasons passing understanding. I’d planned to replace the primary router with it, but that proved a bad plan. I decided to use this extra router to fill the role oregano had filled, of separating the internal and external networks.

So this router, named obelisk, performs dhcp and dns forwarding. Henceforth Oregano will be just another box on the internal network.

boot issues, fedora, linux

Automatic update of Fedora Fails

I tried doing an update of fedora the other day, with the dnf system upgrade business, to upgrade in place. I should have known better. The failure is almost certainly related to the ongoing frustration of the graphics card.

One is offered the choice of two nice poisons. One may elect to use the open source nouveau driver, in which case the graphics driver will spontaneously crash about once a week forcing a reboot. Or, alternatively, one may choose instead to install the proprietary nvidia driver, in which case every few months one will get a new kernel that invalidates the driver, and the machine will suddenly not boot at all, requiring that you get in and fool around with grub and intitramfs until you can get it up enough to download and rebuild a new nvidia driver. Continue reading Automatic update of Fedora Fails →

boot issues, linux, ubuntu

GDM fails to start

Experimenting with VMs on Ubuntu, I had the system running on a relatively small root disk. Was troubleshooting problems in the disk array, doing a lot of booting, and crashing, when suddenly the boot wouldn’t finish. Fortunately the boot wasn’t quiet and I could watch as it tried 20-30 times to start the gdm service.

I change the default to multi-user.target and got it up, and what do you know, the root filesystem is at 100%. Cleaned up some logs and crash logs and it came right up.

Interesting that it doesn’t have a way to at least alert you that is what its problem is.

boot issues, fedora, linux, mail, ubuntu

Clamav and Amavisd

I find the whole clamav subsystem to be fragile. I think this is because it is written as a tool which stands on its own, but I’m only using it as a subsystem hung onto the side of amavisd. So there is some hand-waving and jiggery pokery with the sockets and the permissions to enable the two to communicate, which has to be done manually, and is not properly a part of either subsystem.

I have another article on setting up this subsystem here, which records some of the stuff being done. I think basically, amavisd has to know where the shared socket is, in order to send messages to clamav to check, and they have to agree on the ownership and permissions of the socket and its directory.

Once in a while that stuff gets crosswise, and since I only vaguely understood what was going on, and only did the hand-waving by rote, I got annoyed with it. I’ve grown used to being able to just have things slot in and work, without my having to actually dig in and understand them. The nerve of these people, to expect me to know what is going on in order to make it work! Irony intended. Continue reading Clamav and Amavisd →

backup, boot issues, btrfs, fedora, linux

Adjusting the size of the tarragondata volume

At one point I was getting low on space on tarragondata, so I added an additional physical device to the btrfs filesystem containing tarragondata. 

[root@tarragon backup_scripts]# btrfs fi show
Label: 'tarragon_data' uuid: d6e4b6fc-8745-4e6e-b6b4-8548142b5154
 Total devices 2 FS bytes used 92.04GiB
 devid 1 size 120.00GiB used 120.00GiB path /dev/xvdf1
 devid 2 size 30.00GiB used 30.00GiB path /dev/xvdg

This is fine, but there are a couple of problems. The main one is that I can no longer use the EC2 snapshot capability on tarragondata, which meant that the nightly EC2 snapshot feature I was using had to be deimplemented.

But now I am about to create a new tarragon instance, and it would be really helpful to be able to snapshot tarragondata (Amazon snapshot, not btrfs snapshot) and then create a new Amazon volume with a consistent snapshot for testing. Continue reading Adjusting the size of the tarragondata volume →

boot issues, encryption, fedora, linux, ubuntu

Fail: Resizing /boot on Cinnamon

The current 128GB (119.2GiB) Plextor M.2 disk on cinnamon was created with a very small (256MB/243MiB) /boot partition. Turns out this was a mistake, as it will only hold 3 kernels, so I’m constantly having to go in and remove kernel n-3 to make room.
The remainder of the drive is a single extended partition, containing a 119GiB LUKS container, which contains a 119GiB LVM volume group, which has a root of 107.3GiB and an 11.7GiB swap. The root is only 13% utilized, and while it may grow a little it is in no danger or running out of space. The swap is also far larger than needed. I want to steal a couple of GB from one of them to increase the /boot, but I want to do this while retaining the contents of both the / and the /boot.
Long story short, turns out I wasn’t able to do it. But the journey was a great learning experience for me, so I’ve kept the notes, for future reference.
Continue reading Fail: Resizing /boot on Cinnamon →

boot issues, fedora, linux

Upgrading to Fedora 25

Every upgrade is a little easier, as my restore scripts get better. This was probably the easiest in more than a decade.

This earlier post describes how I put the persistent parts of the system onto btrfs subvolumes, so that on a new system I only need to symlink most of the stuff.

I created a usb stick, containing the resore script, a keyfile for the encrypted drive containing the btrfs subvolumes (the drive is encrypted), and the Nvidia driver installer (which I didn’t need). Halted the system, changed to a different boot ssd drive in the 2.5″ removable drawer, and installed onto it from the downloaded Fedora 25 installer.

This time I made the /boot partition much larger, so that I’m not having to clean up old kernels so often.
Continue reading Upgrading to Fedora 25 →

boot issues, fedora, linux, ubuntu

Useful kernel parameters

This is just a memory aid.

Most kernel parameters are documented at www.kernel.org

Kernel parameters: doc/html/latest/admin-guide/kernel-parameters.txt

  • boot_delay=n    delay n milliseconds between printk. n<10000
  • console=ttyS1 =ttyUSB0
  • debug                     enable kernel debugging
  • module.dnydbg     see Documentation/dynamic-debug-howto.txt
  • enforcing=0/1  initial SELinux enforcing status
  • ignore_loglevel  ALL printk to the console
  • loglevel=0-7  All printk <n will be printed to console
  • module_blacklist=  comma separated
  • S single mode
  • selinux=0/1 disable selinux
  • apparmor=0/1 disable apparmor

Continue reading Useful kernel parameters →

boot issues, fedora, linux, ubuntu

Not starting X Windows

I had some trouble with the graphics drivers, and needed to start the system in runlevel 3. How to do that? Can’t just use the old inittab configuration file and set initdefault to 3 anymore.

I ended up changing the systemd default target. On the running system the /etc/systemd/system/default.target is a symlink to /usr/lib/systemd/system/graphical.target, but if I replace that with a symlink to /usr/lib/systemd/system/multi-user.target, that system will come up to runlevel 3. The easy way to do this is in systemctl

systemctl set-default multi-user-target
systemctl set-default graphical-target

I didn’t do it, but I also found a post recently that said you can simply put a runlevel on the end of the grub command line, i.e. add 3

boot issues, btrfs, encryption, fedora, linux

New disk layout using btrfs

Preparatory to doing an upgrade of Fedora on one of my linux boxes, I decided it was time to revise the disk layouts. Specifically, I finally got around to building a separate partition for /home. After doing a little reading, it looked to me like a good way to do this was to use btrfs. Initially I also made a separate /var, but decided that was both unneeded and error prone, and went another way. The main objective of this work was to reduce the drag in upgrading Fedora.

Root is on a small (64GB) SSD, LUKS encrypted which has /boot and an LVM VG for / and swap. / is ext4. I built a 2GB mdadm raid-1 mirror with LUKS encryption on top of it, and then layered a btrfs file system on that called oreganodata. The root of oreganodata is not typically mounted, but has the following subvolumes automounted at boot time:

  1. /home -> oreganodata/home
  2. /mail->oreganodata/mail,  /var/lib/imap and /var/spool/imap symlink to the /mail/lib and /mail/spool directories.
  3. /var/lib/mysql->oreganodata/maria
  4. /usr/local/bin->oreganodata/bin
  5. /etc/pki/mycerts->oreganodata/certs
  6. /var/webdata->oreganodata/webdata (websites have symlinks to /var/webdata as needed)

Continue reading New disk layout using btrfs →