linux, mysql, ubuntu

MYSQL on Ubuntu 15.10

I haven’t researched whether this has changed, in 15.10, or whether it has been this way since ubuntu switched to systemd, which is probably the case.

Under systemd, ubuntu no longer uses the /etc/init.d/mysql script, but instead uses a systemd unit in /lib/systemd/system/mysql.service which invokes /usr/bin/mysqld_safe to start and stop mysqld.

I have had a lot of trouble with this, and had to do a lot of debugging to figure out what is going on. Probably I would not have had trouble if I were not trying to port over a running mysql installation manually, i.e. if I just installed mysql-server and proceeded to create new databases, new entries in mysql etc.

One issue is that a mysql install creates a file in /etc/mysql called debian.cnf which contains a user/password for user debian-sys-maint with a generated password, and this is put into the mysql users table, to enable various operations to be performed by mysqladmin using these credentials.

The first problem was that when I copied over the mysql table from the previous installation, I was copying in the old password for debian-sys-main, which didn’t match the debian.cnf file which was installed when I did the apt-get install mysql-server. So I had to read the debian.cnf file, extract the password and change the password in the mysql table.
Continue reading MYSQL on Ubuntu 15.10 →

backup, cloud, linux, ubuntu

S3cmd on ubuntu 15.04

After installing ubuntu 15.04 my backups to S3 stopped working.

I tried running them manually to see what was happening and I got errors – some goofy stuff about the url I was using net.wmbuck.backups….s3.amazonaws.com not being part of *s3.amazonaws.com. When I searched the net I found that there was a change in python 2.7.9 having to do with evaluating certificates, and some conflict with the wildcard cert being used by Amazon S3, with the result that there is an error which occurs whenever an S3 bucket happens to contain the “.” character in its title.

My buckets are all named net.wmbuck.x so I am vulnerable to this error.

There is a fix for this in S3cmd version 1.6.0 but the latest ubuntu as of this writing has only S3cmd 1.5.x and attempting to upgrade using apt-get doesn’t get anything new.

I did an apt-get remove of s3cmd, and then downloaded a tarball, and installed it into /usr/local/bin.

Ubuntu 15.10 will be coming out next month, and when I get around to installing that perhaps the version of s3cmd will have the fix.

top of page

kodi, xbmc

Setting up Documentaries menu in Kodi

The first thing I did was to actually separate the documentaries from the tv shows on the disk. I created a separate directory within my /emedia folder called documentaries, alongside the tvshows folder, and then moved all the documentaries into it. Then I had to adjust the nfs exporting stuff on cinnamon to mount a share named documentaries and export it, so that the other computers could see it. Then in Kodi, first I cleaned the library (to remove all the previous pointers to the documentaries which were formerly inside the /tvshows folder), then I went to Videos and created a new source called documentaries, pointing to it. When it asks for the type of content, and I tell it they are tv shows, it scrapes them all, as tvshows, and so now the library is back where it was – with all the tv series and documentaries mixed up together in the library, under the general heading of “tv shows”. What I want now is to differentiate between them – those that are from /emedia/documentaries/ are documentaries, while those that are from /emedia/tvshows/ are the tv series.

I experimented with the video node editor, and with manually creating or modifying nodes by copying the system library into the userdata/library folder. I created a documentaries.xml, and had it working. But in the end there was a much easier way using just smart playlists.

I had to use a different skin, because Confluence doesn’t have the required features. I used Aeon Nox, but I saw some stuff on the web suggesting one might also use Aeon MQ – I thought it was MQ4, but I look now and see that there is also an MQ5 and MQ6. I may experiment with them, but right now I am using Aeon Nox.

First I created two “Smart Playlists” (Main Menu->Videos->Playlists->New Smart Playlist). I named them “TV Series” and “Documentaries”. In the smart playlist I did “add new rule”. A rule is in the form <field> <operator> <value>. They have a drop down for the first two. The first one starts initialized at title, but I cycled through to find ‘path’, then set the <operator> to ‘contains’ (for Documentaries) and ‘does not contain’ (for TV Series) and set the value to ‘/documentaries/’. I also set the “order” field for TV Series to be last added, but left the documentaries as alphabetical by title.

Continue reading Setting up Documentaries menu in Kodi →

asterisk, kodi, xbmc

Getting Caller-ID Notifications in Kodi

Had a notion to try to get incoming calls to show caller id on the Kodi screen. I looked around and found an add-on, called service.script.call-notifications (at github) by “mablae”, which sounds like it does what I want. It wants to get call data from one of three types of sources, one of which is named NCID (network caller ID). Looked into that and it sounds like just the thing.
NCID styles itself a client-server-gateway caller id system, and is somewhat like the callblocker thing which I built a few years back for the Raspberry Pi. But this one has the added feature that it will put caller id stuff out on the network, where users, like the Kodi add-on above, can use it. It will also do blacklisting, which was the purpose of the thing I built. And, like mine, it will run on the Raspberry Pi.
NCID says it wants to run on the Raspbian OS (a debian spin specifically for the pi). It can be compiled for other linux distros, but I thought I might as well use Raspbian. I had a spare 8GB SD Card, so I downloaded Raspbian, burned the card, brought it up on the Pi and spent a few hours getting it set up right (got rid of user “pi”, installed user “dee”, got the certificates set up right so I can ssh to it, and it can ssh to others,etc.), and then I installed the NCID code.
Installation of the code was a little unusual. It isn’t in any of the apt sources I have. I downloaded the deb files they had, and then used a program called gdebi to install them. gdebi seems to be a debian thing. I haven’t used gdebi before. I had to install it (with apt-get). Apparently it is like dpkg, but it handles dependencies, which dpkg doesn’t. Usage is just gdebi <deb pkg name>.
Had a little trouble with the modem being recognized and mapped correctly, but after a little futzing around it got added by udev as ttyACM0, and the daemon ncidd was then able to find it. A little program called ncid which is a little graphical client was able to connect, and able to detect calls. I set up ncidd to start automatically. It has a config file /etc/ncid/ncidd.conf in which I specify the device (ttyACM0) and how to hang up on blacklisted entries (I chose 2: play a fax tone first, then hangup). I left other things at default.
Continue reading Getting Caller-ID Notifications in Kodi →

apache, website building

Subversion ‘/svn/!svn/me’: 404 Not Found

I’m almost embarrassed to talk about this one, but I had a hard time figuring it out, so perhaps somebody could possibly be helped by my screwup.

In the process of tracking down some other problem on a client’s website, I was looking at apache status displays, and noted that I’d never set this site up with a “default” virtual host. The first virtual host described is the default one, which receives any inputs that don’t match any other virtual host – which automatically includes, for example, requests directed at an ip address. So I implemented a default virtual host. That isn’t important.

What is important is that in doing so I allowed the other virtual hosts to fall into their default order, which is to say directory order, which is to say alphabetical order.

kodi, linux, mac, ubuntu, video, xbmc

Notes on setup of HDHomeRun, tvheadend, kodi live tv

HDHomeRun provides a source of tv in htsp format. They provide an app for windows/linux/mac which enables watching the tv stream directly, and changing channels. They also provide a Kodi Add-on which allows watching the streamed material directly from there. However, this is just watching, and doesn’t enable the guide, PVR etc.

To use the built in features in Kodi for “live tv”, you have to have another piece of software, which Kodi calls the “backend”. There are apparently different backends supporting different hardware, but one of the backends is called “tvheadend”, and it supports HDHomeRun, and is supported by Kodi.

The tvheadend software has to be installed. apt-cache search tvheadend shows:
tvheadend – Tvheadend
tvheadend-dbg – Debug symbols for Tvheadend
kodi-pvr-tvheadend-hts – Kodi PVR Addon TvHeadend Hts – PVR API:1.9.2
kodi-pvr-hts – TVHeadEnd PVR for Kodi
kodi-pvr-hts-dbg – debugging symbols for TVHeadEnd PVR for Kodi

The “kodi-pvr” bits are kodi add-ons that have to be added to kodi (in linux only) in order to provide the api between kodi and the backend. Kodi for mac and windows has the pvr bits included, but they have to be added in linux.After the apt install, add-ons->my add-ons->PVR clients,  select TVHeadend HTSP Client and configure it, then Activate it.

Since the kodi I watch is on coriander (the mac mini), the pvr stuff is already installed with kodi. I only needed to install the tvheadend piece somewhere, and I put it on cinnamon where the large file media array is, so that PVR recorded material can be stored there too.
Continue reading Notes on setup of HDHomeRun, tvheadend, kodi live tv →

fedora, linux

Setting up amavisd and clamav

I set up a new server instance on Amazon, tarragon_3, which is Fedora 22. I’ve had a lot of trouble over the years with clamav and amavisd, mostly with the nonsense about having to mess about with the user under which clamav runs, allowing a shared group, location of the socket, ability to create logs. I’ve never had it operate satifsactorily for a long time, and never looked into why. I resolved to look closer this time.

I installed amavisd-new clamav clamav-server-systemd clamav-scanner clamav-update. I am not sure but I think:
clamav is the basic package, and has the engine, and perhaps libraries;
clamav-server-systemd is the machinery to start the clam daemon under systemd;
clamav-scanner I expect has clamscan and clamdscan for command line scanning;
clamav-update is probably freshclam, the stuff for getting updates.

Initially, I was making the same sort of blind changes I’d made in the past, but I read around a little. It seems that some of this has been changed, so I’m trying a new approach. I’m somewhat following instructions in http://www.server-world.info/en/note?os=Fedora_21&p=mail&f=6. Part of the reason for trusting those instructions a little is that they refer to new things which are in the distro, for example, the /etc/tmpfiles.d which I haven’t seen before now. So I think the instructions are relatively recent. On the other hand, the instructions called for modifying /usr/lib/systemd/system/clamd@.service, which is simply wrong.
Continue reading Setting up amavisd and clamav →

mac, xbmc

XBMC Mac Mini Stuttering

During a marathon 12 day stint I upgraded the Raid 5 array to Raid 6. It took almost 12 days to reshape the array. Awful.

Almost immediately thereafter, playback on XBMC became unwatchable, jerking and stuttering. For 2 nights I have been able to watch nothing. Is this coincidence, or is this problem owing to the new Raid setup?

Well, it wasn’t the raid array, proving once again that coincidences actually do happen. I moved a movie to the local hard drive, and played it there – the raid array not involved. And it still has the issue.

After some diagnostics and some reading, I did two things together. I reset the SMC, and I cleaned the air intakes of dust. I should have done them separately, but I didn’t. I powered down, unplugged everything, took off the bottom, blew out the dust with a can of air, plugged it all back in. Having unplugged I have reset the SMC. Some reading suggests that the need to do SMC resets may be a harbinger of impending power supply failure. We will see. For now it works.

——————–

It worked for half a day, then began to fail again. In the end, the way I fixed this had to do with Boinc. I’ve always run Boinc, and never have had any problem with competition for resources. But this time, after a lot of attempts at a lot of different things, it turned out that if I stop Boinc the movies play, if I run Boinc they don’t. I changed the parameters on the mac mini so Boinc will only compete if the cpus are idle and have been idle for 3 minutes. Problem solved.

fedora, linux, ubuntu

Cyrus-Imap Administration

Every time I have to mess with cyrus-imap mailboxes I spend a half hour trying to figure out how to get cyradm to run. While I have by no means figured it all out, I do have one piece of lore worthy of being written down.

My imap server forbids plaintext logins unless they are within a TLS session, so /etc/imapd.conf has the setting allowplaintext: 0

But, cyradm uses imap authentication (witness all the failed attempts to get cyradm to authenticate putting entries in the /var/log/secure log using pam_unix imap:auth). The problem of course is that cyradm doesn’t have a tls session, so allowplaintext rejecting the plaintext password.

Reset /etc/imapd.conf to allowplaintext:1 temporarily, systemctl restart cyrus-imapd, and then, as root,  cyradm tarragon. Make all the mailboxes you want. Then reverse and turn plaintext back off.

cloud, encryption, fedora, linux, ubuntu, website building

Managing passwords on this server

This blog is running on my wmbuck.net server, tarragon, in the Amazon cloud. This server, in addition to hosting this blog, hosts about 20-25 websites (for friends, most of them very low traffic), including my own. It also operates mail for myself and a few others, and provides some other services.
One of the weaknesses has been that most of the people who use the server aren’t really very unix literate, and they don’t really WANT to be. Perhaps they want a website, or they want to have a good place to manage their mail. But in general, the last thing they want is to learn how to ssh into the server to change their password.
So, for most of them, they just use whatever password I set up for them.
One of my friends, who just began using mail on the server, was surprised that it was not convenient to change his password. That spurred me to address the long standing problem. How to let people manage their password for access to services.
The blog now has a new menu on the left, for access to the backend, and for linking to the reset-password screen. There is also a reset password link on the login page https://wmbuck.net/index/login.
The same password is used for all the wmbuck.net stuff: the password for access to mail, the password to get access to protected websites in apache, and the password for logging in to the wmbuck.net backend website.
Continue reading Managing passwords on this server →