Category Archives: kodi

fedora, kodi, linux, mac, ubuntu, xbmc

Correcting Corrupted Kodi Database

I’ve had several cases where one of the Kodi sqlite3 databases becomes corrupted on Coriander. It has happened to Textures, Addons and EPG at different times.  The error that manifests in the Kodi log is The Database Disk Image is Malformed, and once it happens the log is full of this and transactions in the affected database don’t take place.

Did some reading, and discovered that usually you can dump the database, and the dump will be ok.  Try this:

echo '.dump' | sqlite3 fred | sqlite3 new_fred
mv fred bad_fred
mv new_fred fred
asterisk, fedora, kodi, linux, mac, ubuntu, xbmc

Caller Id in Kodi – Redux

I posted in this post about how I arranged to get caller-Id to show up in Kodi. At that time I was still using POTS, but since then I have switched to VoIP and set up Asterisk.

I did find another plugin that would connect to Asterisk and obtain information about incoming calls. But until now it was only half way working. I finally got around to paying attention to it. It was originally provided by “hmronline” and I found it at code.google.com. It was last worked on with Dharma, and I had to do some things to get it ready for Jarvis.

The source code is now in the tarragon svn repository, under the name KodiAsterisk. On coriander, where the production Kodi lives, there is a checked out directory under /Users/wmb called kodiasterisk. For Kodi to install it it has to be zipped, so the sequence is to svn update the directory, then zip the directory as script.service.dee-kodi-asterisk.zip. Then install it within Kodi.

It attaches to asterisk on cinnamon using the Asterisk manager interface.  Authentication on the manager interface uses files in /etc/asterisk/manager.d which has one file per authorized user, containing credentials.

The code has features to

  • give a notification of the caller id
  • put up a picture of the caller
  • pause a video if one is playing
  • redirect the call (within asterisk) if a video is playing
  • indicate whether there are voice mails

I have the notification running, the picture coming up, and the video being paused. I don’t use the redirect feature. Also, since my voice mail is not on Asterisk the features of voice mail aren’t being used.

I did put pictures into a folder within /Storage/Pictures/KodiPictures. The pictures here are named with the phone number, and contain a snapshot of the caller, in jpeg. If there is no picture in the folder for a particular number nothing is put up, but the notification and pausing still occur.

 

apache, encryption, fedora, kodi, linux, mysql, ubuntu, website building, xbmc

Making MySQL serve UTF8 correctly

If the MySQL server decides that its default environment is UTF8, and that its client actually wants Latin1, it will translate the return values.

I’ve never before had to be careful of the distinction. Perhaps once in a blue moon I would have a record with a “real” quotation mark or a character with an accent, but if it didn’t work correctly, it was never much of a bother.

Once it became important, I had to understand what was happening. I have a table that has filenames in it, and some of those filenames contain characters (a acute, e grave, o umlaut, etc). The actual files on disk have the names encoded in utf8. The records in the database are also recorded in utf8. But the records were being translated by mysql from utf8 to latin1 as they came in. So “Mamá” was recorded on disk as Mam\xc3\xa1 in the directory, and in the database, but when I got the row into memory, the filename field said Mam\xe1. The difference between latin1 and utf8 for this purpose, is that all these many “western/latin special characters” were actually mapped in latin1 to values within the 256 characters available with 1 byte. So the first 128 in the latin1 codespace were ordinary ascii, and the high order 128 had as many of the western/latin diacriticals as possible crammed in there. And in latin1, e9 is a-acute.

But on the web these days, utf8 is much preferred. Latin1 is ok if all you want is the carefully selected subset of 128 characters that can be shoehorned into the high end of the code-point space. But utf8 is a far more general solution. Using a multibyte sequence to represent over a million characters and special symbols.

Turns out mysql has a bunch of variables to control character set and collating sequence. With phpmyadmin, one can look at database->Variables and see characters_set_database, character_set_filesystem, character_set_result, character_set_server, character_set_system, and a bunch more. Or in mysql client one can show varaiables like ‘%character_set%’;

My problem was that the server had come up believing that some of these were set to utf8 and some were set to latin1. I haven’t tried to figure out the logic of how it figures out its default – I don’t want it to default, I want to tell it what I want. So the solution was to add the directive: “character-set-server=utf8” to the mysql configuration file (on cinnamon it was /etc/mysql/mysql.conf.d/mysqld.conf. After restarting all of the relevant character_set_xxx variables come up as utf8.

Update: 8/9/17

I used these changes also on oregano and tarragon, but it results in a different problem for me on blogforacure data. The blogforacure database, built a long time ago, has lots of tables in latin1. There are not a lot of non-ascii characters, but there are a few. One frequent source is people typing double space after a period, which the ckeditor tries to preserve by creating a non-breaking space, which is hex A0 in latin 1. When the site reads this back, if mysql is told that the database is actually utf8, then it displays this as Â. So if I see a bunch of A circumflex in the output, it means I actually have latin 1 characters in the database, which I am interpreting as if they were utf-8.

Removing the specification of character-set-server=utf8 causes the negotiation to give the right result, and the latin1 non-breaking space appears correctly in the output.

kodi, video, xbmc

Saving YouTube Video

Trying to collect the old episodes of NYPD Blue, and there were some I couldn’t find on usenet. Turns out that the DVDs past season 4 were never produced. But there are old episodes, mostly captured from VHS that are out on Youtube.
After a little research, there is a cute way to capture the video files from youtube.
Click the episode on youtube to start it, and then capture the url in the browser location bar.
Now open VLC, select to play a network stream, and give it that url. It will start to play.
At this point, one can just tell VLC to convert and save, but that takes a long time. An easier and faster technique, as long as the youtube webm format is ok:
In vlc, with the stream playing, open tools->codec information and copy the link that is in the “Location” text box at the bottom.
Paste that link into a browser and go, and now the video plays directly from the Youtube servers.
Now you can just right click on the browser window where it is playing, and save video as.

kodi, mac, xbmc

Kodi Keymap Files

Kodi control with the Harmony 1100

I have a Harmony 1100 Remote Control which I bought when I had nearly a dozen different devices that I wanted to control (cable boxes and several tivos, as well as tv, receiver, dvd player, vcr, etc). I don’t have most of this stuff anymore, since I dumped the cable and started using Kodi. So I could probably live without the Harmony Remote, but I paid a lot of money for it, so I want it to work correctly. A big part of this project, therefore, was figuring out how to get that remote to control Kodi properly. I also wanted to improve the handling of keyboards, and understand better how I could use the configuration capabilities in Kodi to get better control.

The Harmony remote software for the Harmony 1100 permits assigning to each hard button or touchscreen button, a “command”, and a “device” to which the command applies.

When a “device” is set up for control by the Harmony remote, one selects what type of device it is from the Harmony database. The database has various device types (tv, dvd player etc), including one called Media Center PC. Within each device type there are a number of specific “devices” with manufacturer (Sony, Samsung) and model number. For the device type Media Center PC, there is a manufacturer “Plex” and model “Plex Player”. There are other “manufacturers” in the database that look promising (like “kodi”), but they turn out not to have anything useful in them. The only one that has been useful for me is the one called “Plex”.

The database for a device, in this case “Plex”, has a set of what it calls “commands”, which to the user look like “Play”, “Pause”, “F3”, “Channel Up”, etc. When configuring the buttons on the remote, one can choose from that offered set of commands, and no others. For other devices with manufacturer supplied remote controls, one can create ones own “commands” by having the harmony app learn from the remote, but since there is no manufacturer supplied physical remote control for Kodi, the commands in the database are the only ones which can be assigned. One has to make do with the set of commands which are provided.
Continue reading Kodi Keymap Files →

kodi, xbmc

Setting up Documentaries menu in Kodi

The first thing I did was to actually separate the documentaries from the tv shows on the disk. I created a separate directory within my /emedia folder called documentaries, alongside the tvshows folder, and then moved all the documentaries into it. Then I had to adjust the nfs exporting stuff on cinnamon to mount a share named documentaries and export it, so that the other computers could see it. Then in Kodi, first I cleaned the library (to remove all the previous pointers to the documentaries which were formerly inside the /tvshows folder), then I went to Videos and created a new source called documentaries, pointing to it. When it asks for the type of content, and I tell it they are tv shows, it scrapes them all, as tvshows, and so now the library is back where it was – with all the tv series and documentaries mixed up together in the library, under the general heading of “tv shows”. What I want now is to differentiate between them – those that are from /emedia/documentaries/ are documentaries, while those that are from /emedia/tvshows/ are the tv series.

I experimented with the video node editor, and with manually creating or modifying nodes by copying the system library into the userdata/library folder. I created a documentaries.xml, and had it working. But in the end there was a much easier way using just smart playlists.

I had to use a different skin, because Confluence doesn’t have the required features. I used Aeon Nox, but I saw some stuff on the web suggesting one might also use Aeon MQ – I thought it was MQ4, but I look now and see that there is also an MQ5 and MQ6. I may experiment with them, but right now I am using Aeon Nox.

First I created two “Smart Playlists” (Main Menu->Videos->Playlists->New Smart Playlist). I named them “TV Series” and “Documentaries”. In the smart playlist I did “add new rule”. A rule is in the form <field> <operator> <value>. They have a drop down for the first two. The first one starts initialized at title, but I cycled through to find ‘path’, then set the <operator> to ‘contains’ (for Documentaries) and ‘does not contain’ (for TV Series) and set the value to ‘/documentaries/’. I also set the “order” field for TV Series to be last added, but left the documentaries as alphabetical by title.

Continue reading Setting up Documentaries menu in Kodi →

asterisk, kodi, xbmc

Getting Caller-ID Notifications in Kodi

Had a notion to try to get incoming calls to show caller id on the Kodi screen. I looked around and found an add-on, called service.script.call-notifications (at github) by “mablae”, which sounds like it does what I want. It wants to get call data from one of three types of sources, one of which is named NCID (network caller ID). Looked into that and it sounds like just the thing.
NCID styles itself a client-server-gateway caller id system, and is somewhat like the callblocker thing which I built a few years back for the Raspberry Pi. But this one has the added feature that it will put caller id stuff out on the network, where users, like the Kodi add-on above, can use it. It will also do blacklisting, which was the purpose of the thing I built. And, like mine, it will run on the Raspberry Pi.
NCID says it wants to run on the Raspbian OS (a debian spin specifically for the pi). It can be compiled for other linux distros, but I thought I might as well use Raspbian. I had a spare 8GB SD Card, so I downloaded Raspbian, burned the card, brought it up on the Pi and spent a few hours getting it set up right (got rid of user “pi”, installed user “dee”, got the certificates set up right so I can ssh to it, and it can ssh to others,etc.), and then I installed the NCID code.
Installation of the code was a little unusual. It isn’t in any of the apt sources I have. I downloaded the deb files they had, and then used a program called gdebi to install them. gdebi seems to be a debian thing. I haven’t used gdebi before. I had to install it (with apt-get). Apparently it is like dpkg, but it handles dependencies, which dpkg doesn’t. Usage is just gdebi <deb pkg name>.
Had a little trouble with the modem being recognized and mapped correctly, but after a little futzing around it got added by udev as ttyACM0, and the daemon ncidd was then able to find it. A little program called ncid which is a little graphical client was able to connect, and able to detect calls. I set up ncidd to start automatically. It has a config file /etc/ncid/ncidd.conf in which I specify the device (ttyACM0) and how to hang up on blacklisted entries (I chose 2: play a fax tone first, then hangup). I left other things at default.
Continue reading Getting Caller-ID Notifications in Kodi →

kodi, linux, mac, ubuntu, video, xbmc

Notes on setup of HDHomeRun, tvheadend, kodi live tv

HDHomeRun provides a source of tv in htsp format. They provide an app for windows/linux/mac which enables watching the tv stream directly, and changing channels. They also provide a Kodi Add-on which allows watching the streamed material directly from there. However, this is just watching, and doesn’t enable the guide, PVR etc.

To use the built in features in Kodi for “live tv”, you have to have another piece of software, which Kodi calls the “backend”. There are apparently different backends supporting different hardware, but one of the backends is called “tvheadend”, and it supports HDHomeRun, and is supported by Kodi.

The tvheadend software has to be installed. apt-cache search tvheadend shows:
tvheadend – Tvheadend
tvheadend-dbg – Debug symbols for Tvheadend
kodi-pvr-tvheadend-hts – Kodi PVR Addon TvHeadend Hts – PVR API:1.9.2
kodi-pvr-hts – TVHeadEnd PVR for Kodi
kodi-pvr-hts-dbg – debugging symbols for TVHeadEnd PVR for Kodi

The “kodi-pvr” bits are kodi add-ons that have to be added to kodi (in linux only) in order to provide the api between kodi and the backend. Kodi for mac and windows has the pvr bits included, but they have to be added in linux.After the apt install, add-ons->my add-ons->PVR clients,  select TVHeadend HTSP Client and configure it, then Activate it.

Since the kodi I watch is on coriander (the mac mini), the pvr stuff is already installed with kodi. I only needed to install the tvheadend piece somewhere, and I put it on cinnamon where the large file media array is, so that PVR recorded material can be stored there too.
Continue reading Notes on setup of HDHomeRun, tvheadend, kodi live tv →