tvheadend jots

This is a lot of random jots as I set up Tvheadend. As it was a bit of a battle, things may not be consistent below. There are some details below specific to my own environment (e.g. LAN addresses).

Environment: Linux Mint 18.1 running a 4.4.0-53-generic kernel, receiving DVB-T in Haifa, Israel with a HD-901T2 DVB-T2 USB stick.

Installation

This is the backend server for accessing the DVB tuner. Any software that is used to view live TV expects some kind of backend like this.

Add TVheadend repository, following this guide: The following file added as /etc/apt/sources.list.d/tvheadend.list:

deb https://dl.bintray.com/tvheadend/deb xenial stable

and loaded the key:

# apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 379CE192D401AB61
Executing: /tmp/tmp.3BLJZuWTSs/gpg.1.sh --keyserver
hkp://keyserver.ubuntu.com:80
--recv-keys
379CE192D401AB61
gpg: requesting key D401AB61 from hkp server keyserver.ubuntu.com
gpg: key D401AB61: public key "Bintray (by JFrog) <bintray@bintray.com>" imported
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)

and as usual,

# apt-get update

And install

# apt-get install tvheadend xmltv-util

ignoring the lack of verification (living on the edge):

WARNING: The following packages cannot be authenticated!
  tvheadend
Install these packages without verification? [y/N] y

xmltv-util was added since it was suggested when requesting the installation of tvheadend.

As part of the installation of Tvheadend, I was prompted for a username a password for the dedicated tvheadend user. This is the user/password pair for accessing the web interface, and not the Linux user which is generated during the installation (named “hts” was created, with no password assigned, i.e. impossible to login with password).

And then configure the server by connecting to port 9981 with a plain web browser.

Following this guide:

• Went for the DVB Inputs tab, where /dev/dvb/adapter0 was found. Uhhh, there are three entries, two for Panasonic MN88473 (with DVB-C and DVB-T each) and an entry for Realtek RTL2832 (DVB-T). Picked the Realtek DVB-T #0 entry, as the Panasonic adapters can’t supply BER and SNR (and I killed it), as indicated in the debug output when this is attempted. Ticked the “enabled” checkbox. “Initial Scan” and “Idle scan” should be ticked as well. Click Save.
• Went to the Networks Tab, and created a new DVB-T network, named it Idan Plus. Check “Skip Initial scan” (it slows down startup). Set Pre-defined Muxes to Israel: il-All.
• Going back to the DVB Inputs tab, associate the DVB-T entry with the Network just created (drop-down menu for Networks). Click Save.
• Under “Muxes”, monitor the scan status on the 538 MHz mux (for Haifa) should go from PEND to IDLE and the result to “OK”. It takes a couple of minutes or so before anything happens.
• Disable the failed mux (not sure if it matters)
• Go to Services, and find a nice list of them.
• Map channels: On the Channel/EPG tab, click on the “Map Services” button, and accept the defaults.
• Add an access entry for kodi: Set the username to e.g. kodiclient, set a password. Allow all interfaces except admin, set the streaming profile to htsp, and check all functionalities (video recorder to All DVR-rw).
• Add an access entry for no-password users: Set the network prefix to “10.1.1.0/24;10.10.0.0/16;127.0.0.1/32″ with no credential request: Set the username to *, and the password to * (that is, a single “*” char each). Allow all interfaces, set the streaming profile to htsp, and check all functionalities (video recorder to All DVR-rw).

This is what a failed scan looks like (on tvheadend’s debug output):

2017-03-10 11:03:37.000 [   INFO] mpegts: 514MHz in Idan Plus - tuning on Realtek RTL2832 (DVB-T) : DVB-T #0
2017-03-10 11:03:37.000 [   INFO] opentv-ausat: registering mux 514MHz in Idan Plus
2017-03-10 11:03:37.004 [   INFO] subscription: 0002: "scan" subscribing to mux "514MHz", weight: 5, adapter: "Realtek RTL2832 (DVB-T) : DVB-T #0", network: "Idan Plus", service: "Raw PID Subscription"
2017-03-10 11:03:40.055 [WARNING] linuxdvb: Unhandled signal scale: 0
2017-03-10 11:03:40.069 [WARNING] linuxdvb: Unable to provide UNC value.
2017-03-10 11:03:42.000 [   INFO] mpegts: 514MHz in Idan Plus - scan no data, failed
2017-03-10 11:03:42.000 [   INFO] subscription: 0002: "scan" unsubscribing
2017-03-10 11:03:42.002 [   INFO] mpegts: 538MHz in Idan Plus - tuning on Realtek RTL2832 (DVB-T) : DVB-T #0
2017-03-10 11:03:42.002 [   INFO] opentv-ausat: registering mux 538MHz in Idan Plus
2017-03-10 11:03:42.009 [   INFO] subscription: 0004: "scan" subscribing to mux "538MHz", weight: 5, adapter: "Realtek RTL2832 (DVB-T) : DVB-T #0", network: "Idan Plus", service: "Raw PID Subscription"
2017-03-10 11:03:47.000 [   INFO] mpegts: 538MHz in Idan Plus - scan no data, failed
2017-03-10 11:03:47.000 [   INFO] subscription: 0004: "scan" unsubscribing

THIS IS THE SETTING FOR THE OLD Sin Hon TDH 601 / HD809 USB dongle. Just skip.

• Went for the DVB Inputs tab, where /dev/dvb/adapter0 was found. Picked the DVB-T #0 entry (there was a DVB-C entry as well, but this is for cable TV frequencies). Ticked the “enabled” checkbox. “Initial Scan” and “Idle scan” should be ticked as well. Click Save.
• Went to the Networks Tab, and created a new DVB-T network, named it Idan Plus. Uncheck “Skip Initial scan”. Set Pre-defined Muxes to Israel: il-All.
• Going back to the DVB Inputs tab, associate the DVB-T entry with the Network just created (drop-down menu for Networks). Click Save.
• Under “Muxes”, monitor the scan status on the 538 MHz mux (for Haifa) should go from PEND to IDLE and the result to “OK”. It takes a couple of minutes or so before anything happens.
• Disable the failed mux (not sure if it matters)
• Go to Services, and find a nice list of them.
• Map channels: On the Channel/EPG tab, click on the “Map Services” button, and accept the defaults.
• Add an access entry for kodi: Set the username to e.g. kodiclient, set a password. Allow all interfaces except admin, set the streaming profile to htsp, and check all functionalities (video recorder to All DVR-rw).

# service tvheadend stop

or, alternatively:

# systemctl stop tvheadend.service

(and start for turning it on). After stopping, running the service from console, as root:

# /usr/bin/tvheadend -f -u hts -g video

This launches a deamon (-f means “fork as daemon”), after changing the user ID and group as given by the flags.

To run it in the foreground, with debug messages to console,

# /usr/bin/tvheadend --trace --debug -d -u hts -g video

Which outputs during a successful scan of Idan Plus in Haifa:

2017-02-06 14:24:58.000 [   INFO] mpegts: 538MHz in Idan Plus - tuning on Silicon Labs Si2168 : DVB-T #0
2017-02-06 14:24:58.001 [   INFO] opentv-ausat: registering mux 538MHz in Idan Plus
2017-02-06 14:24:58.007 [   INFO] subscription: 0007: "scan" subscribing to mux "538MHz", weight: 5, adapter: "Silicon Labs Si2168 : DVB-T #0", network: "Idan Plus", service: "Raw PID Subscription"
2017-02-06 14:24:59.028 [WARNING] eit: invalid checksum (len 489, errors 1)
2017-02-06 14:24:59.698 [WARNING] mpegts: mux 514MHz in Idan Plus changed from DVBT freq 514000000 bw 8MHz cons QAM/16 hier NONE code_rate 2/3:NONE guard 1/4 trans 8k plp_id -1
2017-02-06 14:24:59.698 [WARNING] mpegts: mux 514MHz in Idan Plus changed to   DVBT freq 514000000 bw 8MHz cons QAM/16 hier NONE code_rate 2/3:2/3 guard 1/4 trans 8k plp_id -1
2017-02-06 14:24:59.700 [WARNING] mpegts: mux 538MHz in Idan Plus changed from DVBT freq 538000000 bw 8MHz cons QAM/16 hier NONE code_rate 2/3:NONE guard 1/4 trans 8k plp_id -1
2017-02-06 14:24:59.700 [WARNING] mpegts: mux 538MHz in Idan Plus changed to   DVBT freq 538000000 bw 8MHz cons QAM/16 hier NONE code_rate 2/3:2/3 guard 1/4 trans 8k plp_id -1
2017-02-06 14:25:03.000 [   INFO] mpegts: 538MHz in Idan Plus - scan complete
2017-02-06 14:25:03.000 [   INFO] subscription: 0007: "scan" unsubscribing
2017-02-06 14:25:03.004 [   INFO] mpegts: disabling service Idan Plus/538MHz/{PMT:0} [sid 0027/39] (missing in PAT/SDT)
2017-02-06 14:25:03.004 [   INFO] mpegts: disabling service Idan Plus/538MHz/{PMT:0} [sid 0026/38] (missing in PAT/SDT)
2017-02-06 14:25:03.004 [   INFO] mpegts: disabling service Idan Plus/538MHz/{PMT:0} [sid 0025/37] (missing in PAT/SDT)
2017-02-06 14:25:03.004 [   INFO] mpegts: disabling service Idan Plus/538MHz/{PMT:0} [sid 0023/35] (missing in PAT/SDT)
2017-02-06 14:25:03.004 [   INFO] mpegts: disabling service Idan Plus/538MHz/{PMT:0} [sid 0022/34] (missing in PAT/SDT)
2017-02-06 14:25:03.004 [   INFO] mpegts: disabling service Idan Plus/538MHz/{PMT:0} [sid 0021/33] (missing in PAT/SDT)
2017-02-06 14:25:03.004 [   INFO] mpegts: disabling service Idan Plus/538MHz/{PMT:0} [sid 0020/32] (missing in PAT/SDT)
2017-02-06 14:25:03.004 [   INFO] mpegts: disabling service Idan Plus/538MHz/{PMT:0} [sid 001F/31] (missing in PAT/SDT)
2017-02-06 14:25:03.005 [   INFO] mpegts: disabling service Idan Plus/538MHz/{PMT:0} [sid 0014/20] (missing in PAT/SDT)
2017-02-06 14:25:03.005 [   INFO] mpegts: disabling service Idan Plus/538MHz/{PMT:0} [sid 0013/19] (missing in PAT/SDT)
2017-02-06 14:25:03.005 [   INFO] mpegts: disabling service Idan Plus/538MHz/{PMT:0} [sid 0012/18] (missing in PAT/SDT)
2017-02-06 14:25:03.005 [   INFO] mpegts: disabling service Idan Plus/538MHz/{PMT:0} [sid 0011/17] (missing in PAT/SDT)
2017-02-06 14:25:03.005 [   INFO] mpegts: disabling service Idan Plus/538MHz/{PMT:0} [sid 0010/16] (missing in PAT/SDT)
2017-02-06 14:25:03.005 [   INFO] mpegts: disabling service Idan Plus/538MHz/{PMT:0} [sid 000F/15] (missing in PAT/SDT)
2017-02-06 14:25:03.005 [   INFO] mpegts: disabling service Idan Plus/538MHz/{PMT:0} [sid 000E/14] (missing in PAT/SDT)
2017-02-06 14:25:03.005 [   INFO] mpegts: disabling service Idan Plus/538MHz/{PMT:0} [sid 000D/13] (missing in PAT/SDT)
2017-02-06 14:25:03.005 [   INFO] mpegts: disabling service Idan Plus/538MHz/{PMT:0} [sid 000C/12] (missing in PAT/SDT)
2017-02-06 14:25:03.005 [   INFO] mpegts: disabling service Idan Plus/538MHz/{PMT:0} [sid 000B/11] (missing in PAT/SDT)
2017-02-06 14:25:03.005 [   INFO] mpegts: disabling service Idan Plus/538MHz/{PMT:0} [sid 000A/10] (missing in PAT/SDT)
2017-02-06 14:25:03.005 [   INFO] mpegts: 514MHz in Idan Plus - tuning on Silicon Labs Si2168 : DVB-T #0
2017-02-06 14:25:03.005 [   INFO] opentv-ausat: registering mux 514MHz in Idan Plus
2017-02-06 14:25:03.012 [   INFO] subscription: 0009: "scan" subscribing to mux "514MHz", weight: 5, adapter: "Silicon Labs Si2168 : DVB-T #0", network: "Idan Plus", service: "Raw PID Subscription"
2017-02-06 14:25:08.000 [   INFO] mpegts: 514MHz in Idan Plus - scan no data, failed
2017-02-06 14:25:08.000 [   INFO] subscription: 0009: "scan" unsubscribing

An .m3u8 playlist can be obtained with http://localhost:9981/playlist (or was it http://localhost:9981/playlist/channels?). Anyhow, if Tvheadend doesn’t like the URL, it goes “1 Unknown code”).

The playlist can be used with e.g. VLC connect to to play channels, possible several different channels from the same DVB adapter, if they share the same broadcast frequency ( = same mux in Tvheadend terminology).

Where tvheadend keeps its configuration

Short answer: /home/hts/.hts/tvheadend/. Don’t get scared by all those SHA1-named files. It’s not a git repo. These are plain, readable, text files which are easy to understand.

In particular, the DVR timers are stored in /home/hts/.hts/tvheadend/dvr/timerec/, one file for each timer task. Really simple. The file names don’t change when the configuration changes, so I guess these files can be tracked with version control.

Source digging

Grabbing the repository:

$ git clone https://github.com/tvheadend/tvheadend

Using “git blame” on src/webui/webui.c, it’s evident that Jaroslav Kysela added the /xmltv feature on Oct. 28th 2015 (as he said himself), and there are functional changes running as far as commit ID a87689803093e303ac4d458d35ac92c1bfc41f96 of Nov 11th 2015, which does XML escaping.

This post shows some JSON API as well.

Upgrade tvheadend

Since Kodi’s HTSP Tvheadend client worked horribly, I went for IPTV simple client from one of those IPTV providers in the UK. But in order to get the EPG feature working, I need a Tvheadend version that exports an XMLTV file through http. Ironically, the upgrade improved things with the HTSP client significantly, but there was still a need to double-click a channel twice to zap.

Change /etc/apt/sources.list.d/tvheadend.list to

deb https://dl.bintray.com/tvheadend/deb xenial unstable

and go

# apt-get update
# apt-get install tvheadend

Ignore warning that the package can’t be authenticated, and go ahead installing (upgrading, in my case). The new revision was 4.1-2405~geb495a0~xenial, according to “About” in the Web UI, which is git commit eb495a0984245c37a4474c7d3984acdd34edc071 from Jan 2nd, 2017. So not the absolutely latest, but definitely covers the /xmltv feature.

Files that were updated: /usr/bin/tvheadend, files under /usr/share/doc/tvheadend and a lot under /usr/share/tvheadend, and the man page (plus minus, this is based upon files an overlay filesystem).

Alternatively, this is the manual installation of the same. Look for tvheadend_4.1-2405~geb495a0~xenial_amd64.deb (it’s on my hard disk as well, under /storage/tv-mediacenter-stuff). Remove old version and install new:

# apt-get remove tvheadend
# dpkg -i tvheadend_4.1-2405~geb495a0~xenial_amd64.deb

This revision works great.

Some useful access URLs

Assuming that the server is on localhost:

• Obtain m3U playlist of all live TV channels: http://localhost:9981/playlist
• Obtain XMLTV file for EPG: http://localhost:9981/xmltv (or better, http://localhost:9981/xmltv/channels)
• Watch TV channels directly from a browser (rather crude look, but works): http://localhost:9981/tv.html
• Recorder log: http://localhost:9981/simple.html

And as mentioned above, there as several URLs for JSON formatted info on this post. I suppose I’ll add more as I come across these…